[GitHub] rabbah closed pull request #129: enable routemgmt and catalog installation Jobs

[GitBox <gi...@apache.org>]

rabbah closed pull request #129: enable routemgmt and catalog installation Jobs
URL: https://github.com/apache/incubator-openwhisk-deploy-kube/pull/129
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/README.md b/README.md
index c182012..ddf2c80 100644
--- a/README.md
+++ b/README.md
@@ -38,55 +38,35 @@ You can also provision a Kubernetes cluster from a cloud provider, subject to th
 
 # Configuring OpenWhisk
 
-## Initial Configuration
+## Initial Cluster Configuration
 
 * Follow the steps for initial [Cluster Setup](kubernetes/cluster-setup/README.md)
 
-## Deploy Components
+## Configure or Deploy CouchDB
+
+Do one of the following:
+* For development and testing purposes, this repo includes a configuration
+  for deploying a [simple non-persistent CouchDB instance](kubernetes/couchdb/README.md)
+  within the Kubernetes cluster.
+* For a production level CouchDB instance, take a look at the main
+  OpenWhisk [documentation for configuring CouchDB](https://github.com/apache/incubator-openwhisk/blob/master/tools/db/README.md).
+
+## Deploy Remaining Components
 
 To deploy OpenWhisk on Kubernetes, you must deploy its components in
-the proper order. Detailed instructions and the supporting configuration
-files can be found in the kubernetes directory tree. You will need to
-follow the instructions for each step in order.
-
-* Configure or deploy CouchDB.
-    * For development and testing purposes, this repo includes a configuration
-      for deploying a [simple non-persistent CouchDB instance](kubernetes/couchdb/README.md)
-      within the Kubernetes cluster.
-    * For a production level CouchDB instance, take a look at the main
-      OpenWhisk [documentation for configuring CouchDB](https://github.com/apache/incubator-openwhisk/blob/master/tools/db/README.md).
+an order that respects their dependencies.  Detailed instructions and
+the supporting configuration files can be found in the kubernetes
+directory tree. Follow the instructions for each step in order.
+
 * Deploy [ApiGateway](kubernetes/apigateway/README.md)
 * Deploy [Zookeeper](kubernetes/zookeeper/README.md)
 * Deploy [Kafka](kubernetes/kafka/README.md)
 * Deploy [Controller](kubernetes/controller/README.md)
-* Deploy [Invoker](kubernetes/invoker/README.md)
 * Deploy [Nginx](kubernetes/nginx/README.md)
-* Deploy [Ingress](kubernetes/ingress/README.md)
-
-## Configure the OpenWhisk CLI
-
-If you don't already have the wsk cli, follow the instructions [here](https://github.com/apache/incubator-openwhisk-cli) to get it.
-
-Configure the wsk cli by setting the auth and apihost properties (replace API_HOST with the URL appropriate for the Ingress you deployed).
-
-```
-wsk property set --auth `cat kubernetes/cluster-setup/auth.guest` --apihost https://API_HOST
-```
-
-## Install the initial catalog
-
-To do this, you will need to set the `OPENWHISK_HOME` environment variable to a git clone of the main OpenWhisk repository and
-replace API_HOST with the URL appropriate for the Ingress you deployed:
-
-```
-export OPENWHISK_HOME [location of your OpenWhisk clone]
-
-pushd /tmp
-  git clone https://github.com/apache/incubator-openwhisk-catalog
-  cd incubator-openwhisk-catalog/packages
-  ./installCatalog.sh 789c46b1-71f6-4ed5-8c54-816aa4f8c502:abczO3xZCLrMN6v2BKK1dXYFpXlPkccOFqm12CdAsMgRU4VrNZ9lyGVCGuMDGIwP https://API_HOST
-popd
-```
+* Deploy [Ingress](kubernetes/ingress/README.md), including configuring the wsk CLI.
+* Deploy [Invoker](kubernetes/invoker/README.md)
+* Deploy [RouteMgmt](kubernetes/routemgmt/README.md)
+* Deploy [Package Catalog](kubernetes/openwhisk-catalog/README.md)
 
 ## Verify
 
diff --git a/kubernetes/cluster-setup/README.md b/kubernetes/cluster-setup/README.md
index 38d7cd6..b86758b 100644
--- a/kubernetes/cluster-setup/README.md
+++ b/kubernetes/cluster-setup/README.md
@@ -2,9 +2,9 @@ Cluster Setup
 -------------
 
 Before deploying the components of OpenWhisk to a Kubernetes cluster,
-some initial configuration must be done to create a namespace,
-configuration map, and other artifacts that are used by the
-deployments and services that make up OpenWhisk.
+some initial configuration must be done to create a namespace
+and authorization secrets that are used by the deployments and
+services that make up OpenWhisk.
 
 Perform the following steps to prepare your cluster for OpenWhisk.
 
@@ -14,24 +14,16 @@ Perform the following steps to prepare your cluster for OpenWhisk.
 kubectl apply -f namespace.yml
 ```
 
-### Customize whisk.conf and create configmap
-
-* Edit whisk.conf to match your deployment.
-* Create a config map from it.
-```
-kubectl -n openwhisk create configmap whisk --from-env-file=whisk.env
-```
-
 ### Create authorization secrets
 
-The example commands below install the default guest and system
-authorization credentials from the upstream open source project. In
-production deployments, you should obviously use private credentials
-to create these secrets.  The secrets auth.guest and auth.whisk.system
-are used in some subsequent deployment steps to authorize pods to
-perform actions. They must be defined or those steps will fail.
+The command below installs the default guest and system authorization
+credentials from the upstream open source project. In production
+deployments, you should obviously use private credentials to create
+these secrets.  The whisk.auth secret is used in subsequent deployment
+steps to authorize pods to install actions and packages into the
+deployed OpenWhisk. If it is not defined those steps will fail.
 
 ```
-kubectl -n openwhisk create secret generic auth.guest --from-file=auth.guest
-kubectl -n openwhisk create secret generic auth.whisk.system --from-file=auth.whisk.system
+kubectl -n openwhisk create secret generic whisk.auth --from-file=system=auth.whisk.system --from-file=guest=auth.guest
+
 ```
diff --git a/kubernetes/cluster-setup/whisk.env b/kubernetes/cluster-setup/whisk.env
deleted file mode 100644
index 505f861..0000000
--- a/kubernetes/cluster-setup/whisk.env
+++ /dev/null
@@ -1,2 +0,0 @@
-# External hostname used to access your OpenWhisk deployment
-whisk_api_host_name=localhost
diff --git a/kubernetes/ingress/README.md b/kubernetes/ingress/README.md
index 5de062c..4c7892f 100644
--- a/kubernetes/ingress/README.md
+++ b/kubernetes/ingress/README.md
@@ -1,15 +1,40 @@
 Ingress
 -------
 
-To make your OpenWhisk deployment available outside of Kubernetes, you
-need to configure an Ingress to expose the nginx service.
-Unfortunately, the exact details of configuring an Ingress vary across
-cloud providers.  The instructions below describe multiple possible
-Ingress configurations.  We welcome contributions from the community
-to describe how to configure ingress for all the major cloud provider
-providers.
+The goal of this step is to define a Kubernetes Ingress that will make
+OpenWhisk available outside of your Kubernetes cluster. In the
+commands given in the Configuration Steps, replace API_HOST with the
+actual value for your Ingress as determined by the detailed
+instructions in the appropriate subsection of Possible Ingress Types.
+
+# Configuration Steps
+
+1. Create an Ingress, thus determining a value you should use for
+API_HOST.  Unfortunately, the exact details of configuring an Ingress
+vary across cloud providers.  The detailed instructions
+[below](#possible-ingress-types) describe multiple possible Ingress
+configurations.  We welcome contributions from the community to
+describe how to configure Ingress for additional cloud providers.
+
+2. Record the value of API_HOST in a Kubernetes configmap for later
+use within the OpenWhisk deployment:
+```
+kubectl -n openwhisk create configmap whisk.ingress --from-literal=api_host=API_HOST
+```
+
+3. Configure the OpenWhisk CLI, wsk, by setting the auth and apihost
+properties (if you don't already have the wsk cli, follow the
+instructions [here](https://github.com/apache/incubator-openwhisk-cli)
+to get it).
+
+```
+wsk property set --auth `cat ../cluster-setup/auth.guest` --apihost API_HOST
+```
+
+# Possible Ingress Types
+
 
-# NodePort
+## NodePort
 
 When it was deployed, the nginx service was configured to expose
 itself via a NodePort [see](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/nginx/nginx.yml#L10)
@@ -32,7 +57,7 @@ kubectl -n openwhisk describe service nginx | grep https-api | grep NodePort| aw
 Use IP_ADDR:PUBLIC_PORT as your API_HOST
 
 
-# Simple Service Ingress
+## Simple Service Ingress
 
 A basic ingress that simply connects through to the nginx
 service. With this ingress, TLS termination will be handled by the
@@ -45,9 +70,9 @@ kubectl apply -f ingress-simple.yml
 Use `kubectl get ingress` to determine the IP address and port to use
 to define API_HOST for a simple service ingress.
 
-# IBM Cloud
+## IBM Cloud
 
-## IBM Cloud Lite cluster
+### IBM Cloud Lite cluster
 
 The only available ingress method for a Lite cluster is to use a
 NodePort (see above).  By determining the IP address of a worker node
@@ -68,7 +93,7 @@ kubectl -n openwhisk describe service nginx | grep https-api | grep NodePort| aw
  ```
 Use PublicIP:PORT as your API_HOST
 
-## IBM Cloud standard cluster
+### IBM Cloud standard cluster
 
 A template file ingress-ibm.yml is provided.  You will need to edit
 this file to replace <ibmdomain> and <ibmtlssecret> with the correct
@@ -104,6 +129,6 @@ kubectl apply -f ingress-ibm.yml
 Your OpenWhisk API_HOST will be <ibmdomain>/openwhisk
 
 
-# Other cloud providers
+## Other cloud providers
 
 Please submit Pull Requests with instructions for other cloud providers.
diff --git a/kubernetes/invoker/invoker.yml b/kubernetes/invoker/invoker.yml
index 226a030..1f3381a 100644
--- a/kubernetes/invoker/invoker.yml
+++ b/kubernetes/invoker/invoker.yml
@@ -61,8 +61,8 @@ spec:
           - name: "WHISK_API_HOST_NAME"
             valueFrom:
               configMapKeyRef:
-                name: whisk
-                key: whisk_api_host_name
+                name: whisk.ingress
+                key: api_host
           - name: "WHISK_VERSION_BUILDNO"
             value: "latest"
           - name: "INVOKER_CONTAINER_NETWORK"
diff --git a/kubernetes/openwhisk-catalog/install-catalog.yml b/kubernetes/openwhisk-catalog/install-catalog.yml
index 49e7530..c959aee 100644
--- a/kubernetes/openwhisk-catalog/install-catalog.yml
+++ b/kubernetes/openwhisk-catalog/install-catalog.yml
@@ -2,6 +2,7 @@ apiVersion: batch/v1
 kind: Job
 metadata:
   name: install-catalog
+  namespace: openwhisk
 spec:
   activeDeadlineSeconds: 600
   template:
@@ -17,11 +18,11 @@ spec:
           - name: "WHISK_AUTH"
             valueFrom:
               secretKeyRef:
-                name: auth.whisk.system
-                key: auth.whisk.system
+                name: whisk.auth
+                key: system
           - name: "WHISK_API_HOST_NAME"
             valueFrom:
               configMapKeyRef:
-                name: whisk
-                key: whisk_api_host_name
+                name: whisk.ingress
+                key: api_host
       restartPolicy: Never
diff --git a/kubernetes/routemgmt/install-routemgmt.yml b/kubernetes/routemgmt/install-routemgmt.yml
index 1d559d8..8aebd2b 100644
--- a/kubernetes/routemgmt/install-routemgmt.yml
+++ b/kubernetes/routemgmt/install-routemgmt.yml
@@ -2,6 +2,7 @@ apiVersion: batch/v1
 kind: Job
 metadata:
   name: install-routemgmt
+  namespace: openwhisk
 spec:
   activeDeadlineSeconds: 600
   template:
@@ -17,13 +18,13 @@ spec:
           - name: "WHISK_AUTH"
             valueFrom:
               secretKeyRef:
-                name: auth.whisk.system
-                key: auth.whisk.system
+                name: whisk.auth
+                key: system
           - name: "WHISK_API_HOST_NAME"
             valueFrom:
               configMapKeyRef:
-                name: whisk
-                key: whisk_api_host_name
+                name: whisk.ingress
+                key: api_host
           - name: "WHISK_NAMESPACE"
             value: "/whisk.system"
           - name: "WHISK_API_GATEWAY_HOST_V2"
diff --git a/tools/admin/cleanup.sh b/tools/admin/cleanup.sh
index 4b581f6..723ecdc 100755
--- a/tools/admin/cleanup.sh
+++ b/tools/admin/cleanup.sh
@@ -15,12 +15,11 @@ kubectl -n openwhisk delete deployment nginx
 
 # delete configmaps
 kubectl -n openwhisk delete cm nginx
-kubectl -n openwhisk delete cm cluster
+kubectl -n openwhisk delete cm whisk.ingress
 
 # delete secrets
 kubectl -n openwhisk delete secret nginx
-kubectl -n openwhisk delete secret auth.guest
-kubectl -n openwhisk delete secret auth.whisk.system
+kubectl -n openwhisk delete secret auth
 
 # delete ingress
 kubectl -n openwhisk delete ingress ow-ingress
diff --git a/tools/travis/build.sh b/tools/travis/build.sh
index 32d4273..87b7066 100755
--- a/tools/travis/build.sh
+++ b/tools/travis/build.sh
@@ -94,6 +94,38 @@ statefulsetHealthCheck () {
 
 }
 
+jobHealthCheck () {
+  if [ -z "$1" ]; then
+    echo "Error, job health check called without a component parameter"
+    exit 1
+  fi
+
+  PASSED=false
+  TIMEOUT=0
+  until $PASSED || [ $TIMEOUT -eq 30 ]; do
+    KUBE_SUCCESSFUL_JOB=$(kubectl -n openwhisk get jobs -o wide | grep "$1" | awk '{print $3}')
+    if [ "$KUBE_SUCCESSFUL_JOB" == "1" ]; then
+      PASSED=true
+      break
+    fi
+
+    kubectl get jobs --all-namespaces -o wide --show-all
+
+    let TIMEOUT=TIMEOUT+1
+    sleep 10
+  done
+
+  if [ "$PASSED" = false ]; then
+    echo "Failed to finish running $1"
+
+    kubectl -n openwhisk logs jobs/$1
+    exit 1
+  fi
+
+  echo "$1 completed"
+}
+
+
 #################
 # Main body of script -- deploy OpenWhisk
 #################
@@ -114,9 +146,7 @@ kubectl describe nodes
 echo "Performing steps from cluster-setup"
 pushd kubernetes/cluster-setup
   kubectl apply -f namespace.yml
-  kubectl -n openwhisk create configmap whisk --from-env-file=whisk.env
-  kubectl -n openwhisk create secret generic auth.guest --from-file=auth.guest
-  kubectl -n openwhisk create secret generic auth.whisk.system --from-file=auth.whisk.system
+  kubectl -n openwhisk create secret generic whisk.auth --from-file=system=auth.whisk.system --from-file=guest=auth.guest
 popd
 
 # setup couchdb
@@ -159,15 +189,6 @@ pushd kubernetes/controller
   statefulsetHealthCheck "controller"
 popd
 
-# setup the invoker
-echo "Deploying invoker"
-pushd kubernetes/invoker
-  kubectl apply -f invoker.yml
-
-  # wait until the invoker is ready
-  deploymentHealthCheck "invoker"
-popd
-
 # setup nginx
 echo "Deploying nginx"
 pushd kubernetes/nginx
@@ -185,32 +206,46 @@ pushd kubernetes/nginx
   deploymentHealthCheck "nginx"
 popd
 
-AUTH_WSK_SECRET=789c46b1-71f6-4ed5-8c54-816aa4f8c502:abczO3xZCLrMN6v2BKK1dXYFpXlPkccOFqm12CdAsMgRU4VrNZ9lyGVCGuMDGIwP
-AUTH_GUEST=23bc46b1-71f6-4ed5-8c54-816aa4f8c502:123zO3xZCLrMN6v2BKK1dXYFpXlPkccOFqm12CdAsMgRU4VrNZ9lyGVCGuMDGIwP
-WSK_PORT=$(kubectl -n openwhisk describe service nginx | grep https-api | grep NodePort| awk '{print $3}' | cut -d'/' -f1)
+# configure Ingress and wsk CLI
+# We use a NodePort for Travis CI testing
+pushd kubernetes/ingress
+  WSK_PORT=$(kubectl -n openwhisk describe service nginx | grep https-api | grep NodePort| awk '{print $3}' | cut -d'/' -f1)
+  WSK_HOST=$(kubectl describe nodes | grep Hostname: | awk '{print $2}')
+  kubectl -n openwhisk create configmap whisk.ingress --from-literal=api_host=$WSK_HOST:$WSK_PORT
+  wsk property set --auth `cat ../cluster-setup/auth.guest` --apihost $WSK_HOST:$WSK_PORT
+popd
 
-# download and setup the wsk cli
-wget -q https://github.com/apache/incubator-openwhisk-cli/releases/download/latest/OpenWhisk_CLI-latest-linux-amd64.tgz
-tar xzf OpenWhisk_CLI-latest-linux-amd64.tgz
-sudo cp wsk /usr/local/bin/wsk
+# setup the invoker
+echo "Deploying invoker"
+pushd kubernetes/invoker
+  kubectl apply -f invoker.yml
 
-./wsk property set --auth $AUTH_GUEST --apihost https://localhost:$WSK_PORT
+  # wait until the invoker is ready
+  deploymentHealthCheck "invoker"
+popd
 
+# install routemgmt
+echo "Installing routemgmt"
+pushd kubernetes/routemgmt
+  kubectl apply -f install-routemgmt.yml
+  jobHealthCheck "install-routemgmt"
+popd
 
-# setup the catalog
-pushd /tmp
-  git clone https://github.com/apache/incubator-openwhisk
-  export OPENWHISK_HOME=$PWD/incubator-openwhisk
+# install openwhisk-catalog
+echo "Installing catalog"
+pushd kubernetes/openwhisk-catalog
+  kubectl apply -f install-catalog.yml
+  jobHealthCheck "install-catalog"
+popd
 
-  git clone https://github.com/apache/incubator-openwhisk-catalog
+# list packages and actions now installed in /whisk.system
+wsk -i --auth `cat kubernetes/cluster-setup/auth.whisk.system` package list
+wsk -i --auth `cat kubernetes/cluster-setup/auth.whisk.system` action list
 
-  pushd incubator-openwhisk-catalog/packages
-    export WHISK_CLI_PATH=/usr/local/bin/wsk
 
-    # This script currently has an issue where the cli path is the 4th argument
-    ./installCatalog.sh $AUTH_WSK_SECRET https://localhost:$WSK_PORT $WHISK_CLI_PATH
-  popd
-popd
+#################
+# Sniff test: create and invoke a simple Hello world action
+#################
 
 # create wsk action
 cat > hello.js << EOL
@@ -219,12 +254,12 @@ function main() {
 }
 EOL
 
-./wsk -i action create hello hello.js
+wsk -i action create hello hello.js
 
 sleep 5
 
 # run the new hello world action
-RESULT=$(./wsk -i action invoke --blocking hello | grep "\"status\": \"success\"")
+RESULT=$(wsk -i action invoke --blocking hello | grep "\"status\": \"success\"")
 
 if [ -z "$RESULT" ]; then
   echo "FAILED! Could not invoked custom action"
@@ -237,4 +272,4 @@ if [ -z "$RESULT" ]; then
   exit 1
 fi
 
-echo "PASSED! Deployed openwhisk and invoked custom action"
+echo "PASSED! Deployed openwhisk and invoked Hello action"
diff --git a/tools/travis/collect-logs.sh b/tools/travis/collect-logs.sh
index 0670bde..56a4fe3 100755
--- a/tools/travis/collect-logs.sh
+++ b/tools/travis/collect-logs.sh
@@ -19,6 +19,8 @@ kubectl -n openwhisk logs controller-0 >& logs/controller-0.log
 kubectl -n openwhisk logs controller-1 >& logs/controller-1.log
 kubectl -n openwhisk logs -lname=invoker >& logs/invoker.log
 kubectl -n openwhisk logs -lname=nginx >& logs/nginx.log
+kubectl -n openwhisk logs jobs/install-routemgmt >& logs/routemgmt.log
+kubectl -n openwhisk logs jobs/install-catalog >& logs/catalog.log
 kubectl get pods --all-namespaces -o wide --show-all >& logs/all-pods.txt
 
 # System level logs from minikube
diff --git a/tools/travis/setup.sh b/tools/travis/setup.sh
index a894232..d952477 100755
--- a/tools/travis/setup.sh
+++ b/tools/travis/setup.sh
@@ -3,6 +3,11 @@
 
 set -x
 
+# download and install the wsk cli
+wget -q https://github.com/apache/incubator-openwhisk-cli/releases/download/latest/OpenWhisk_CLI-latest-linux-amd64.tgz
+tar xzf OpenWhisk_CLI-latest-linux-amd64.tgz
+sudo cp wsk /usr/local/bin/wsk
+
 # set docker0 to promiscuous mode
 sudo ip link set docker0 promisc on
 


 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services