You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@openwhisk.apache.org by GitBox <gi...@apache.org> on 2017/12/15 16:45:44 UTC
[GitHub] rabbah closed pull request #123: add cluster-setup step to deployment workflow and Job to install routemgmt actions
rabbah closed pull request #123: add cluster-setup step to deployment workflow and Job to install routemgmt actions
URL: https://github.com/apache/incubator-openwhisk-deploy-kube/pull/123
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/.travis.yml b/.travis.yml
index 7619d6b..f47846a 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -24,5 +24,6 @@ deploy:
- provider: script
script:
- ./tools/travis/publish.sh openwhisk kube-couchdb latest docker/couchdb
+ - ./tools/travis/publish.sh openwhisk kube-routemgmt latest docker/routemgmt
on:
branch: master
diff --git a/README.md b/README.md
index bd1fd8c..c182012 100644
--- a/README.md
+++ b/README.md
@@ -18,7 +18,7 @@ Several requirements must be met for OpenWhisk to deploy on Kubernetes.
**Kubernetes**
* [Kubernetes](https://github.com/kubernetes/kubernetes) version 1.6+. However, avoid Kubernetes 1.6.3 due to an [issue with volume mount subpaths](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.6.md#known-issues-for-v163). Our Travis CI testing uses Kubernetes version 1.7.4.
-* The ability to create Ingresses to expose a Kubernetes service to the outside of a cluster so you can actually use OpenWhisk.
+* The ability to create Ingresses to make a Kubernetes service available outside of the cluster so you can actually use OpenWhisk.
* Endpoints of Kubernetes services must be able to loopback to themselves ("hairpin mode").
**OpenWhisk**
@@ -40,14 +40,14 @@ You can also provision a Kubernetes cluster from a cloud provider, subject to th
## Initial Configuration
-* Create the openwhisk namespace: `kubectl apply -f configure/openwhisk_kube_namespace.yml`
+* Follow the steps for initial [Cluster Setup](kubernetes/cluster-setup/README.md)
## Deploy Components
To deploy OpenWhisk on Kubernetes, you must deploy its components in
-the proper order. Detailed instructions and the supporting .yml files
-can be found in the kubernetes directory tree. You will need to follow
-the instructions for each step in order.
+the proper order. Detailed instructions and the supporting configuration
+files can be found in the kubernetes directory tree. You will need to
+follow the instructions for each step in order.
* Configure or deploy CouchDB.
* For development and testing purposes, this repo includes a configuration
@@ -70,7 +70,7 @@ If you don't already have the wsk cli, follow the instructions [here](https://gi
Configure the wsk cli by setting the auth and apihost properties (replace API_HOST with the URL appropriate for the Ingress you deployed).
```
-wsk property set --auth 23bc46b1-71f6-4ed5-8c54-816aa4f8c502:123zO3xZCLrMN6v2BKK1dXYFpXlPkccOFqm12CdAsMgRU4VrNZ9lyGVCGuMDGIwP --apihost https://API_HOST
+wsk property set --auth `cat kubernetes/cluster-setup/auth.guest` --apihost https://API_HOST
```
## Install the initial catalog
@@ -101,7 +101,7 @@ For this, we want to delete all the OpenWhisk deployments, services, jobs
and whatever else might be there. We provide a script to do this:
```
-./configure/cleanup.sh
+./tools/admin/cleanup.sh
```
# Issues
diff --git a/docker/README.md b/docker/README.md
index de340f7..e0c204c 100644
--- a/docker/README.md
+++ b/docker/README.md
@@ -8,6 +8,8 @@ The built images are:
* couchdb - creates and initializes a CouchDB instance for
dev/testing of OpenWhisk. This image is not intended for
production usage.
+ * routemgmt - installs OpenWhisk's route management package
+ in the system namespace of the OpenWhisk deployment.
The nginx and kafka images are not officially built and published
because they are considered to be temporary. We are working on
diff --git a/docker/routemgmt/Dockerfile b/docker/routemgmt/Dockerfile
new file mode 100644
index 0000000..cc8d632
--- /dev/null
+++ b/docker/routemgmt/Dockerfile
@@ -0,0 +1,17 @@
+from ubuntu:latest
+
+RUN apt-get -y update && apt-get -y install \
+ git \
+ wget \
+ zip \
+ python-dev \
+ python-pip
+
+RUN pip install --upgrade setuptools
+RUN pip install argcomplete
+RUN pip install ansible==2.3.0.0
+
+COPY init.sh /init.sh
+RUN chmod +X /init.sh
+
+CMD ["/init.sh"]
diff --git a/docker/routemgmt/init.sh b/docker/routemgmt/init.sh
new file mode 100755
index 0000000..0f69067
--- /dev/null
+++ b/docker/routemgmt/init.sh
@@ -0,0 +1,32 @@
+#!/bin/bash
+
+set -x
+
+export OPENWHISK_HOME=/openwhisk
+
+# Clone openwhisk repo to get latest installRouteMgmt.sh and core/routemgmt
+# TODO: when OpenWhisk has releases, download release artifacts instead!
+git clone https://github.com/apache/incubator-openwhisk openwhisk
+
+cd $OPENWHISK_HOME
+
+# Download and install openwhisk cli
+pushd bin
+ wget -q https://github.com/apache/incubator-openwhisk-cli/releases/download/$WHISK_CLI_VERSION/OpenWhisk_CLI-$WHISK_CLI_VERSION-linux-amd64.tgz
+ tar xzf OpenWhisk_CLI-$WHISK_CLI_VERSION-linux-amd64.tgz
+popd
+
+# Generate whisk.properties.
+# TODO: Refactor upstream ansible/roles/routemgmt/files/installRouteMgmt.sh to enable
+# override of apigw values from environment so we don't have to bother running
+# ansible here to generate whisk.properties just so the script can extract 3 values.
+pushd ansible
+ ansible-playbook setup.yml
+ ansible-playbook properties.yml -e apigw_host_v2=$WHISK_API_GATEWAY_HOST_V2
+popd
+
+# Run installRouteMgmt.sh
+pushd ansible/roles/routemgmt/files
+ ./installRouteMgmt.sh $WHISK_AUTH $WHISK_API_HOST_NAME $WHISK_NAMESPACE $OPENWHISK_HOME/bin/wsk
+popd
+
diff --git a/kubernetes/cluster-setup/README.md b/kubernetes/cluster-setup/README.md
new file mode 100644
index 0000000..38d7cd6
--- /dev/null
+++ b/kubernetes/cluster-setup/README.md
@@ -0,0 +1,37 @@
+Cluster Setup
+-------------
+
+Before deploying the components of OpenWhisk to a Kubernetes cluster,
+some initial configuration must be done to create a namespace,
+configuration map, and other artifacts that are used by the
+deployments and services that make up OpenWhisk.
+
+Perform the following steps to prepare your cluster for OpenWhisk.
+
+### Create the openwhisk namespace
+
+```
+kubectl apply -f namespace.yml
+```
+
+### Customize whisk.conf and create configmap
+
+* Edit whisk.conf to match your deployment.
+* Create a config map from it.
+```
+kubectl -n openwhisk create configmap whisk --from-env-file=whisk.env
+```
+
+### Create authorization secrets
+
+The example commands below install the default guest and system
+authorization credentials from the upstream open source project. In
+production deployments, you should obviously use private credentials
+to create these secrets. The secrets auth.guest and auth.whisk.system
+are used in some subsequent deployment steps to authorize pods to
+perform actions. They must be defined or those steps will fail.
+
+```
+kubectl -n openwhisk create secret generic auth.guest --from-file=auth.guest
+kubectl -n openwhisk create secret generic auth.whisk.system --from-file=auth.whisk.system
+```
diff --git a/kubernetes/cluster-setup/auth.guest b/kubernetes/cluster-setup/auth.guest
new file mode 100644
index 0000000..3156274
--- /dev/null
+++ b/kubernetes/cluster-setup/auth.guest
@@ -0,0 +1 @@
+23bc46b1-71f6-4ed5-8c54-816aa4f8c502:123zO3xZCLrMN6v2BKK1dXYFpXlPkccOFqm12CdAsMgRU4VrNZ9lyGVCGuMDGIwP
diff --git a/kubernetes/cluster-setup/auth.whisk.system b/kubernetes/cluster-setup/auth.whisk.system
new file mode 100644
index 0000000..e44545b
--- /dev/null
+++ b/kubernetes/cluster-setup/auth.whisk.system
@@ -0,0 +1 @@
+789c46b1-71f6-4ed5-8c54-816aa4f8c502:abczO3xZCLrMN6v2BKK1dXYFpXlPkccOFqm12CdAsMgRU4VrNZ9lyGVCGuMDGIwP
diff --git a/configure/openwhisk_kube_namespace.yml b/kubernetes/cluster-setup/namespace.yml
similarity index 100%
rename from configure/openwhisk_kube_namespace.yml
rename to kubernetes/cluster-setup/namespace.yml
diff --git a/kubernetes/cluster-setup/whisk.env b/kubernetes/cluster-setup/whisk.env
new file mode 100644
index 0000000..505f861
--- /dev/null
+++ b/kubernetes/cluster-setup/whisk.env
@@ -0,0 +1,2 @@
+# External hostname used to access your OpenWhisk deployment
+whisk_api_host_name=localhost
diff --git a/kubernetes/invoker/README.md b/kubernetes/invoker/README.md
index 02c13db..ee1c7a7 100644
--- a/kubernetes/invoker/README.md
+++ b/kubernetes/invoker/README.md
@@ -18,16 +18,7 @@ kubectl label nodes [node name] openwhisk=invoker
$ kubectl label nodes 127.0.0.1 openwhisk=invoker
```
-If you would then like to restrict nodes farther so that
-they only run Invoker pods, you can set some taints:
-
-```
-kubectl taint nodes [node name] dedicated=invoker:NoSchedule
-kubectl taint nodes [node name] dedicated=invoker:NoExecute
-```
-
-The taint nodes are optional, but once the invoker label is applied,
-you can create the invokers with:
+Once the invoker label is applied, you can create the invokers with:
```
kubectl apply -f invoker.yml
@@ -44,35 +35,11 @@ that the Kubernetes host image is Ubuntu. During the deploy there could be an
issue and if the Invoker fails to deploy, see the [Troubleshooting](#troubleshooting)
section below.
-# Invoker Deployment Changes
-## Increase Invoker Count
-
-To increase the number of Invokers, edit the
-[replicas](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/invoker/invoker.yml#L9)
-line. Secondly, you will need to update the
-[INVOKER_INSTANCES](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/invoker/invoker.yml#L70)
-to with the same replica count.
-
-## Deploying Invoker to Specific Kube Nodes
-
-To deploy an Invoker to specific Kube nodes, you will need to edit the
-[invoker.yml](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/invoker/invoker.yml)
-file with Kubernetes [NodeSelectors](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/).
-
# Troubleshooting
-## Deploying to Minikube
+## No invokers are deployed
-When deploying the Invoker to [Minikube](https://kubernetes.io/docs/getting-started-guides/minikube/)
-you might need to edit the Invoker's Docker Api Version.
-This is because Minikube uses Docker version 1.11.x.
-To do this, you will need to add the following properties
-to the invoker.yml file.
+Verify that you actually have nodes with the label openwhisk=invoker.
-```
-env:
- - name: "DOCKER_API_VERSION"
- value: "1.23"
-```
## Kubernetes Host Linux Versions
Unfortunately when Deploying OpenWhisk on Kubernetes it currently mounts some
diff --git a/kubernetes/invoker/invoker.yml b/kubernetes/invoker/invoker.yml
index 8abddb2..226a030 100644
--- a/kubernetes/invoker/invoker.yml
+++ b/kubernetes/invoker/invoker.yml
@@ -59,7 +59,10 @@ spec:
- name: "SERVICE_CHECK_INTERVAL"
value: "15s"
- name: "WHISK_API_HOST_NAME"
- value: "nginx.openwhisk"
+ valueFrom:
+ configMapKeyRef:
+ name: whisk
+ key: whisk_api_host_name
- name: "WHISK_VERSION_BUILDNO"
value: "latest"
- name: "INVOKER_CONTAINER_NETWORK"
@@ -75,16 +78,6 @@ spec:
- name: "DOCKER_REGISTRY"
value: ""
- # Invoker instance count. Needs to match replica count
- - name: "INVOKER_INSTANCES"
- value: "1"
-
- # Invoker assigned name. Derived from hostname
- - name: "INVOKER_NAME"
- valueFrom:
- fieldRef:
- fieldPath: metadata.name
-
# Java options
- name: "JAVA_OPTS"
value: "-Xmx2g"
@@ -105,7 +98,7 @@ spec:
- name: "RUNTIMES_MANIFEST"
value: '{ "defaultImagePrefix": "openwhisk", "defaultImageTag": "latest", "runtimes": { "nodejs": [ { "kind": "nodejs", "image": { "name": "nodejsaction" }, "deprecated": true }, { "kind": "nodejs:6", "default": true, "image": { "name": "nodejs6action" }, "deprecated": false } ], "python": [ { "kind": "python", "image": { "name": "python2action" }, "deprecated": false }, { "kind": "python:2", "default": true, "image": { "name": "python2action" }, "deprecated": false }, { "kind": "python:3", "image": { "name": "python3action" }, "deprecated": false } ], "swift": [ { "kind": "swift", "image": { "name": "swiftaction" }, "deprecated": true }, { "kind": "swift:3", "image": { "name": "swift3action" }, "deprecated": true }, { "kind": "swift:3.1.1", "default": true, "image": { "name": "action-swift-v3.1.1" }, "deprecated": false } ], "java": [ { "kind": "java", "default": true, "image": { "name": "java8action" }, "deprecated": false, "attached": { "attachmentName": "jarfile", "attachmentType": "application/java-archive" }, "sentinelledLogs": false, "requireMain": true } ] }, "blackboxes": [ { "name": "dockerskeleton" } ] }'
- # Default to empty logs dir. This is because logs should go to stdout
+ # Default to empty logs dir. This is because logs should go to stdout on kube
- name: "WHISK_LOGS_DIR"
value: ""
diff --git a/kubernetes/routemgmt/README.md b/kubernetes/routemgmt/README.md
new file mode 100644
index 0000000..2c73a55
--- /dev/null
+++ b/kubernetes/routemgmt/README.md
@@ -0,0 +1,13 @@
+Route Management
+-----
+
+Once the system is deployed, we need to run a job
+to install packages that support route management.
+
+# Deploying
+
+To run the Job, you just need to run:
+
+```
+kubectl apply -f install-routemgmt.yml
+```
diff --git a/kubernetes/routemgmt/install-routemgmt.yml b/kubernetes/routemgmt/install-routemgmt.yml
new file mode 100644
index 0000000..1d559d8
--- /dev/null
+++ b/kubernetes/routemgmt/install-routemgmt.yml
@@ -0,0 +1,32 @@
+apiVersion: batch/v1
+kind: Job
+metadata:
+ name: install-routemgmt
+spec:
+ activeDeadlineSeconds: 600
+ template:
+ metadata:
+ name: install-routemgmt
+ spec:
+ containers:
+ - name: routemgmt
+ image: openwhisk/kube-routemgmt
+ env:
+ - name: "WHISK_CLI_VERSION"
+ value: "latest"
+ - name: "WHISK_AUTH"
+ valueFrom:
+ secretKeyRef:
+ name: auth.whisk.system
+ key: auth.whisk.system
+ - name: "WHISK_API_HOST_NAME"
+ valueFrom:
+ configMapKeyRef:
+ name: whisk
+ key: whisk_api_host_name
+ - name: "WHISK_NAMESPACE"
+ value: "/whisk.system"
+ - name: "WHISK_API_GATEWAY_HOST_V2"
+ value: "http://$(APIGATEWAY_SERVICE_HOST):$(APIGATEWAY_SERVICE_PORT_API)/v2"
+
+ restartPolicy: Never
diff --git a/configure/cleanup.sh b/tools/admin/cleanup.sh
similarity index 76%
rename from configure/cleanup.sh
rename to tools/admin/cleanup.sh
index 48956b6..4b581f6 100755
--- a/configure/cleanup.sh
+++ b/tools/admin/cleanup.sh
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
-# this script is used to cleanup the OpenWhisk deployment
+# this script is used to completely remove the OpenWhisk deployment
set -x
@@ -15,9 +15,12 @@ kubectl -n openwhisk delete deployment nginx
# delete configmaps
kubectl -n openwhisk delete cm nginx
+kubectl -n openwhisk delete cm cluster
# delete secrets
kubectl -n openwhisk delete secret nginx
+kubectl -n openwhisk delete secret auth.guest
+kubectl -n openwhisk delete secret auth.whisk.system
# delete ingress
kubectl -n openwhisk delete ingress ow-ingress
@@ -29,3 +32,6 @@ kubectl -n openwhisk delete service zookeeper
kubectl -n openwhisk delete service kafka
kubectl -n openwhisk delete service controller
kubectl -n openwhisk delete service nginx
+
+# delete namespace
+kubectl delete namespace openwhisk
diff --git a/tools/travis/build.sh b/tools/travis/build.sh
index b711e50..37f56c5 100755
--- a/tools/travis/build.sh
+++ b/tools/travis/build.sh
@@ -1,18 +1,8 @@
#!/bin/bash
-set -x
-
-SCRIPTDIR=$(cd $(dirname "$0") && pwd)
-ROOTDIR="$SCRIPTDIR/../../"
-
-cd $ROOTDIR
-
-echo "Creating openwhisk namespace"
-kubectl apply -f configure/openwhisk_kube_namespace.yml
-
-echo "Labeling invoker node"
-kubectl label nodes --all openwhisk=invoker
-kubectl describe nodes
+#################
+# Helper functions for verifying pod creation
+#################
couchdbHealthCheck () {
# wait for the pod to be created before getting the job name
@@ -104,6 +94,31 @@ statefulsetHealthCheck () {
}
+#################
+# Main body of script -- deploy OpenWhisk
+#################
+
+set -x
+
+SCRIPTDIR=$(cd $(dirname "$0") && pwd)
+ROOTDIR="$SCRIPTDIR/../../"
+
+cd $ROOTDIR
+
+# Label invoker nodes (needed for daemonset-based invoker deployment)
+echo "Labeling invoker node"
+kubectl label nodes --all openwhisk=invoker
+kubectl describe nodes
+
+# Initial cluster setup
+echo "Performing steps from cluster-setup"
+pushd kubernetes/cluster-setup
+ kubectl apply -f namespace.yml
+ kubectl -n openwhisk create configmap whisk --from-env-file=whisk.env
+ kubectl -n openwhisk create secret generic auth.guest --from-file=auth.guest
+ kubectl -n openwhisk create secret generic auth.whisk.system --from-file=auth.whisk.system
+popd
+
# setup couchdb
echo "Deploying couchdb"
pushd kubernetes/couchdb
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services