You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@openwhisk.apache.org by ra...@apache.org on 2018/01/03 19:12:18 UTC
[incubator-openwhisk-deploy-kube] branch master updated: fixes for
apigateway; restructure deployment to eliminate cycles (#133)
This is an automated email from the ASF dual-hosted git repository.
rabbah pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-deploy-kube.git
The following commit(s) were added to refs/heads/master by this push:
new d5dfa34 fixes for apigateway; restructure deployment to eliminate cycles (#133)
d5dfa34 is described below
commit d5dfa34dcd0e6bef4dd73dda3b920f7d070d05a9
Author: David Grove <dg...@users.noreply.github.com>
AuthorDate: Wed Jan 3 14:12:15 2018 -0500
fixes for apigateway; restructure deployment to eliminate cycles (#133)
* fixes for apigateway; restructure deployment to eliminate cycles
Several small fixes to get apigateway functionality working including
adding an ingress definition for apigateway.
Restructure deployment so that all services and ingresses get created
during initial cluster configuration. This breaks circular
dependencies since it makes the service host/port and ingress
information available before any pods are deployed.
Closes #109.
* use PUBLIC_GATEWAY_URL for apigateway
Take advantage of new PUBLIC_GATEWAY_URL support added to
incubator-openwhisk-apigateway to get full control over the base URL
used by the apigateway. Part of the fix for allowing both openwhisk
and apigateway to be exposed as services from a single host using URL
rewriting in a front-end proxy.
---
README.md | 18 ++++--
kubernetes/apigateway/README.md | 7 +++
kubernetes/apigateway/apigateway.yml | 26 ++-------
kubernetes/cluster-setup/README.md | 6 ++
kubernetes/cluster-setup/services.yml | 100 ++++++++++++++++++++++++++++++++++
kubernetes/controller/README.md | 8 +--
kubernetes/controller/controller.yml | 16 ------
kubernetes/couchdb/README.md | 5 +-
kubernetes/couchdb/couchdb.yml | 2 +
kubernetes/ingress/README.md | 40 ++++++++++----
kubernetes/ingress/ingress-ibm.yml | 8 ++-
kubernetes/ingress/ingress-simple.yml | 5 +-
kubernetes/kafka/kafka.yml | 16 ------
kubernetes/nginx/nginx.yml | 23 --------
kubernetes/zookeeper/zookeeper.yml | 22 --------
tools/travis/build.sh | 38 +++++++------
16 files changed, 200 insertions(+), 140 deletions(-)
diff --git a/README.md b/README.md
index ddf2c80..0dd241c 100644
--- a/README.md
+++ b/README.md
@@ -38,9 +38,17 @@ You can also provision a Kubernetes cluster from a cloud provider, subject to th
# Configuring OpenWhisk
+The first time you deploy OpenWhisk on Kubernetes, we recommend
+following the steps below manually so you can inspect the results and
+debug your setup. After you are confident that OpenWhisk deploys
+smoothly on your cluster, you might find it useful to drive your
+deployments using the script [build.sh](tools/travis/build.sh) that we
+use to deploy OpenWhisk on Kubernetes for our Travis CI testing.
+
## Initial Cluster Configuration
* Follow the steps for initial [Cluster Setup](kubernetes/cluster-setup/README.md)
+* Configure your [Ingresses](kubernetes/ingress/README.md), including configuring the wsk CLI.
## Configure or Deploy CouchDB
@@ -62,11 +70,13 @@ directory tree. Follow the instructions for each step in order.
* Deploy [Zookeeper](kubernetes/zookeeper/README.md)
* Deploy [Kafka](kubernetes/kafka/README.md)
* Deploy [Controller](kubernetes/controller/README.md)
-* Deploy [Nginx](kubernetes/nginx/README.md)
-* Deploy [Ingress](kubernetes/ingress/README.md), including configuring the wsk CLI.
* Deploy [Invoker](kubernetes/invoker/README.md)
-* Deploy [RouteMgmt](kubernetes/routemgmt/README.md)
-* Deploy [Package Catalog](kubernetes/openwhisk-catalog/README.md)
+* Deploy [Nginx](kubernetes/nginx/README.md)
+
+## Install system actions and the openwhisk catalog
+
+* Install [RouteMgmt](kubernetes/routemgmt/README.md)
+* Install [Package Catalog](kubernetes/openwhisk-catalog/README.md)
## Verify
diff --git a/kubernetes/apigateway/README.md b/kubernetes/apigateway/README.md
index 91db8e2..672016a 100644
--- a/kubernetes/apigateway/README.md
+++ b/kubernetes/apigateway/README.md
@@ -8,3 +8,10 @@ To deploy the ApiGateway, you only need to run the following command:
```
kubectl apply -f apigateway.yml
```
+
+Note: The URL returned from `wsk api create` may contain a spurious
+:8080 due to its assumption about the meaning of PUBLIC_MANAGEDURL_HOST.
+Working on a fix to the upstream incubator-openwhisk-apigateway project
+to weaken the assumption that the API URL is constructed by concatenating
+PUBLIC_MANAGEDURL_HOST:PUBLIC_MANAGEDURL_PORT as this is not always
+appropriate for kube-based deployments.
diff --git a/kubernetes/apigateway/apigateway.yml b/kubernetes/apigateway/apigateway.yml
index 6d1fa3c..98d8d3b 100644
--- a/kubernetes/apigateway/apigateway.yml
+++ b/kubernetes/apigateway/apigateway.yml
@@ -1,23 +1,4 @@
---
-apiVersion: v1
-kind: Service
-metadata:
- name: apigateway
- namespace: openwhisk
- labels:
- name: apigateway
-spec:
- selector:
- name: apigateway
- ports:
- - port: 8080
- targetPort: 8080
- name: mgmt
- - port: 9000
- targetPort: 9000
- name: api
-
----
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
@@ -39,7 +20,7 @@ spec:
imagePullPolicy: IfNotPresent
image: redis:3.2
- name: apigateway
- imagePullPolicy: IfNotPresent
+ imagePullPolicy: Always
image: openwhisk/apigateway
ports:
- name: mgmt
@@ -51,3 +32,8 @@ spec:
value: "127.0.0.1"
- name: "REDIS_PORT"
value: "6379"
+ - name: "PUBLIC_GATEWAY_URL"
+ valueFrom:
+ configMapKeyRef:
+ name: whisk.ingress
+ key: apigw_url
diff --git a/kubernetes/cluster-setup/README.md b/kubernetes/cluster-setup/README.md
index b86758b..d44adbf 100644
--- a/kubernetes/cluster-setup/README.md
+++ b/kubernetes/cluster-setup/README.md
@@ -14,6 +14,12 @@ Perform the following steps to prepare your cluster for OpenWhisk.
kubectl apply -f namespace.yml
```
+### Create services
+
+```
+kubectl apply -f services.yml
+```
+
### Create authorization secrets
The command below installs the default guest and system authorization
diff --git a/kubernetes/cluster-setup/services.yml b/kubernetes/cluster-setup/services.yml
new file mode 100644
index 0000000..0e02ff1
--- /dev/null
+++ b/kubernetes/cluster-setup/services.yml
@@ -0,0 +1,100 @@
+########
+# First define services that are exposed via Ingresses to the outside world
+########
+---
+apiVersion: v1
+kind: Service
+metadata:
+ name: nginx
+ namespace: openwhisk
+ labels:
+ name: nginx
+spec:
+ type: NodePort
+ selector:
+ name: nginx
+ ports:
+ - port: 80
+ targetPort: 80
+ name: http
+ - port: 443
+ targetPort: 443
+ name: https-api
+ - port: 8443
+ targetPort: 8443
+ name: https-admin
+---
+apiVersion: v1
+kind: Service
+metadata:
+ name: apigateway
+ namespace: openwhisk
+ labels:
+ name: apigateway
+spec:
+ type: NodePort
+ selector:
+ name: apigateway
+ ports:
+ - port: 8080
+ targetPort: 8080
+ name: mgmt
+ - port: 9000
+ targetPort: 9000
+ name: api
+
+########
+# Second define internal services used by the OpenWhisk implementation
+########
+---
+apiVersion: v1
+kind: Service
+metadata:
+ name: controller
+ namespace: openwhisk
+ labels:
+ name: controller
+spec:
+ selector:
+ name: controller
+ clusterIP: None
+ ports:
+ - port: 8080
+ targetPort: 8080
+ name: http
+---
+apiVersion: v1
+kind: Service
+metadata:
+ name: kafka
+ namespace: openwhisk
+ labels:
+ name: kafka
+spec:
+ selector:
+ name: kafka
+ ports:
+ - port: 9092
+ targetPort: 9092
+ name: kafka
+---
+apiVersion: v1
+kind: Service
+metadata:
+ name: zookeeper
+ namespace: openwhisk
+ labels:
+ name: zookeeper
+spec:
+ selector:
+ name: zookeeper
+ ports:
+ - port: 2181
+ targetPort: 2181
+ name: zookeeper
+ - port: 2888
+ targetPort: 2888
+ name: server
+ - port: 3888
+ targetPort: 3888
+ name: leader-election
diff --git a/kubernetes/controller/README.md b/kubernetes/controller/README.md
index 755c154..db0505c 100644
--- a/kubernetes/controller/README.md
+++ b/kubernetes/controller/README.md
@@ -15,10 +15,10 @@ kubectl apply -f controller.yml
# Controller Deployment Changes
## Increase Controller Count
-You will need to update the replication count for the
-Controllers [here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L26)
-and the value of CONTROLLER_INSTANCES [here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L82)
-and the value of AKKA_CLUSTER_SEED_NODES [here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L112)
+You will need to update the number of replicas for the
+Controllers [here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L10)
+and the value of CONTROLLER_INSTANCES [here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L60)
+and the value of AKKA_CLUSTER_SEED_NODES [here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L88)
and redeploy.
[StatefulSet]: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/
diff --git a/kubernetes/controller/controller.yml b/kubernetes/controller/controller.yml
index 4eb90d7..993caf4 100644
--- a/kubernetes/controller/controller.yml
+++ b/kubernetes/controller/controller.yml
@@ -1,20 +1,4 @@
---
-apiVersion: v1
-kind: Service
-metadata:
- name: controller
- namespace: openwhisk
- labels:
- name: controller
-spec:
- selector:
- name: controller
- clusterIP: None
- ports:
- - port: 8080
- targetPort: 8080
- name: http
----
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
diff --git a/kubernetes/couchdb/README.md b/kubernetes/couchdb/README.md
index 642a76d..3ce9c6a 100644
--- a/kubernetes/couchdb/README.md
+++ b/kubernetes/couchdb/README.md
@@ -20,8 +20,7 @@ This is because it might not yet be configured. To check if the
DB has been setup, you can look at the Pod logs with
```
- export COUCH_DB_POD=$(kubectl -n openwhisk get pods -o wide --show-all | grep "couchdb" | awk '{print $1}')
- kubectl -n openwhisk logs $COUCH_DB_POD
+kubectl -n openwhisk logs -lname=couchdb
```
In the logs, you should see the line:
@@ -36,7 +35,7 @@ This indicates that the CouchDB instancs is up and running.
## Usernames and Passwords
To configure custom usernames and passwords, you can edit
-the CouchDB [setup pod](https://github.com/apache/incubator-openwhisk-deploy-kube/blob/master/kubernetes/couchdb/couchdb.yml#L46-L49).
+the CouchDB [setup pod](https://github.com/apache/incubator-openwhisk-deploy-kube/blob/master/kubernetes/couchdb/couchdb.yml#L48-L51).
**NOTE** If the CouchDB username and password properties
are updated, then you will need to update the Controller
diff --git a/kubernetes/couchdb/couchdb.yml b/kubernetes/couchdb/couchdb.yml
index 5bd67e9..30a277f 100644
--- a/kubernetes/couchdb/couchdb.yml
+++ b/kubernetes/couchdb/couchdb.yml
@@ -1,3 +1,5 @@
+# The couchdb service is defined here instead of in cluster-config/services
+# because some deployments will not put the database within the kube cluster.
---
apiVersion: v1
kind: Service
diff --git a/kubernetes/ingress/README.md b/kubernetes/ingress/README.md
index 4c7892f..8cc8cc6 100644
--- a/kubernetes/ingress/README.md
+++ b/kubernetes/ingress/README.md
@@ -16,10 +16,13 @@ vary across cloud providers. The detailed instructions
configurations. We welcome contributions from the community to
describe how to configure Ingress for additional cloud providers.
-2. Record the value of API_HOST in a Kubernetes configmap for later
-use within the OpenWhisk deployment:
+2. Record the value of API_HOST and APIGW_URL in a Kubernetes configmap
+for later use within the OpenWhisk deployment. Note that API_HOST is
+expected to be either a host or host:port pair, but APIGW_URL is
+expected to be a URL, including protocol (http or https depending on
+your ingress):
```
-kubectl -n openwhisk create configmap whisk.ingress --from-literal=api_host=API_HOST
+kubectl -n openwhisk create configmap whisk.ingress --from-literal=api_host=API_HOST --from-literal=apigw_url=APIGW_URL
```
3. Configure the OpenWhisk CLI, wsk, by setting the auth and apihost
@@ -36,26 +39,33 @@ wsk property set --auth `cat ../cluster-setup/auth.guest` --apihost API_HOST
## NodePort
-When it was deployed, the nginx service was configured to expose
-itself via a NodePort [see](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/nginx/nginx.yml#L10)
+When it was deployed, the apigateway and nginx services were
+configured to expose themselves via a NodePort
+[see](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/cluster-setup/services.yml#L13)
By determining the IP address of a worker node and the exposed port
-number, you can determine your API_HOST. There are no additional files
-to apply. TLS termination is handled by the nginx service.
+numbers, you can determine your API_HOST and APIGW_URL. There are no
+additional files to apply. TLS termination is handled by the nginx
+service.
1. Obtain the IP address of the Kubernetes nodes.
-
```
kubectl get nodes
```
2. Obtain the public port for https port of the openwhisk.nginx Service
-
```
kubectl -n openwhisk describe service nginx | grep https-api | grep NodePort| awk '{print $3}' | cut -d'/' -f1
```
Use IP_ADDR:PUBLIC_PORT as your API_HOST
+3. Obtain the public port for https port of the openwhisk.apigateway Service
+ ```
+kubectl -n openwhisk describe service apigateway | grep mgmt | grep NodePort| awk '{print $3}' | cut -d'/' -f1
+ ```
+
+Use http://IP_ADDR:PUBLIC_PORT as your APIGW_URL
+
## Simple Service Ingress
@@ -93,6 +103,14 @@ kubectl -n openwhisk describe service nginx | grep https-api | grep NodePort| aw
```
Use PublicIP:PORT as your API_HOST
+3. Obtain the public port for https port of the openwhisk.apigateway Service
+
+ ```
+kubectl -n openwhisk describe service apigateway | grep mgmt | grep NodePort| awk '{print $3}' | cut -d'/' -f1
+ ```
+
+Use http://IP_ADDR:PUBLIC_PORT as your APIGW_URL
+
### IBM Cloud standard cluster
A template file ingress-ibm.yml is provided. You will need to edit
@@ -126,8 +144,8 @@ After editing the template file, deploy it.
kubectl apply -f ingress-ibm.yml
```
-Your OpenWhisk API_HOST will be <ibmdomain>/openwhisk
-
+Your API_HOST will be <ibmdomain>/openwhisk
+Your APIGW_URL will be https://<ibmdomain>/apigateway
## Other cloud providers
diff --git a/kubernetes/ingress/ingress-ibm.yml b/kubernetes/ingress/ingress-ibm.yml
index 505004f..0947c34 100644
--- a/kubernetes/ingress/ingress-ibm.yml
+++ b/kubernetes/ingress/ingress-ibm.yml
@@ -4,7 +4,7 @@ metadata:
name: ow-ingress
namespace: openwhisk
annotations:
- ingress.bluemix.net/rewrite-path: "serviceName=nginx rewrite=/"
+ ingress.bluemix.net/rewrite-path: "serviceName=nginx rewrite=/;serviceName=apigateway rewrite=/"
spec:
tls:
- hosts:
@@ -17,4 +17,8 @@ spec:
- path: /openwhisk/
backend:
serviceName: nginx
- servicePort: 80
+ servicePort: http
+ - path: /apigateway/
+ backend:
+ serviceName: apigateway
+ servicePort: mgmt
diff --git a/kubernetes/ingress/ingress-simple.yml b/kubernetes/ingress/ingress-simple.yml
index 90ad2a7..fba89ea 100644
--- a/kubernetes/ingress/ingress-simple.yml
+++ b/kubernetes/ingress/ingress-simple.yml
@@ -6,4 +6,7 @@ metadata:
spec:
backend:
serviceName: nginx
- servicePort: 443
+ servicePort: https-api
+ backend:
+ serviceName: apigateway
+ servicePort: mgmt
diff --git a/kubernetes/kafka/kafka.yml b/kubernetes/kafka/kafka.yml
index 19817cf..9f67ece 100644
--- a/kubernetes/kafka/kafka.yml
+++ b/kubernetes/kafka/kafka.yml
@@ -1,20 +1,4 @@
---
-apiVersion: v1
-kind: Service
-metadata:
- name: kafka
- namespace: openwhisk
- labels:
- name: kafka
-spec:
- selector:
- name: kafka
- ports:
- - port: 9092
- targetPort: 9092
- name: kafka
-
----
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
diff --git a/kubernetes/nginx/nginx.yml b/kubernetes/nginx/nginx.yml
index f9659a4..486a66d 100644
--- a/kubernetes/nginx/nginx.yml
+++ b/kubernetes/nginx/nginx.yml
@@ -1,27 +1,4 @@
---
-apiVersion: v1
-kind: Service
-metadata:
- name: nginx
- namespace: openwhisk
- labels:
- name: nginx
-spec:
- type: NodePort
- selector:
- name: nginx
- ports:
- - port: 80
- targetPort: 80
- name: http
- - port: 443
- targetPort: 443
- name: https-api
- - port: 8443
- targetPort: 8443
- name: https-admin
-
----
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
diff --git a/kubernetes/zookeeper/zookeeper.yml b/kubernetes/zookeeper/zookeeper.yml
index 50123ed..cd292c5 100644
--- a/kubernetes/zookeeper/zookeeper.yml
+++ b/kubernetes/zookeeper/zookeeper.yml
@@ -1,26 +1,4 @@
---
-apiVersion: v1
-kind: Service
-metadata:
- name: zookeeper
- namespace: openwhisk
- labels:
- name: zookeeper
-spec:
- selector:
- name: zookeeper
- ports:
- - port: 2181
- targetPort: 2181
- name: zookeeper
- - port: 2888
- targetPort: 2888
- name: server
- - port: 3888
- targetPort: 3888
- name: leader-election
-
----
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
diff --git a/tools/travis/build.sh b/tools/travis/build.sh
index 87b7066..4fbed1a 100755
--- a/tools/travis/build.sh
+++ b/tools/travis/build.sh
@@ -146,9 +146,20 @@ kubectl describe nodes
echo "Performing steps from cluster-setup"
pushd kubernetes/cluster-setup
kubectl apply -f namespace.yml
+ kubectl apply -f services.yml
kubectl -n openwhisk create secret generic whisk.auth --from-file=system=auth.whisk.system --from-file=guest=auth.guest
popd
+# configure Ingress and wsk CLI
+# We use the NodePorts for nginx and apigateway services for Travis CI testing
+pushd kubernetes/ingress
+ WSK_PORT=$(kubectl -n openwhisk describe service nginx | grep https-api | grep NodePort| awk '{print $3}' | cut -d'/' -f1)
+ APIGW_PORT=$(kubectl -n openwhisk describe service apigateway | grep mgmt | grep NodePort| awk '{print $3}' | cut -d'/' -f1)
+ WSK_HOST=$(kubectl describe nodes | grep Hostname: | awk '{print $2}')
+ kubectl -n openwhisk create configmap whisk.ingress --from-literal=api_host=$WSK_HOST:$WSK_PORT --from-literal=apigw_url=http://$WSK_HOST:$APIGW_PORT
+ wsk property set --auth `cat ../cluster-setup/auth.guest` --apihost $WSK_HOST:$WSK_PORT
+popd
+
# setup couchdb
echo "Deploying couchdb"
pushd kubernetes/couchdb
@@ -189,6 +200,15 @@ pushd kubernetes/controller
statefulsetHealthCheck "controller"
popd
+# setup the invoker
+echo "Deploying invoker"
+pushd kubernetes/invoker
+ kubectl apply -f invoker.yml
+
+ # wait until the invoker is ready
+ deploymentHealthCheck "invoker"
+popd
+
# setup nginx
echo "Deploying nginx"
pushd kubernetes/nginx
@@ -206,24 +226,6 @@ pushd kubernetes/nginx
deploymentHealthCheck "nginx"
popd
-# configure Ingress and wsk CLI
-# We use a NodePort for Travis CI testing
-pushd kubernetes/ingress
- WSK_PORT=$(kubectl -n openwhisk describe service nginx | grep https-api | grep NodePort| awk '{print $3}' | cut -d'/' -f1)
- WSK_HOST=$(kubectl describe nodes | grep Hostname: | awk '{print $2}')
- kubectl -n openwhisk create configmap whisk.ingress --from-literal=api_host=$WSK_HOST:$WSK_PORT
- wsk property set --auth `cat ../cluster-setup/auth.guest` --apihost $WSK_HOST:$WSK_PORT
-popd
-
-# setup the invoker
-echo "Deploying invoker"
-pushd kubernetes/invoker
- kubectl apply -f invoker.yml
-
- # wait until the invoker is ready
- deploymentHealthCheck "invoker"
-popd
-
# install routemgmt
echo "Installing routemgmt"
pushd kubernetes/routemgmt
--
To stop receiving notification emails like this one, please contact
['"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>'].