You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@openwhisk.apache.org by GitBox <gi...@apache.org> on 2018/04/20 22:31:11 UTC
[GitHub] csantanapr closed pull request #186: Helm chart improvements
csantanapr closed pull request #186: Helm chart improvements
URL: https://github.com/apache/incubator-openwhisk-deploy-kube/pull/186
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/helm/README.md b/helm/README.md
index 3624054..6389115 100644
--- a/helm/README.md
+++ b/helm/README.md
@@ -1,50 +1,26 @@
## Overview
-[Apache OpenWhisk](http://www.openwhisk.org) is a popular container-native serverless platform.
-This chart targets to deploy an OpenWhisk cluster over Kubernetes cluster.
+This directory contains a Helm chart can be used to deploy Apache OpenWhisk to a Kubernetes cluster.
-This chart is designed to deploy the following 6 kinds of containers on a running Kubernetes cluster:
+It currently does not support all of the options available in the more
+manual deployment process described in the top-level README.md,
+however we are actively working on closing the gaps.
-- `zookeeper`: deployed in Kubernetes as a `Service`(backended by a `Deployment`). Provides `zookeeper` service for `kafka` node.
-- `kafka`: deployed in Kubernetes as a `StatefulSet`. Depends on `zookeeper` service and provides message service for `controller` node and `invoker` node.
-- `couchdb`: deployed in Kubernetes as a `Service`(backended by a `Deployment`). Provides database service for `controller` node and `invoker` node.
-- `controller`: OpenWhisk controller component, deployed as a `StatefulSet`.
-- `invoker`: OpenWhisk invoker component, deployed as a `StatefulSet`.
-- `nginx`: deployed in Kubernetes as a combination of a `Service`(of type `NodePort`, backended by a `Deployment`), a `ConfigMap` and a `Secret`. Provides public access endpoint for end user to visit OpenWhisk.
-
-By default, deploying this chart will result in deploying 6 containers in a Kubernetes cluster, which forms a minimum deployment of a workable OpenWhisk environment. Please modify `values.yaml` and corresponding template files to deploy a bigger customized OpenWhisk cluster.
+The default values used in the chart are designed to deploy a minimal
+OpenWhisk deployment suitable for local development or testing on
+minikube or a single node Kubernetes cluster. We will soon provide a
+second set of default values suitable for larger-scale deployments.
## Deployment Steps
Please follow the following steps in this section to use [Helm](https://github.com/kubernetes/helm) to deploy this chart.
-### Step 1. Prepare Docker Images
-
-The first step is to prepare docker images used by this chart on your Kubernetes node. Here is a summary of the images needed:
-
-- `zookeeper`: uses `zookeeper:3.4` at present.
-- `kafka`: uses `wurstmeister/kafka:0.11.0.1`.
-- `couchdb`: uses `openwhisk/kube-couchdb`, this is a pre-built docker image made from official `openwhisk-on-kubernetes`. For more information, please check the [docker file](https://github.com/apache/incubator-openwhisk-deploy-kube/blob/master/docker/couchdb/Dockerfile) for this image.
-- `controller`: uses OpenWhisk official `openwhisk/controller` image.
-- `invoker`: uses OpenWhisk official `openwhisk/invoker` image.
-- `nginx`: uses `nginx:1.11`.
-
-This chart provides default images for all of the above containers, so you can try deploy without building any images in advance.
-
-This chart will also automatically pull the following action images for `invoker` once deployment is finished:
-
-- openwhisk/nodejsactionbase
-- openwhisk/nodejs6action
-- openwhisk/dockerskeleton
-- openwhisk/python2action
-- openwhisk/python3action
-- openwhisk/action-swift-v3.1.1
-- openwhisk/swift3action
-- openwhisk/java8action
+### Step 1. Prepare Kubernetes and Helm
-### Step 2. Prepare Kubernetes and Helm
+Make sure that you have a running Kubernetes cluster and a `kubectl`
+client connected to this cluster as described in the [Requriements section](../README.md#requirements) of the main README.md.
-Make sure that you have a running Kubernetes cluster and a `kubectl` client connected to this cluster.
+### Step 2. Install and configure Helm
Then please install [Helm](https://github.com/kubernetes/helm) and run the following command to init `Helm Tiller`:
```shell
@@ -63,14 +39,26 @@ Then grant corresponding cluster role to `Helm` user:
kubectl create clusterrolebinding tiller-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default
```
-Finally, create a new namespace named `openwhisk`:
+### Step 3. Prepare your cluster for OpenWhisk
+
+1. Create a new namespace named `openwhisk`:
```shell
kubectl create namespace openwhisk
```
-Now, you should have an available environment to deploy OpenWhisk!
+2. Identify the Kubernetes worker nodes that should be used to execute
+user containers. Do this by labeling each node with
+`openwhisk-role=invoker`. For a single node cluster, simply do
+```shell
+kubectl label nodes --all openwhisk-role=invoker
+```
+If you have a multi-node cluster, for each node <INVOKER_NODE_NAME>
+you want to be an invoker, execute
+```shell
+$ kubectl label nodes <INVOKER_NODE_NAME> openwhisk-role=invoker
+```
-### Deploy Charts
+### Step 4. Deploy Charts
Deployment can be done by using the following single command:
```shell
@@ -86,11 +74,11 @@ kubectl get pods -n openwhisk
Install an [OpenWhisk client](https://github.com/apache/incubator-openwhisk/tree/master/docs) to test the deployed OpenWhisk environment.
-For now, we are using nginx to provide web access for OpenWhisk client. By default, the nginx service is configured to run at port 30000 for HTTP connection and 30001 for HTTPS connection.
+For now, we are using nginx to provide web access for OpenWhisk client. By default, the nginx service is configured to run at port 31000 for HTTP connection and 31001 for HTTPS connection.
As a result, please run the following command to config your OpenWhisk client:
```shell
-wsk property set --apihost http://<nginx_node_IP>:30000
+wsk property set --apihost http://<nginx_node_IP>:31000
wsk property set --auth 23bc46b1-71f6-4ed5-8c54-816aa4f8c502:123zO3xZCLrMN6v2BKK1dXYFpXlPkccOFqm12CdAsMgRU4VrNZ9lyGVCGuMDGIwP
```
@@ -132,6 +120,3 @@ or with a `--purge` option if you want to completely remove the deployment from
helm delete <release_name> --purge
```
-## References
-Part of this chart is based on the project [OpenWhisk Deployment for Kubernetes](https://github.com/apache/incubator-openwhisk-deploy-kube).
-This is a project using [ansible](https://www.ansible.com) to deploy OpenWhisk on Kubernetes. Please visit the project on github to get more details.
diff --git a/helm/charts/couchdb/templates/deployment.yaml b/helm/charts/couchdb/templates/deployment.yaml
index 010a196..a98775a 100644
--- a/helm/charts/couchdb/templates/deployment.yaml
+++ b/helm/charts/couchdb/templates/deployment.yaml
@@ -16,14 +16,8 @@ spec:
{{- if .Values.global.affinity.enabled }}
affinity:
- nodeAffinity:
- requiredDuringSchedulingIgnoredDuringExecution:
- nodeSelectorTerms:
- - matchExpressions:
- - key: nodeType
- operator: In
- values:
- - {{ .Values.global.affinity.kafkaNodeLabel }}
+{{ include "affinity.controlPlane" . | indent 8 }}
+{{ include "affinity.selfAntiAffinity" ( include "couchdb_deployment_name" . ) | indent 8 }}
{{- end }}
containers:
@@ -50,9 +44,7 @@ spec:
readinessProbe:
httpGet:
port: {{ template "couchdb_port" . }}
- # Tempting to say "/$(DB_PREFIX)_activations", but probe runs
- # externally, therefore can't access the container environment...
- path: "/test_activations"
+ path: /{{ template "activations_table_unquoted" . }}
initialDelaySeconds: 60
periodSeconds: 10
failureThreshold: 10
diff --git a/helm/charts/kafka/templates/service.yaml b/helm/charts/kafka/templates/service.yaml
index a3d012b..d7f00cb 100644
--- a/helm/charts/kafka/templates/service.yaml
+++ b/helm/charts/kafka/templates/service.yaml
@@ -7,6 +7,5 @@ spec:
ports:
- name: broker
port: {{ template "kafka_port" . }}
- clusterIP: None
selector:
app: {{ template "kafka_statefulset_name" . }}
diff --git a/helm/charts/kafka/templates/statefulset.yaml b/helm/charts/kafka/templates/statefulset.yaml
index b349f45..33681e0 100644
--- a/helm/charts/kafka/templates/statefulset.yaml
+++ b/helm/charts/kafka/templates/statefulset.yaml
@@ -13,21 +13,12 @@ spec:
spec:
{{- if .Values.global.affinity.enabled }}
affinity:
- nodeAffinity:
- requiredDuringSchedulingIgnoredDuringExecution:
- nodeSelectorTerms:
- - matchExpressions:
- - key: nodeType
- operator: In
- values:
- - {{ .Values.global.affinity.kafkaNodeLabel }}
+{{ include "affinity.controlPlane" . | indent 8 }}
+{{ include "affinity.selfAntiAffinity" ( include "kafka_statefulset_name" . ) | indent 8 }}
{{- end }}
initContainers:
- - name: "wait-for-zookeeper"
- image: "busybox"
- imagePullPolicy: "IfNotPresent"
- command: ["sh", "-c", "result=1; until [ $result -eq 0 ]; do nslookup {{ .Values.global.zookeeperServiceName}}.{{ .Release.Namespace }}; result=$?; echo waiting for {{ .Values.global.zookeeperServiceName}}.{{ .Release.Namespace }}; sleep 1; done;"]
+{{ include "readiness.waitForZookeeper" . | indent 6 }}
containers:
- name: {{ template "kafka_statefulset_name" . }}
diff --git a/helm/charts/nginx/templates/deployment.yaml b/helm/charts/nginx/templates/deployment.yaml
index 2e790b5..d0a7ad0 100644
--- a/helm/charts/nginx/templates/deployment.yaml
+++ b/helm/charts/nginx/templates/deployment.yaml
@@ -11,17 +11,15 @@ spec:
metadata:
labels:
name: {{ template "nginx_deployment_name" . }}
- annotations:
- pod.beta.kubernetes.io/init-containers: '[
- {
- "name": "wait-for-controller",
- "image": "busybox",
- "imagePullPolicy": "IfNotPresent",
- "command": ["sh", "-c", "until nslookup {{ template "controller_url_without_port" . }}; do echo waiting for {{ template "controller_url_without_port" . }}; sleep 1; done;"]
- }
- ]'
spec:
restartPolicy: Always
+
+ {{- if .Values.global.affinity.enabled }}
+ affinity:
+{{ include "affinity.controlPlane" . | indent 8 }}
+{{ include "affinity.selfAntiAffinity" ( include "nginx_deployment_name" . ) | indent 8 }}
+ {{- end }}
+
volumes:
- name: nginx-certs
secret:
@@ -31,6 +29,7 @@ spec:
name: {{ template "nginx_configmap_name" . }}
- name: logs
emptyDir: {}
+
containers:
- name: {{ template "nginx_deployment_name" . }}
imagePullPolicy: {{ .Values.imagePullPolicy | default "IfNotPresent" | quote }}
diff --git a/helm/charts/redis/Chart.yaml b/helm/charts/redis/Chart.yaml
deleted file mode 100644
index 249a3d9..0000000
--- a/helm/charts/redis/Chart.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-apiVersion: v1
-description: A Helm chart to deploy redis service for Kubernetes
-name: redis
-version: 0.1.0
-keywords:
- - redis
-maintainers:
- - name: Tom Xing
- email: xingzhou@cn.ibm.com
diff --git a/helm/charts/redis/templates/NOTES.txt b/helm/charts/redis/templates/NOTES.txt
deleted file mode 100644
index 29863d3..0000000
--- a/helm/charts/redis/templates/NOTES.txt
+++ /dev/null
@@ -1,8 +0,0 @@
-Thank you for installing {{ .Chart.Name }}.
-
-Your release is named {{ .Release.Name }}.
-
-To learn more about the release, try:
-
- $ helm status {{ .Release.Name }}
- $ helm get {{ .Release.Name }}
diff --git a/helm/charts/redis/templates/_helpers.tpl b/helm/charts/redis/templates/_helpers.tpl
deleted file mode 100644
index d0c0826..0000000
--- a/helm/charts/redis/templates/_helpers.tpl
+++ /dev/null
@@ -1,14 +0,0 @@
-{ Get redis service name */}}
-{{- define "redis_service_name" -}}
-{{ .Values.global.redisServiceName | default "redis" | quote }}
-{{- end -}}
-
-{ Get redis deployment name */}}
-{{- define "redis_deployment_name" -}}
-{{ .Values.deploymentName | default "redis" | quote }}
-{{- end -}}
-
-{{/* Set port */}}
-{{- define "redis_port" -}}
-{{ .Values.global.redisServicePort | default 6379 }}
-{{- end -}}
diff --git a/helm/charts/redis/templates/deployment.yaml b/helm/charts/redis/templates/deployment.yaml
deleted file mode 100644
index f2634e2..0000000
--- a/helm/charts/redis/templates/deployment.yaml
+++ /dev/null
@@ -1,23 +0,0 @@
-apiVersion: extensions/v1beta1
-kind: Deployment
-metadata:
- name: {{ template "redis_deployment_name" . }}
- namespace: {{ .Release.Namespace | quote }}
- labels:
- name: {{ template "redis_deployment_name" . }}
-spec:
- replicas: {{ .Values.replicaCount | default 1 }}
- template:
- metadata:
- labels:
- name: {{ template "redis_deployment_name" . }}
- spec:
- restartPolicy: Always
-
- containers:
- - name: redis
- imagePullPolicy: {{ .Values.imagePullPolicy | default "IfNotPresent" | quote }}
- image: {{ .Values.image | default "redis:3.2" | quote }}
- ports:
- - name: redis-port
- containerPort: {{ template "redis_port" . }}
\ No newline at end of file
diff --git a/helm/charts/redis/templates/service.yaml b/helm/charts/redis/templates/service.yaml
deleted file mode 100644
index 3c7649e..0000000
--- a/helm/charts/redis/templates/service.yaml
+++ /dev/null
@@ -1,14 +0,0 @@
-apiVersion: v1
-kind: Service
-metadata:
- name: {{ template "redis_service_name" . }}
- namespace: {{ .Release.Namespace | quote }}
- labels:
- name: {{ template "redis_service_name" . }}
-spec:
- selector:
- name: {{ template "redis_deployment_name" . }}
- ports:
- - port: {{ template "redis_port" . }}
- targetPort: {{ template "redis_port" . }}
- name: redis
\ No newline at end of file
diff --git a/helm/charts/zookeeper/templates/deployment.yaml b/helm/charts/zookeeper/templates/deployment.yaml
index 39f0482..40f01d5 100644
--- a/helm/charts/zookeeper/templates/deployment.yaml
+++ b/helm/charts/zookeeper/templates/deployment.yaml
@@ -16,14 +16,8 @@ spec:
{{- if .Values.global.affinity.enabled }}
affinity:
- nodeAffinity:
- requiredDuringSchedulingIgnoredDuringExecution:
- nodeSelectorTerms:
- - matchExpressions:
- - key: nodeType
- operator: In
- values:
- - {{ .Values.global.affinity.kafkaNodeLabel }}
+{{ include "affinity.controlPlane" . | indent 8 }}
+{{ include "affinity.selfAntiAffinity" ( include "zookeeper_deployment_name" . ) | indent 8 }}
{{- end }}
containers:
diff --git a/helm/templates/_affinity.tpl b/helm/templates/_affinity.tpl
new file mode 100644
index 0000000..6f12a73
--- /dev/null
+++ b/helm/templates/_affinity.tpl
@@ -0,0 +1,54 @@
+# This file defines template snippets for scheduler affinity and anti-affinity
+
+{{/* Generic control-plane affinity */}}
+{{- define "affinity.controlPlane" -}}
+# prefer to not run on an invoker node (only prefer because of single node clusters)
+nodeAffinity:
+ preferredDuringSchedulingIgnoredDuringExecution:
+ - weight: 100
+ preference:
+ matchExpressions:
+ - key: openwhisk-role
+ operator: NotIn
+ values:
+ - {{ .Values.global.affinity.invokerNodeLabel }}
+# prefer to run on a control-plane node
+nodeAffinity:
+ preferredDuringSchedulingIgnoredDuringExecution:
+ - weight: 80
+ preference:
+ matchExpressions:
+ - key: openwhisk-role
+ operator: In
+ values:
+ - {{ .Values.global.affinity.controlPlaneNodeLabel }}
+{{- end -}}
+
+
+{{/* Invoker node affinity */}}
+{{- define "affinity.invoker" -}}
+# run only on nodes labeled with openwhisk-role={{ .Values.global.affinity.invokerNodeLabel }}
+nodeAffinity:
+ requiredDuringSchedulingIgnoredDuringExecution:
+ nodeSelectorTerms:
+ - matchExpressions:
+ - key: openwhisk-role
+ operator: In
+ values:
+ - {{ .Values.global.affinity.invokerNodeLabel }}
+{{- end -}}
+
+
+{{/* Self anti-affinity */}}
+{{- define "affinity.selfAntiAffinity" -}}
+# Fault tolerance: prevent multiple instances of {{ . }} from running on the same node
+podAntiAffinity:
+ requiredDuringSchedulingIgnoredDuringExecution:
+ - labelSelector:
+ matchExpressions:
+ - key: name
+ operator: In
+ values:
+ - {{ . }}
+ topologyKey: "kubernetes.io/hostname"
+{{- end -}}
\ No newline at end of file
diff --git a/helm/templates/_helpers.tpl b/helm/templates/_helpers.tpl
index 0a74efa..d938c46 100644
--- a/helm/templates/_helpers.tpl
+++ b/helm/templates/_helpers.tpl
@@ -18,6 +18,11 @@
{{ .Values.activationsTable | default "test_activations" | quote }}
{{- end -}}
+{{/* Set activations table */}}
+{{- define "activations_table_unquoted" -}}
+{{ .Values.activationsTable | default "test_activations" }}
+{{- end -}}
+
{{/* Set actions table */}}
{{- define "actions_table" -}}
{{ .Values.actionsTable | default "test_whisks" | quote }}
@@ -28,9 +33,9 @@
{{ .Values.authTable | default "test_subjects" | quote }}
{{- end -}}
-{{/* Set invoker statefulset name */}}
-{{- define "invoker_statefulset_name" -}}
-{{ .Values.invokerStatefulsetName | default "invoker" | quote }}
+{{/* Set invoker "deployment" name */}}
+{{- define "invoker_deployment_name" -}}
+{{ .Values.invokerDeploymentName | default "invoker" | quote }}
{{- end -}}
{{/* Generate kafka url without port */}}
@@ -79,11 +84,6 @@
{{ .Values.deploymentName | default "apigateway" | quote }}
{{- end -}}
-{{/* Generate redis service url */}}
-{{- define "redis_url" -}}
-{{ .Values.global.redisServiceName | default "redis" }}.{{ .Release.Namespace }}
-{{- end -}}
-
{{/* Runtimes manifest */}}
{{- define "runtimes_manifest" -}}
{{- if .Values.global.travis -}}
@@ -91,4 +91,26 @@
{{- else -}}
{{ .Files.Get "runtimes.json" | quote }}
{{- end -}}
+{{- end -}}
+
+{{/* Environment variables required for accessing CouchDB */}}
+{{- define "whisk.dbEnvVars" -}}
+- name: "CONFIG_whisk_couchdb_username"
+ value: {{ template "couchdb_username" . }}
+- name: "CONFIG_whisk_couchdb_password"
+ value: {{ template "couchdb_password" . }}
+- name: "CONFIG_whisk_couchdb_port"
+ value: {{ include "couchdb_port" . | quote}}
+- name: "CONFIG_whisk_couchdb_protocol"
+ value: "http"
+- name: "CONFIG_whisk_couchdb_host"
+ value: {{ include "couchdb_url_without_port" . | quote }}
+- name: "CONFIG_whisk_couchdb_provider"
+ value: "CouchDB"
+- name: "CONFIG_whisk_couchdb_databases_WhiskActivation"
+ value: {{ template "activations_table" . }}
+- name: "CONFIG_whisk_couchdb_databases_WhiskEntity"
+ value: {{ template "actions_table" . }}
+- name: "CONFIG_whisk_couchdb_databases_WhiskAuth"
+ value: {{ template "auths_table" . }}
{{- end -}}
\ No newline at end of file
diff --git a/helm/templates/_readiness.tpl b/helm/templates/_readiness.tpl
new file mode 100644
index 0000000..3f9c844
--- /dev/null
+++ b/helm/templates/_readiness.tpl
@@ -0,0 +1,38 @@
+{{/* Init container that waits for couchdb to be ready */}}
+{{- define "readiness.waitForCouchDB" -}}
+- name: "wait-for-couchdb"
+ image: "busybox"
+ imagePullPolicy: "IfNotPresent"
+ env:
+ - name: "READINESS_URL"
+ value: http://{{ .Values.global.couchdbServiceName }}.{{ .Release.Namespace }}.svc.cluster.local:{{ .Values.global.couchdbPort }}/{{ template "activations_table_unquoted" . }}
+ command: ["sh", "-c", "result=1; until [ $result -eq 0 ]; do echo verifying CouchDB readiness; wget -T 5 --spider $READINESS_URL; result=$?; sleep 1; done;"]
+{{- end -}}
+
+{{/* Init container that waits for kafka to be ready */}}
+{{- define "readiness.waitForKafka" -}}
+- name: "wait-for-kafka"
+ image: "busybox"
+ imagePullPolicy: "IfNotPresent"
+ # TODO: I haven't found an easy external test to determine that kafka is up, so as a hack we wait for zookeeper and then sleep for 10 seconds and cross our fingers!
+ command: ["sh", "-c", 'result=1; until [ $result -eq 0 ]; do OK=$(echo ruok | nc -w 1 {{ .Values.global.zookeeperServiceName}}.{{ .Release.Namespace }}.svc.cluster.local {{ template "zookeeper_port" . }}); if [ "$OK" == "imok" ]; then result=0; fi; echo waiting for zookeeper to be ready; sleep 1; done; echo zookeeper is up, sleeping for 10 seconds; sleep 10;']
+{{- end -}}
+
+{{/* Init container that waits for zookeeper to be ready */}}
+{{- define "readiness.waitForZookeeper" -}}
+- name: "wait-for-zookeeper"
+ image: "busybox"
+ imagePullPolicy: "IfNotPresent"
+ command: ["sh", "-c", 'result=1; until [ $result -eq 0 ]; do OK=$(echo ruok | nc -w 1 {{ .Values.global.zookeeperServiceName}}.{{ .Release.Namespace }}.svc.cluster.local {{ template "zookeeper_port" . }}); if [ "$OK" == "imok" ]; then result=0; fi; echo waiting for zookeeper to be ready; sleep 1; done']
+{{- end -}}
+
+{{/* Init container that waits for controller to be ready */}}
+{{- define "readiness.waitForController" -}}
+- name: "wait-for-controller"
+ image: "busybox"
+ imagePullPolicy: "IfNotPresent"
+ env:
+ - name: "READINESS_URL"
+ value: http://{{ .Values.global.controllerServiceName }}.{{ .Release.Namespace }}.svc.cluster.local:{{ .Values.global.controllerPort }}/ping
+ command: ["sh", "-c", "result=1; until [ $result -eq 0 ]; do echo verifying controller readiness; wget -T 5 --spider $READINESS_URL; result=$?; sleep 1; done;"]
+{{- end -}}
diff --git a/helm/templates/apigateway_deployment.yml b/helm/templates/apigateway_deployment.yaml
similarity index 51%
rename from helm/templates/apigateway_deployment.yml
rename to helm/templates/apigateway_deployment.yaml
index f7cd6ee..913b8e2 100644
--- a/helm/templates/apigateway_deployment.yml
+++ b/helm/templates/apigateway_deployment.yaml
@@ -1,4 +1,3 @@
-{{- if .Values.apigatwayEnabled }}
---
apiVersion: v1
kind: Service
@@ -35,18 +34,27 @@ spec:
spec:
restartPolicy: Always
+ {{- if .Values.global.affinity.enabled }}
+ affinity:
+{{ include "affinity.controlPlane" . | indent 8 }}
+{{ include "affinity.selfAntiAffinity" ( include "apigateway_deployment_name" . ) | indent 8 }}
+ {{- end }}
+
containers:
- - name: apigateway
- imagePullPolicy: {{ .Values.apigatewayImagePullPolicy | default "IfNotPresent" | quote }}
- image: {{ .Values.apigatewayImage | default "openwhisk/apigateway" | quote }}
- ports:
- - name: mgmt
- containerPort: {{ template "apigateway_mgmt_port" . }}
- - name: api
- containerPort: {{ template "apigateway_api_port" . }}
- env:
- - name: "REDIS_HOST"
- value: {{ include "redis_url" . | quote }}
- - name: "REDIS_PORT"
- value: {{ .Values.global.redisPort | default "6379" | quote }}
-{{ end }}
+ - name: redis
+ imagePullPolicy: {{ .Values.apigatewayImagePullPolicy | default "IfNotPresent" | quote }}
+ image: redis:3.2
+
+ - name: apigateway
+ imagePullPolicy: {{ .Values.apigatewayImagePullPolicy | default "IfNotPresent" | quote }}
+ image: {{ .Values.apigatewayImage | default "openwhisk/apigateway" | quote }}
+ ports:
+ - name: mgmt
+ containerPort: {{ template "apigateway_mgmt_port" . }}
+ - name: api
+ containerPort: {{ template "apigateway_api_port" . }}
+ env:
+ - name: "REDIS_HOST"
+ value: "127.0.0.1"
+ - name: "REDIS_PORT"
+ value: "6379"
diff --git a/helm/templates/controller_statefulset.yml b/helm/templates/controller_statefulset.yaml
similarity index 68%
rename from helm/templates/controller_statefulset.yml
rename to helm/templates/controller_statefulset.yaml
index c0d7898..aea535f 100644
--- a/helm/templates/controller_statefulset.yml
+++ b/helm/templates/controller_statefulset.yaml
@@ -9,7 +9,6 @@ metadata:
spec:
selector:
name: {{ template "controller_statefulset_name" . }}
- clusterIP: None
ports:
- port: {{ template "controller_port" . }}
targetPort: {{ template "controller_port" . }}
@@ -34,30 +33,14 @@ spec:
{{- if .Values.global.affinity.enabled }}
affinity:
- nodeAffinity:
- requiredDuringSchedulingIgnoredDuringExecution:
- nodeSelectorTerms:
- - matchExpressions:
- - key: nodeType
- operator: In
- values:
- - {{ .Values.global.affinity.controllerNodeLabel }}
- podAntiAffinity:
- requiredDuringSchedulingIgnoredDuringExecution:
- - labelSelector:
- matchExpressions:
- - key: name
- operator: In
- values:
- - {{ template "controller_statefulset_name" . }}
- topologyKey: "kubernetes.io/hostname"
+{{ include "affinity.controlPlane" . | indent 8 }}
+{{ include "affinity.selfAntiAffinity" ( include "controller_statefulset_name" . ) | indent 8 }}
{{- end }}
initContainers:
- - name: "wait-for-kafka"
- image: "busybox"
- imagePullPolicy: "IfNotPresent"
- command: ["sh", "-c", "result=1; until [ $result -eq 0 ]; do nslookup {{ .Values.global.kafkaServiceName}}.{{ .Release.Namespace }}; result=$?; echo waiting for {{ .Values.global.kafkaServiceName}}.{{ .Release.Namespace }}; sleep 1; done;"]
+ # The controller must wait for kafka and couchdb to be ready before it starts
+{{ include "readiness.waitForKafka" . | indent 6 }}
+{{ include "readiness.waitForCouchDB" . | indent 6 }}
containers:
- name: {{ template "controller_statefulset_name" . }}
@@ -111,24 +94,7 @@ spec:
value: ""
# properties for DB connection
- - name: "CONFIG_whisk_couchdb_username"
- value: {{ template "couchdb_username" . }}
- - name: "CONFIG_whisk_couchdb_password"
- value: {{ template "couchdb_password" . }}
- - name: "CONFIG_whisk_couchdb_port"
- value: {{ include "couchdb_port" . | quote}}
- - name: "CONFIG_whisk_couchdb_protocol"
- value: "http"
- - name: "CONFIG_whisk_couchdb_host"
- value: {{ include "couchdb_url_without_port" . | quote }}
- - name: "CONFIG_whisk_couchdb_provider"
- value: "CouchDB"
- - name: "CONFIG_whisk_couchdb_databases_WhiskActivation"
- value: {{ template "activations_table" . }}
- - name: "CONFIG_whisk_couchdb_databases_WhiskEntity"
- value: {{ template "actions_table" . }}
- - name: "CONFIG_whisk_couchdb_databases_WhiskAuth"
- value: {{ template "auths_table" . }}
+{{ include "whisk.dbEnvVars" . | indent 8 }}
- name: "LIMITS_ACTIONS_SEQUENCE_MAXLENGTH"
value: {{ .Values.controllerLimitsActionsSequenceMaxlength | default 1000 | quote }}
diff --git a/helm/templates/invoker_deployment.yml b/helm/templates/invoker_daemonset.yaml
similarity index 55%
rename from helm/templates/invoker_deployment.yml
rename to helm/templates/invoker_daemonset.yaml
index 8d62f5f..1e5c380 100644
--- a/helm/templates/invoker_deployment.yml
+++ b/helm/templates/invoker_daemonset.yaml
@@ -1,20 +1,21 @@
apiVersion: extensions/v1beta1
-kind: Deployment
+kind: DaemonSet
metadata:
- name: {{ template "invoker_statefulset_name" . }}
+ name: {{ template "invoker_deployment_name" . }}
namespace: {{ .Release.Namespace | quote }}
labels:
- name: {{ template "invoker_statefulset_name" . }}
+ name: {{ template "invoker_deployment_name" . }}
spec:
- replicas: {{ .Values.invokerReplicasetCount | default 2 }}
- #serviceName: {{ template "invoker_statefulset_name" . }}
template:
metadata:
labels:
- name: {{ template "invoker_statefulset_name" . }}
+ name: {{ template "invoker_deployment_name" . }}
spec:
restartPolicy: Always
+ affinity:
+{{ include "affinity.invoker" . | indent 8 }}
+
volumes:
- name: cgroup
hostPath:
@@ -28,32 +29,12 @@ spec:
- name: dockersock
hostPath:
path: "/var/run/docker.sock"
- - name: apparmor
- hostPath:
- path: "/usr/lib/x86_64-linux-gnu/libapparmor.so.1"
- {{- if .Values.global.affinity.enabled }}
- affinity:
- nodeAffinity:
- requiredDuringSchedulingIgnoredDuringExecution:
- nodeSelectorTerms:
- - matchExpressions:
- - key: nodeType
- operator: In
- values:
- - {{ .Values.global.affinity.invokerNodeLabel }}
- podAntiAffinity:
- requiredDuringSchedulingIgnoredDuringExecution:
- - labelSelector:
- matchExpressions:
- - key: name
- operator: In
- values:
- - {{ template "invoker_statefulset_name" . }}
- topologyKey: "kubernetes.io/hostname"
- {{- end }}
-
initContainers:
+ # Wait for a controller to be up (which implies kafka, zookeeper, couchdb are all up as well).
+{{ include "readiness.waitForController" . | indent 6 }}
+
+ # Pull images for all default runtimes before starting invoker
- name: docker-pull-runtimes
imagePullPolicy: "IfNotPresent"
image: openwhisk/kube-docker-pull
@@ -65,13 +46,8 @@ spec:
- name: "RUNTIMES_MANIFEST"
value: {{ template "runtimes_manifest" . }}
- - name: "wait-for-kafka"
- image: "busybox"
- imagePullPolicy: "IfNotPresent"
- command: ["sh", "-c", "result=1; until [ $result -eq 0 ]; do nslookup {{ .Values.global.kafkaServiceName}}.{{ .Release.Namespace }}; result=$?; echo waiting for {{ .Values.global.kafkaServiceName}}.{{ .Release.Namespace }}; sleep 1; done;"]
-
containers:
- - name: {{ template "invoker_statefulset_name" . }}
+ - name: {{ template "invoker_deployment_name" . }}
imagePullPolicy: {{ .Values.invokerImagePullPolicy | default "IfNotPresent" | quote }}
image: {{ .Values.invokerImage | default "openwhisk/invoker" | quote }}
command: [ "/bin/bash", "-c", "COMPONENT_NAME=$(hostname | cut -d'-' -f2) /init.sh"]
@@ -97,10 +73,11 @@ spec:
- name: "DOCKER_REGISTRY"
value: ""
+ # Invoker name is name of the Kube node when using DaemonSet
- name: "INVOKER_NAME"
valueFrom:
fieldRef:
- fieldPath: metadata.name
+ fieldPath: spec.nodeName
# Java options
- name: "JAVA_OPTS"
@@ -123,26 +100,13 @@ spec:
value: "2017-01-01T00:00:00Z"
# properties for DB connection
- - name: "CONFIG_whisk_couchdb_username"
- value: {{ template "couchdb_username" . }}
- - name: "CONFIG_whisk_couchdb_password"
- value: {{ template "couchdb_password" . }}
- - name: "CONFIG_whisk_couchdb_port"
- value: {{ include "couchdb_port" . | quote }}
- - name: "CONFIG_whisk_couchdb_protocol"
- value: "http"
- - name: "CONFIG_whisk_couchdb_host"
- value: {{ include "couchdb_url_without_port" . | quote }}
- - name: "CONFIG_whisk_couchdb_provider"
- value: "CouchDB"
- - name: "CONFIG_whisk_couchdb_databases_WhiskActivation"
- value: {{ template "activations_table" . }}
- - name: "CONFIG_whisk_couchdb_databases_WhiskEntity"
- value: {{ template "actions_table" . }}
- - name: "CONFIG_whisk_couchdb_databases_WhiskAuth"
- value: {{ template "auths_table" . }}
+{{ include "whisk.dbEnvVars" . | indent 10 }}
+
+ # properties for kafka connection
- name: "KAFKA_HOSTS"
value: "{{ .Values.global.kafkaServiceName }}.{{ .Release.Namespace }}:{{ .Values.global.kafkaPort | default 9092 }}"
+
+ # properties for zookeeper connection
- name: "ZOOKEEPER_HOSTS"
value: "{{ .Values.global.zookeeperServiceName }}.{{ .Release.Namespace }}:{{ .Values.global.zookeeperPort | default 2181 }}"
@@ -158,5 +122,3 @@ spec:
mountPath: "/var/run/docker.sock"
- name: dockerrootdir
mountPath: "/containers"
- - name: apparmor
- mountPath: "/usr/lib/x86_64-linux-gnu/libapparmor.so.1"
diff --git a/helm/values.yaml b/helm/values.yaml
index 2928e06..fa1bb11 100644
--- a/helm/values.yaml
+++ b/helm/values.yaml
@@ -33,8 +33,6 @@ couchdb:
size: 8Gi
storageClass: CouchDBSC
accessMode: ReadWriteOnce
- # nfs_path: /home/nfs
- # nfs_server: 172.20.5.175
# Nginx configurations
@@ -46,21 +44,13 @@ nginx:
image: "nginx:1.11"
imagePullPolicy: "IfNotPresent"
httpPort: 80
- httpNodePort: 30000
+ httpNodePort: 31000
httpsPort: 443
- httpsNodePort: 30001
+ httpsNodePort: 31001
httpsAdminPort: 8443
- httpsAdminNodePort: 30002
+ httpsAdminNodePort: 31002
serviceName: "nginx"
-
-# Redis configurations
-redis:
- deploymentName: "redis"
- replicaCount: 1
- image: redis:3.2
- imagePullPolicy: "IfNotPresent"
-
# Controller configurations
controllerImagePullPolicy: "IfNotPresent"
controllerImage: "openwhisk/controller"
@@ -81,7 +71,6 @@ invokerImage: "openwhisk/invoker"
invokerOptions: ""
# API Gateway configurations
-apigatewayEnabled: false
apigatewayReplicaCount: 1
activationsTable: "test_activations"
@@ -107,7 +96,6 @@ global:
redisServicePort: 6379
# Resolve the pod/node affinity for invoker, controller and other components.
affinity:
- enabled: false
+ enabled: true
invokerNodeLabel: invoker
- controllerNodeLabel: controller
- kafkaNodeLabel: kafka
+ controlPlaneNodeLabel: control-plane
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services