You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@openwhisk.apache.org by ra...@apache.org on 2017/11/27 16:35:05 UTC
[incubator-openwhisk-deploy-kube] branch master updated: Update
READMEs and kafka image to exploit PR-2948 (#89)
This is an automated email from the ASF dual-hosted git repository.
rabbah pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk-deploy-kube.git
The following commit(s) were added to refs/heads/master by this push:
new d0d1340 Update READMEs and kafka image to exploit PR-2948 (#89)
d0d1340 is described below
commit d0d134083f4384cc6a815cdb6c40044deb0656f5
Author: David Grove <dg...@users.noreply.github.com>
AuthorDate: Mon Nov 27 11:35:01 2017 -0500
Update READMEs and kafka image to exploit PR-2948 (#89)
The upstream PR-2948 enabled lazy creation of the
invokerN and completedN topics, so they no longer
needed to be created by the init.sh script in the
kafka image. In conjunction with the change in
PR-85 to switch to active-active controllers, this
means that changing the number of deployed invokers
or controllers is localized to just the invoker.yml
or controller.yml file.
Also switch custom kafka image to dgrove/whisk_kafka
for easier updates and version bump parent image
to ches/kafka 0.10.2.1 to match openwhisk upstream.
---
kubernetes/controller/README.md | 15 +++++----------
kubernetes/invoker/README.md | 4 ++--
kubernetes/kafka/README.md | 27 ---------------------------
kubernetes/kafka/docker/Dockerfile | 2 +-
kubernetes/kafka/docker/init.sh | 22 ----------------------
kubernetes/kafka/kafka.yml | 24 ++----------------------
6 files changed, 10 insertions(+), 84 deletions(-)
diff --git a/kubernetes/controller/README.md b/kubernetes/controller/README.md
index 5976d4f..755c154 100644
--- a/kubernetes/controller/README.md
+++ b/kubernetes/controller/README.md
@@ -15,15 +15,10 @@ kubectl apply -f controller.yml
# Controller Deployment Changes
## Increase Controller Count
-If you want to change the number of controllers deployed,
-you will also need to update properties for Kafka.
-
-* Kafka: Look at the Kafka [README](https://github.com/apache/incubator-openwhisk-deploy-kube/blob/master/kubernetes/kafka/README.md)
-
-* Controller: You will need to update the replication count for the
- Controllers [here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L25)
- and the value of CONTROLLER_INSTANCES [here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L80)
- and the value of AKKA_CLUSTER_SEED_NODES [here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L110)
- and redeploy.
+You will need to update the replication count for the
+Controllers [here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L26)
+and the value of CONTROLLER_INSTANCES [here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L82)
+and the value of AKKA_CLUSTER_SEED_NODES [here](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/controller/controller.yml#L112)
+and redeploy.
[StatefulSet]: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/
diff --git a/kubernetes/invoker/README.md b/kubernetes/invoker/README.md
index 15d6d6e..82bed6b 100644
--- a/kubernetes/invoker/README.md
+++ b/kubernetes/invoker/README.md
@@ -28,9 +28,9 @@ section below.
## Increase Invoker Count
To increase the number of Invokers, edit the
-[replicas](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/invoker/invoker.yml#L27)
+[replicas](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/invoker/invoker.yml#L9)
line. Secondly, you will need to update the
-[INVOKER_COUNT](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/invoker/invoker.yml#L63-L64)
+[INVOKER_INSTANCES](https://github.com/apache/incubator-openwhisk-deploy-kube/tree/master/kubernetes/invoker/invoker.yml#L70)
to with the same replica count.
## Deploying Invoker to Specific Kube Nodes
diff --git a/kubernetes/kafka/README.md b/kubernetes/kafka/README.md
index cf91aaf..f6ffc6d 100644
--- a/kubernetes/kafka/README.md
+++ b/kubernetes/kafka/README.md
@@ -12,33 +12,6 @@ just run:
kubectl apply -f kafka.yml
```
-# Deployment Changes
-## Increase Invoker Pods
-
-When updating the invoker pod count you will need to update some Kafka
-and Invoker properties.
-
-* Kafka: The ["INVOKER_COUNT"](https://github.com/apache/incubator-openwhisk-deploy-kube/blob/master/kubernetes/kafka/kafka.yml#L73)
- property will need to equal the number of Invokers being deployed
- and then you need to redeploy Kafka so that the new `invokerN`
- topics are created.
-
-* Invoker: See the Invoker [README](https://github.com/apache/incubator-openwhisk-deploy-kube/blob/master/kubernetes/invoker/README.md)
-
-## Increase Controller Pods
-
-When updating the Controller pod count, you will need to update the
-Kafka, Controller and Nginx deployments.
-
-* Kafka: The ["CONTROLLER_COUNT"](https://github.com/apache/incubator-openwhisk-deploy-kube/blob/master/kubernetes/kafka/kafka.yml#L63)
- property will need to equal the number of Controllers being deployed
- and then you need to redeploy Kafka so that the new `completedN`
- topics are created.
-
-* Controller: See the Controller [README](https://github.com/apache/incubator-openwhisk-deploy-kube/blob/master/kubernetes/controller/README.md)
-
-* Nginx: See the Nginx [README](https://github.com/apache/incubator-openwhisk-deploy-kube/blob/master/kubernetes/nginx/README.md#increase-controller-count)
-
# Troubleshooting
## Networking errors
diff --git a/kubernetes/kafka/docker/Dockerfile b/kubernetes/kafka/docker/Dockerfile
index fbdfab2..99c6aa7 100644
--- a/kubernetes/kafka/docker/Dockerfile
+++ b/kubernetes/kafka/docker/Dockerfile
@@ -1,4 +1,4 @@
-FROM ches/kafka:0.10.0.1
+FROM ches/kafka:0.10.2.1
COPY init.sh /init.sh
diff --git a/kubernetes/kafka/docker/init.sh b/kubernetes/kafka/docker/init.sh
index 4d9e9b0..1ab2750 100755
--- a/kubernetes/kafka/docker/init.sh
+++ b/kubernetes/kafka/docker/init.sh
@@ -36,26 +36,4 @@ if ! ([[ "$OUTPUT" == *"already exists"* ]] || [[ "$OUTPUT" == *"Created topic"*
exit 1
fi
-echo "Create completed topics"
-CONTROLLER_COUNT=$((CONTROLLER_COUNT - 1))
-for i in `seq 0 $CONTROLLER_COUNT`; do
- OUTPUT=$(kafka-topics.sh --create --topic completed$i --replication-factor $REPLICATION_FACTOR --partitions $PARTITIONS --zookeeper ${ZOOKEEPER_HOST}:${ZOOKEEPER_PORT} --config retention.bytes=$KAFKA_TOPICS_COMPLETED_RETENTIONBYTES --config retention.ms=$KAFKA_TOPICS_COMPLETED_RETENTIONMS --config segment.bytes=$KAFKA_TOPICS_COMPLETED_SEGMENTBYTES)
-
- if ! ([[ "$OUTPUT" == *"already exists"* ]] || [[ "$OUTPUT" == *"Created topic"* ]]); then
- echo "Failed to create completed$i topic"
- exit 1
- fi
-done
-
-echo "Create invoker topics"
-INVOKER_COUNT=$((INVOKER_COUNT - 1))
-for i in `seq 0 $INVOKER_COUNT`; do
- OUTPUT=$(kafka-topics.sh --create --topic invoker$i --replication-factor $REPLICATION_FACTOR --partitions $PARTITIONS --zookeeper ${ZOOKEEPER_HOST}:${ZOOKEEPER_PORT} --config retention.bytes=$KAFKA_TOPICS_INVOKER_RETENTIONBYTES --config retention.ms=$KAFKA_TOPICS_INVOKER_RETENTIONMS --config segment.bytes=$KAFKA_TOPICS_INVOKER_SEGMENTBYTES)
-
- if ! ([[ "$OUTPUT" == *"already exists"* ]] || [[ "$OUTPUT" == *"Created topic"* ]]); then
- echo "Failed to create invoker$i topic"
- exit 1
- fi
-done
-
fg
diff --git a/kubernetes/kafka/kafka.yml b/kubernetes/kafka/kafka.yml
index 5844486..e7b4ff8 100644
--- a/kubernetes/kafka/kafka.yml
+++ b/kubernetes/kafka/kafka.yml
@@ -34,8 +34,8 @@ spec:
containers:
- name: kafka
imagePullPolicy: IfNotPresent
- # custom kafka image. Not sure what we want to do with this.
- image: danlavine/whisk_kafka
+ # custom kafka image. Should be eliminated soon (see issue #26)
+ image: dgrove/whisk_kafka
ports:
- name: kafka
containerPort: 9092
@@ -59,26 +59,6 @@ spec:
- name: "KAFKA_TOPICS_HEALTH_SEGMENTBYTES"
value: "3600000"
- # complete topic settings
- - name: "CONTROLLER_COUNT"
- value: "2"
- - name: "KAFKA_TOPICS_COMPLETED_RETENTIONBYTES"
- value: "536870912"
- - name: "KAFKA_TOPICS_COMPLETED_RETENTIONMS"
- value: "1073741824"
- - name: "KAFKA_TOPICS_COMPLETED_SEGMENTBYTES"
- value: "3600000"
-
- # invoker topic settings
- - name: "INVOKER_COUNT"
- value: "1"
- - name: "KAFKA_TOPICS_INVOKER_RETENTIONBYTES"
- value: "536870912"
- - name: "KAFKA_TOPICS_INVOKER_RETENTIONMS"
- value: "1073741824"
- - name: "KAFKA_TOPICS_INVOKER_SEGMENTBYTES"
- value: "3600000"
-
# zookeeper info
- name: "ZOOKEEPER_HOST"
value: "zookeeper.openwhisk"
--
To stop receiving notification emails like this one, please contact
['"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>'].