You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by mm...@apache.org on 2022/10/18 22:41:48 UTC
[pulsar-helm-chart] branch master updated: Support mechanism to provide external zookeeper-server list to build global/configuration zookeeper (#269)
This is an automated email from the ASF dual-hosted git repository.
mmarshall pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar-helm-chart.git
The following commit(s) were added to refs/heads/master by this push:
new 89f28bc Support mechanism to provide external zookeeper-server list to build global/configuration zookeeper (#269)
89f28bc is described below
commit 89f28bca9c908e9a06e8939928b09ed331df662e
Author: Rajan Dhabalia <rd...@apache.org>
AuthorDate: Tue Oct 18 15:41:43 2022 -0700
Support mechanism to provide external zookeeper-server list to build global/configuration zookeeper (#269)
* Support mechanism to provide external zookeeper-server list to build global/configuration zookeeper
* Add external zk example
* add external zk list into values.yaml
Fixes #268
### Motivation
Right now, [chart dynamically](https://github.com/apache/pulsar-helm-chart/blob/master/charts/pulsar/templates/zookeeper-statefulset.yaml#L140) creates zk cluster with zk pods initialized in the same namespace. However, for global/configuration zookeeper, user requires to build zk clusters with pods deployed in different namespaces. Therefore, user needs a mechanism to pass an external list of zk-servers to the chart and build zk-cluster with pods across different namespaces.
### Modification
- Chart should be considering zk-value's configuration for external zookeeper and generate zk-configuration file with appropriate zk-server list and unique id of that zookeeper.
This PR sets `ZOOKEEPER_SERVERS` value provided by user and also sets override-value flag which will be used by [generate-zookeeper-config.sh](https://github.com/apache/pulsar/blob/master/docker/pulsar/scripts/generate-zookeeper-config.sh) to override external zk list in config file and assign appropriate id to the host.
https://github.com/apache/pulsar/pull/15987 fixes [generate-zookeeper-config.sh](https://github.com/apache/pulsar/blob/master/docker/pulsar/scripts/generate-zookeeper-config.sh) changes.
### Result
- User can add `ZOOKEEPER_SERVERS` string into `zookeeper.configData` in [Values.yaml](https://github.com/apache/pulsar-helm-chart/blob/master/charts/pulsar/values.yaml#L385) file to override external zk-server list.
---
charts/pulsar/templates/zookeeper-statefulset.yaml | 17 +++++++++++++----
charts/pulsar/values.yaml | 2 ++
examples/values-zookeeper-aws.yaml | 3 +++
3 files changed, 18 insertions(+), 4 deletions(-)
diff --git a/charts/pulsar/templates/zookeeper-statefulset.yaml b/charts/pulsar/templates/zookeeper-statefulset.yaml
index 5590a40..a72d50b 100644
--- a/charts/pulsar/templates/zookeeper-statefulset.yaml
+++ b/charts/pulsar/templates/zookeeper-statefulset.yaml
@@ -138,10 +138,19 @@ spec:
containerPort: {{ .Values.zookeeper.ports.clientTls }}
{{- end }}
env:
- - name: ZOOKEEPER_SERVERS
- value:
- {{- $global := . }}
- {{ range $i, $e := until (.Values.zookeeper.replicaCount | int) }}{{ if ne $i 0 }},{{ end }}{{ template "pulsar.fullname" $global }}-{{ $global.Values.zookeeper.component }}-{{ printf "%d" $i }}{{ end }}
+ - name: ZOOKEEPER_SERVERS
+ {{- if .Values.zookeeper.configData.ZOOKEEPER_SERVERS }}
+ value: {{ .Values.zookeeper.configData.ZOOKEEPER_SERVERS }}
+ {{- else }}
+ {{- $global := . }}
+ value: {{ range $i, $e := until (.Values.zookeeper.replicaCount | int) }}{{ if ne $i 0 }},{{ end }}{{ template "pulsar.fullname" $global }}-{{ $global.Values.zookeeper.component }}-{{ printf "%d" $i }}{{ end }}
+ {{- end }}
+ - name: EXTERNAL_PROVIDED_SERVERS
+ {{- if .Values.zookeeper.configData.ZOOKEEPER_SERVERS }}
+ value: "true"
+ {{- else }}
+ value: "false"
+ {{- end }}
envFrom:
- configMapRef:
name: "{{ template "pulsar.fullname" . }}-{{ .Values.zookeeper.component }}"
diff --git a/charts/pulsar/values.yaml b/charts/pulsar/values.yaml
index a602edb..5d970e1 100644
--- a/charts/pulsar/values.yaml
+++ b/charts/pulsar/values.yaml
@@ -397,6 +397,8 @@ zookeeper:
## templates/zookeeper-configmap.yaml
##
configData:
+ # External zookeeper server list in case of global-zk list to create zk cluster across zk deployed on different clusters/namespaces
+ # ZOOKEEPER_SERVERS: "us-east1-pulsar-zookeeper-0.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-east1-pulsar-zookeeper-1.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-east1-pulsar-zookeeper-2.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-west1-pulsar-zookeeper-0.us-west1-pulsar-zookeeper.us-west1.svc.cluster.local:2888:3888,us-west1-pulsar-zookeeper-1.us-west1-pulsar-zookeeper.us-west1.svc.cluster.local:2888:3888,us-west1-pu [...]
PULSAR_MEM: >
-Xms64m -Xmx128m
PULSAR_GC: >
diff --git a/examples/values-zookeeper-aws.yaml b/examples/values-zookeeper-aws.yaml
index 8b870ef..2116f86 100644
--- a/examples/values-zookeeper-aws.yaml
+++ b/examples/values-zookeeper-aws.yaml
@@ -45,6 +45,9 @@ monitoring:
node_exporter: false
zookeeper:
+ configData:
+ # External zookeeper server list in case of global-zk list to create zk cluster across zk deployed on different clusters/namespaces
+ # ZOOKEEPER_SERVERS: "us-east1-pulsar-zookeeper-0.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-east1-pulsar-zookeeper-1.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-east1-pulsar-zookeeper-2.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-west1-pulsar-zookeeper-0.us-west1-pulsar-zookeeper.us-west1.svc.cluster.local:2888:3888,us-west1-pulsar-zookeeper-1.us-west1-pulsar-zookeeper.us-west1.svc.cluster.local:2888:3888,us-west1-pu [...]
volumes:
# use a persistent volume or emptyDir
persistence: true