You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by si...@apache.org on 2019/08/05 06:43:14 UTC
[pulsar] branch master updated: Update kubernetes deployment apis
k8s post 1.9 (#4700)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new bb1108e Update kubernetes deployment apis k8s post 1.9 (#4700)
bb1108e is described below
commit bb1108e8e1e6c1727887336d8a12695156042cc3
Author: Guillaume Braibant <gu...@outlook.com>
AuthorDate: Mon Aug 5 08:43:08 2019 +0200
Update kubernetes deployment apis k8s post 1.9 (#4700)
**Motivation**
Fixes #4698 and #4699
**List of changes**
1. Two new folders under pulsar/deployment/kubernetes/generic :
- original : contains the original scripts (pre Kubernetes 1.9)
- k8s-1-9-and-above : contains the new scripts with APIs ipdated (Kubernetes 1.9 and above)
2. bookie.yaml :
- Migrate DaemonSet api version from extensions/v1beta1 to apps/v1
- Declare the bookie service before the DaemonSet object for bookies
3. broker.yaml
- Migrate Deployment api version from apps/v1beta1 to apps/v1
- Declare the bookie service before the Deployment object for brokers
4. monitoring.yaml :
- Migrate all Deployment api version from apps/v1beta1 to apps/v1
- Declare each service before the Deployment object the service is bound to
5. zookeeper.yaml :
- Migrate StatefulSet api version from apps/v1beta1 to apps/v1
- Declare the service before the StatefulSet object for zookeeper
6. proxy.yaml
- Migrate Deployment api version from apps/v1beta1 to apps/v1
- Declare the bookie service before the Deployment object for proxy.
---
.../generic/{ => k8s-1-9-and-above}/admin.yaml | 0
.../generic/{ => k8s-1-9-and-above}/bookie.yaml | 47 ++++++-----
.../generic/{ => k8s-1-9-and-above}/broker.yaml | 51 ++++++------
.../{ => k8s-1-9-and-above}/cluster-metadata.yaml | 0
.../{ => k8s-1-9-and-above}/monitoring.yaml | 91 ++++++++++------------
.../generic/{ => k8s-1-9-and-above}/proxy.yaml | 54 +++++++------
.../generic/{ => k8s-1-9-and-above}/zookeeper.yaml | 53 ++++++-------
.../kubernetes/generic/{ => original}/admin.yaml | 0
.../kubernetes/generic/{ => original}/bookie.yaml | 0
.../kubernetes/generic/{ => original}/broker.yaml | 0
.../generic/{ => original}/cluster-metadata.yaml | 0
.../generic/{ => original}/monitoring.yaml | 0
.../kubernetes/generic/{ => original}/proxy.yaml | 0
.../generic/{ => original}/zookeeper.yaml | 0
14 files changed, 137 insertions(+), 159 deletions(-)
diff --git a/deployment/kubernetes/generic/admin.yaml b/deployment/kubernetes/generic/k8s-1-9-and-above/admin.yaml
similarity index 100%
copy from deployment/kubernetes/generic/admin.yaml
copy to deployment/kubernetes/generic/k8s-1-9-and-above/admin.yaml
diff --git a/deployment/kubernetes/generic/bookie.yaml b/deployment/kubernetes/generic/k8s-1-9-and-above/bookie.yaml
similarity index 99%
copy from deployment/kubernetes/generic/bookie.yaml
copy to deployment/kubernetes/generic/k8s-1-9-and-above/bookie.yaml
index fd1d044..01148eb 100644
--- a/deployment/kubernetes/generic/bookie.yaml
+++ b/deployment/kubernetes/generic/k8s-1-9-and-above/bookie.yaml
@@ -29,12 +29,32 @@ data:
zkServers: zookeeper
statsProviderClass: org.apache.bookkeeper.stats.prometheus.PrometheusMetricsProvider
---
-
+##
+## Define the Bookie headless service
+## In practice, in this case, it is only useful to have a view of
+## all the bookie pods that are present
+##
+apiVersion: v1
+kind: Service
+metadata:
+ name: bookkeeper
+ labels:
+ app: pulsar
+ component: bookkeeper
+spec:
+ ports:
+ - port: 3181
+ name: server
+ clusterIP: None
+ selector:
+ app: pulsar
+ component: bookkeeper
+---
## BookKeeper servers need to access the local disks and the pods
## cannot be moved across different nodes.
## For this reason, we run BK as a daemon set, one for each node in the
## cluster, unless restricted by label selectors
-apiVersion: extensions/v1beta1
+apiVersion: apps/v1
kind: DaemonSet
metadata:
name: bookie
@@ -107,26 +127,3 @@ spec:
- name: ledgers-disk
hostPath:
path: /mnt/disks/ssd1
-
----
-
-##
-## Define the Bookie headless service
-## In practice, in this case, it is only useful to have a view of
-## all the bookie pods that are present
-##
-apiVersion: v1
-kind: Service
-metadata:
- name: bookkeeper
- labels:
- app: pulsar
- component: bookkeeper
-spec:
- ports:
- - port: 3181
- name: server
- clusterIP: None
- selector:
- app: pulsar
- component: bookkeeper
diff --git a/deployment/kubernetes/generic/broker.yaml b/deployment/kubernetes/generic/k8s-1-9-and-above/broker.yaml
similarity index 99%
copy from deployment/kubernetes/generic/broker.yaml
copy to deployment/kubernetes/generic/k8s-1-9-and-above/broker.yaml
index 031d992..3ef62f5 100644
--- a/deployment/kubernetes/generic/broker.yaml
+++ b/deployment/kubernetes/generic/k8s-1-9-and-above/broker.yaml
@@ -38,9 +38,32 @@ data:
PF_pulsarFunctionsCluster: local
---
##
+## Define the Broker headless service
+## In practice, in this case, it is only useful to have a view of
+## all the broker pods that are present
+##
+apiVersion: v1
+kind: Service
+metadata:
+ name: broker
+ labels:
+ app: pulsar
+ component: broker
+spec:
+ ports:
+ - port: 8080
+ name: http
+ - port: 6650
+ name: pulsar
+ clusterIP: None
+ selector:
+ app: pulsar
+ component: broker
+---
+##
## Broker deployment definition
##
-apiVersion: apps/v1beta1
+apiVersion: apps/v1
kind: Deployment
metadata:
name: broker
@@ -78,29 +101,3 @@ spec:
valueFrom:
fieldRef:
fieldPath: status.podIP
----
-
-##
-## Define the Broker headless service
-## In practice, in this case, it is only useful to have a view of
-## all the broker pods that are present
-##
-apiVersion: v1
-kind: Service
-metadata:
- name: broker
- labels:
- app: pulsar
- component: broker
-spec:
- ports:
- - port: 8080
- name: http
- - port: 6650
- name: pulsar
- clusterIP: None
- selector:
- app: pulsar
- component: broker
-
----
diff --git a/deployment/kubernetes/generic/cluster-metadata.yaml b/deployment/kubernetes/generic/k8s-1-9-and-above/cluster-metadata.yaml
similarity index 100%
copy from deployment/kubernetes/generic/cluster-metadata.yaml
copy to deployment/kubernetes/generic/k8s-1-9-and-above/cluster-metadata.yaml
diff --git a/deployment/kubernetes/generic/monitoring.yaml b/deployment/kubernetes/generic/k8s-1-9-and-above/monitoring.yaml
similarity index 95%
copy from deployment/kubernetes/generic/monitoring.yaml
copy to deployment/kubernetes/generic/k8s-1-9-and-above/monitoring.yaml
index 089690f..2fdfad7 100644
--- a/deployment/kubernetes/generic/monitoring.yaml
+++ b/deployment/kubernetes/generic/k8s-1-9-and-above/monitoring.yaml
@@ -59,10 +59,28 @@ data:
- source_labels: [__meta_kubernetes_pod_name]
action: replace
target_label: kubernetes_pod_name
-
---
-
-apiVersion: apps/v1beta1
+## PROMOTHEUS - Service
+apiVersion: v1
+kind: Service
+metadata:
+ name: prometheus
+ labels:
+ app: pulsar
+ component: prometheus
+spec:
+ type: NodePort
+ ports:
+ - name: prometheus
+ nodePort: 30003
+ port: 9090
+ protocol: TCP
+ selector:
+ app: pulsar
+ component: prometheus
+---
+## PROMOTHEUS - Deployment
+apiVersion: apps/v1
kind: Deployment
metadata:
name: prometheus
@@ -90,32 +108,28 @@ spec:
name: prometheus-config
- name: data-volume
emptyDir: {}
-
---
-
+## GRAFANA - Service
apiVersion: v1
kind: Service
metadata:
- name: prometheus
+ name: grafana
labels:
app: pulsar
- component: prometheus
+ component: grafana
spec:
type: NodePort
ports:
- - name: prometheus
- nodePort: 30003
- port: 9090
- protocol: TCP
+ - name: grafana
+ nodePort: 30004
+ port: 3000
+ protocol: TCP
selector:
app: pulsar
- component: prometheus
-
-
+ component: grafana
---
-## GRAFANA
-
-apiVersion: apps/v1beta1
+## GRAFANA - Deployment
+apiVersion: apps/v1
kind: Deployment
metadata:
name: grafana
@@ -135,31 +149,28 @@ spec:
env:
- name: PROMETHEUS_URL
value: http://prometheus:9090/
-
---
-
+## PULSAR DASHBOARD - Deployment
apiVersion: v1
kind: Service
metadata:
- name: grafana
+ name: pulsar-dashboard
labels:
app: pulsar
- component: grafana
+ component: dashboard
spec:
type: NodePort
ports:
- - name: grafana
- nodePort: 30004
- port: 3000
- protocol: TCP
+ - name: dashboard
+ nodePort: 30005
+ port: 80
+ protocol: TCP
selector:
app: pulsar
- component: grafana
-
+ component: dashboard
---
-## Include detailed Pulsar dashboard
-
-apiVersion: apps/v1beta1
+## PULSAR DASHBOARD - Deployment
+apiVersion: apps/v1
kind: Deployment
metadata:
name: pulsar-dashboard
@@ -179,23 +190,3 @@ spec:
env:
- name: SERVICE_URL
value: http://broker:8080/
-
----
-
-apiVersion: v1
-kind: Service
-metadata:
- name: pulsar-dashboard
- labels:
- app: pulsar
- component: dashboard
-spec:
- type: NodePort
- ports:
- - name: dashboard
- nodePort: 30005
- port: 80
- protocol: TCP
- selector:
- app: pulsar
- component: dashboard
diff --git a/deployment/kubernetes/generic/proxy.yaml b/deployment/kubernetes/generic/k8s-1-9-and-above/proxy.yaml
similarity index 98%
copy from deployment/kubernetes/generic/proxy.yaml
copy to deployment/kubernetes/generic/k8s-1-9-and-above/proxy.yaml
index 8268835..b3f33b5 100644
--- a/deployment/kubernetes/generic/proxy.yaml
+++ b/deployment/kubernetes/generic/k8s-1-9-and-above/proxy.yaml
@@ -29,9 +29,34 @@ data:
clusterName: local
---
##
+## Expose all nodes on port so that you can reach cluster from outside k8
+##
+apiVersion: v1
+kind: Service
+metadata:
+ name: proxy
+ labels:
+ app: pulsar
+ component: proxy
+spec:
+ type: NodePort
+ ports:
+ - name: http
+ nodePort: 30001
+ port: 8080
+ protocol: TCP
+ - name: tcp
+ nodePort: 30002
+ port: 6650
+ protocol: TCP
+ selector:
+ app: pulsar
+ component: proxy
+---
+##
## Proxy deployment definition
##
-apiVersion: apps/v1beta1
+apiVersion: apps/v1
kind: Deployment
metadata:
name: proxy
@@ -60,30 +85,3 @@ spec:
envFrom:
- configMapRef:
name: proxy-config
----
-
-##
-## Expose all nodes on port so that you can reach cluster from outside k8
-##
-apiVersion: v1
-kind: Service
-metadata:
- name: proxy
- labels:
- app: pulsar
- component: proxy
-spec:
- type: NodePort
- ports:
- - name: http
- nodePort: 30001
- port: 8080
- protocol: TCP
- - name: tcp
- nodePort: 30002
- port: 6650
- protocol: TCP
- selector:
- app: pulsar
- component: proxy
----
diff --git a/deployment/kubernetes/generic/zookeeper.yaml b/deployment/kubernetes/generic/k8s-1-9-and-above/zookeeper.yaml
similarity index 99%
copy from deployment/kubernetes/generic/zookeeper.yaml
copy to deployment/kubernetes/generic/k8s-1-9-and-above/zookeeper.yaml
index e0be77d..a50b168 100644
--- a/deployment/kubernetes/generic/zookeeper.yaml
+++ b/deployment/kubernetes/generic/k8s-1-9-and-above/zookeeper.yaml
@@ -17,7 +17,6 @@
# under the License.
#
-
apiVersion: v1
kind: ConfigMap
metadata:
@@ -26,7 +25,6 @@ data:
PULSAR_MEM: "\" -Xms100m -Xmx256m \""
PULSAR_GC: "\" -XX:+UseG1GC -XX:MaxGCPauseMillis=10\""
---
-
## Define a disruption budget to ensure there are at least
## 2 ZK servers running all the time
apiVersion: policy/v1beta1
@@ -39,9 +37,31 @@ spec:
app: zk
minAvailable: 2
---
-
+##
+## Define the ZooKeeper headless service
+##
+apiVersion: v1
+kind: Service
+metadata:
+ annotations:
+ service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
+ name: zookeeper
+ labels:
+ app: pulsar
+ component: zookeeper
+spec:
+ ports:
+ - port: 2888
+ name: server
+ - port: 3888
+ name: leader-election
+ clusterIP: None
+ selector:
+ app: pulsar
+ component: zookeeper
+---
## Define a StatefulSet for ZK servers
-apiVersion: apps/v1beta1
+apiVersion: apps/v1
kind: StatefulSet
metadata:
name: zk
@@ -121,28 +141,3 @@ spec:
volumes:
- name: datadir
emptyDir: {}
-
----
-
-##
-## Define the ZooKeeper headless service
-##
-apiVersion: v1
-kind: Service
-metadata:
- annotations:
- service.alpha.kubernetes.io/tolerate-unready-endpoints: "true"
- name: zookeeper
- labels:
- app: pulsar
- component: zookeeper
-spec:
- ports:
- - port: 2888
- name: server
- - port: 3888
- name: leader-election
- clusterIP: None
- selector:
- app: pulsar
- component: zookeeper
diff --git a/deployment/kubernetes/generic/admin.yaml b/deployment/kubernetes/generic/original/admin.yaml
similarity index 100%
rename from deployment/kubernetes/generic/admin.yaml
rename to deployment/kubernetes/generic/original/admin.yaml
diff --git a/deployment/kubernetes/generic/bookie.yaml b/deployment/kubernetes/generic/original/bookie.yaml
similarity index 100%
rename from deployment/kubernetes/generic/bookie.yaml
rename to deployment/kubernetes/generic/original/bookie.yaml
diff --git a/deployment/kubernetes/generic/broker.yaml b/deployment/kubernetes/generic/original/broker.yaml
similarity index 100%
rename from deployment/kubernetes/generic/broker.yaml
rename to deployment/kubernetes/generic/original/broker.yaml
diff --git a/deployment/kubernetes/generic/cluster-metadata.yaml b/deployment/kubernetes/generic/original/cluster-metadata.yaml
similarity index 100%
rename from deployment/kubernetes/generic/cluster-metadata.yaml
rename to deployment/kubernetes/generic/original/cluster-metadata.yaml
diff --git a/deployment/kubernetes/generic/monitoring.yaml b/deployment/kubernetes/generic/original/monitoring.yaml
similarity index 100%
rename from deployment/kubernetes/generic/monitoring.yaml
rename to deployment/kubernetes/generic/original/monitoring.yaml
diff --git a/deployment/kubernetes/generic/proxy.yaml b/deployment/kubernetes/generic/original/proxy.yaml
similarity index 100%
rename from deployment/kubernetes/generic/proxy.yaml
rename to deployment/kubernetes/generic/original/proxy.yaml
diff --git a/deployment/kubernetes/generic/zookeeper.yaml b/deployment/kubernetes/generic/original/zookeeper.yaml
similarity index 100%
rename from deployment/kubernetes/generic/zookeeper.yaml
rename to deployment/kubernetes/generic/original/zookeeper.yaml