You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by xi...@apache.org on 2019/10/21 07:05:11 UTC
[incubator-pinot] branch update_k8s_deployment_examples created
(now d263290)
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a change to branch update_k8s_deployment_examples
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
at d263290 Adding exmaples for helm chart
This branch includes the following new commits:
new d263290 Adding exmaples for helm chart
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[incubator-pinot] 01/01: Adding exmaples for helm chart
Posted by xi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a commit to branch update_k8s_deployment_examples
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit d26329064fd35fc48505823a005444b3c9254e4c
Author: Xiang Fu <fx...@gmail.com>
AuthorDate: Mon Oct 21 00:04:41 2019 -0700
Adding exmaples for helm chart
---
kubernetes/examples/gke/helm/Chart.yaml | 16 +++
kubernetes/examples/gke/helm/README.md | 96 ++++++++++++++
kubernetes/examples/gke/helm/query-pinot-data.sh | 28 +++++
kubernetes/examples/gke/helm/requirements.yaml | 0
kubernetes/examples/gke/helm/setup.sh | 51 ++++++++
.../examples/gke/helm/templates/_helpers.tpl | 62 +++++++++
.../gke/helm/templates/broker/configmap.yaml | 9 ++
.../gke/helm/templates/broker/service.yaml | 19 +++
.../gke/helm/templates/broker/statefulset.yml | 96 ++++++++++++++
.../gke/helm/templates/controller/configmap.yaml | 13 ++
.../gke/helm/templates/controller/service.yaml | 19 +++
.../gke/helm/templates/controller/statefulset.yaml | 101 +++++++++++++++
.../gke/helm/templates/server/configmap.yaml | 11 ++
.../gke/helm/templates/server/service.yaml | 19 +++
.../gke/helm/templates/server/statefulset.yml | 109 ++++++++++++++++
kubernetes/examples/gke/helm/values.yaml | 138 +++++++++++++++++++++
16 files changed, 787 insertions(+)
diff --git a/kubernetes/examples/gke/helm/Chart.yaml b/kubernetes/examples/gke/helm/Chart.yaml
new file mode 100644
index 0000000..69618e8
--- /dev/null
+++ b/kubernetes/examples/gke/helm/Chart.yaml
@@ -0,0 +1,16 @@
+apiVersion: v1
+appVersion: 0.2.0
+name: pinot
+description: Apache Pinot is a realtime distributed OLAP datastore, which is used to deliver scalable real time analytics with low latency. It can ingest data from offline data sources (such as Hadoop and flat files) as well as online sources (such as Kafka). Pinot is designed to scale horizontally.
+version: 0.2.0
+keywords:
+ - olap
+ - analytics
+ - database
+ - pinot
+home: https://pinot.apache.org/
+sources:
+ - https://github.com/apache/incubator-pinot
+maintainers:
+ - name: pinot-dev
+ email: dev@pinot.apache.org
diff --git a/kubernetes/examples/gke/helm/README.md b/kubernetes/examples/gke/helm/README.md
new file mode 100644
index 0000000..077cd74
--- /dev/null
+++ b/kubernetes/examples/gke/helm/README.md
@@ -0,0 +1,96 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+# Pinot Quickstart on Kubernetes on Google Kubernetes Engine(GKE)
+
+## Prerequisite
+
+- kubectl (https://kubernetes.io/docs/tasks/tools/install-kubectl/)
+- Google Cloud SDK (https://cloud.google.com/sdk/install)
+- Helm (https://helm.sh/docs/using_helm/#installing-helm)
+- Enable Google Cloud Account and create a project, e.g. `pinot-demo`.
+ - `pinot-demo` will be used as example value for `${GCLOUD_PROJECT}` variable in script example.
+ - `pinot-demo@example.com` will be used as example value for `${GCLOUD_EMAIL}`.
+- Configure kubectl to connect to the Kubernetes cluster.
+
+## Create a cluster on GKE
+
+Below script will:
+- Create a gCloud cluster `pinot-quickstart`
+- Request 1 server of type `n1-standard-2` for zookeeper, kafka, pinot controller, pinot broker.
+- Request 1 server of type `n1-standard-8` for Pinot server.
+
+Please fill both environment variables: `${GCLOUD_PROJECT}` and `${GCLOUD_EMAIL}` with your gcloud project and gcloud account email in below script.
+```
+GCLOUD_PROJECT=[your gcloud project name]
+GCLOUD_EMAIL=[Your gcloud account email]
+./setup.sh
+```
+
+E.g.
+```
+GCLOUD_PROJECT=pinot-demo
+GCLOUD_EMAIL=pinot-demo@example.com
+./setup.sh
+```
+
+Feel free to modify the script to pick your preferred sku, e.g. `n1-highmem-32` for Pinot server.
+
+
+## How to connect to an existing cluster
+Simply run below command to get the credential for the cluster you just created or your existing cluster.
+Please modify the Env variables `${GCLOUD_PROJECT}`, `${GCLOUD_ZONE}`, `${GCLOUD_CLUSTER}` accordingly in below script.
+```
+GCLOUD_PROJECT=pinot-demo
+GCLOUD_ZONE=us-west1-b
+GCLOUD_CLUSTER=pinot-quickstart
+gcloud container clusters get-credentials ${GCLOUD_CLUSTER} --zone ${GCLOUD_ZONE} --project ${GCLOUD_PROJECT}
+```
+
+Look for cluster status
+```
+kubectl get all -n pinot-quickstart -o wide
+```
+
+## How to setup a Pinot cluster for demo
+
+The script requests:
+ - Create persistent disk for deep storage and mount it.
+ - Zookeeper
+ - Pinot Controller
+ - Pinot Server
+ - Create Pods for
+ - Zookeeper
+ - Pinot Controller
+ - Pinot Broker
+ - Pinot Server
+ - Pinot Example Loader
+
+
+```
+helm install --namespace "pinot-quickstart" --name "pinot" kubernetes/examples/gke/helm/
+```
+
+## How to query pinot data
+
+Please use below script to do local port-forwarding and open Pinot query console on your web browser.
+```
+./query-pinot-data.sh
+```
diff --git a/kubernetes/examples/gke/helm/query-pinot-data.sh b/kubernetes/examples/gke/helm/query-pinot-data.sh
new file mode 100755
index 0000000..9e2ab08
--- /dev/null
+++ b/kubernetes/examples/gke/helm/query-pinot-data.sh
@@ -0,0 +1,28 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+if [[ $(nc -z localhost 9000) != 0 ]]; then
+ kubectl port-forward service/pinot-controller 9000:9000 -n pinot-quickstart > /dev/null &
+fi
+sleep 2
+open http://localhost:9000/query
+# Just for blocking
+tail -f /dev/null
+pkill -f "kubectl port-forward service/pinot-controller 9000:9000 -n pinot-quickstart"
diff --git a/kubernetes/examples/gke/helm/requirements.yaml b/kubernetes/examples/gke/helm/requirements.yaml
new file mode 100644
index 0000000..e69de29
diff --git a/kubernetes/examples/gke/helm/setup.sh b/kubernetes/examples/gke/helm/setup.sh
new file mode 100755
index 0000000..8f13c60
--- /dev/null
+++ b/kubernetes/examples/gke/helm/setup.sh
@@ -0,0 +1,51 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+set -e
+if [[ -z "${GCLOUD_EMAIL}" ]] || [[ -z "${GCLOUD_PROJECT}" ]]
+then
+ echo "Please set both \$GCLOUD_EMAIL and \$GCLOUD_PROJECT variables. E.g. GCLOUD_PROJECT=pinot-demo GCLOUD_EMAIL=pinot-demo@example.com ./setup.sh"
+ exit 1
+fi
+
+GCLOUD_ZONE=us-west1-b
+GCLOUD_CLUSTER=pinot-quickstart
+GCLOUD_MACHINE_TYPE=n1-standard-2
+GCLOUD_NUM_NODES=1
+gcloud container clusters create ${GCLOUD_CLUSTER} \
+ --num-nodes=${GCLOUD_NUM_NODES} \
+ --machine-type=${GCLOUD_MACHINE_TYPE} \
+ --zone=${GCLOUD_ZONE} \
+ --project=${GCLOUD_PROJECT}
+gcloud container clusters get-credentials ${GCLOUD_CLUSTER} --zone ${GCLOUD_ZONE} --project ${GCLOUD_PROJECT}
+
+
+GCLOUD_MACHINE_TYPE=n1-standar-8
+gcloud container node-pools create pinot-server-pool \
+ --cluster=${GCLOUD_CLUSTER} \
+ --machine-type=${GCLOUD_MACHINE_TYPE} \
+ --num-nodes=${GCLOUD_NUM_NODES} \
+ --zone=${GCLOUD_ZONE} \
+ --project=${GCLOUD_PROJECT}
+
+kubectl create namespace pinot-quickstart
+
+kubectl create clusterrolebinding cluster-admin-binding --clusterrole cluster-admin --user ${GCLOUD_EMAIL}
+kubectl create clusterrolebinding add-on-cluster-admin --clusterrole=cluster-admin --serviceaccount=pinot-quickstart:default
diff --git a/kubernetes/examples/gke/helm/templates/_helpers.tpl b/kubernetes/examples/gke/helm/templates/_helpers.tpl
new file mode 100644
index 0000000..e5044ed
--- /dev/null
+++ b/kubernetes/examples/gke/helm/templates/_helpers.tpl
@@ -0,0 +1,62 @@
+{{/* vim: set filetype=mustache: */}}
+{{/*
+Expand the name of the chart.
+*/}}
+{{- define "pinot.name" -}}
+{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
+{{- end -}}
+
+
+{{/*
+Create a default fully qualified app name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+If release name contains chart name it will be used as a full name.
+*/}}
+{{- define "pinot.fullname" -}}
+{{- if .Values.fullnameOverride -}}
+{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
+{{- else -}}
+{{- $name := default .Chart.Name .Values.nameOverride -}}
+{{- if contains $name .Release.Name -}}
+{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
+{{- else -}}
+{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
+{{- end -}}
+{{- end -}}
+{{- end -}}
+
+
+{{/*
+Create chart name and version as used by the chart label.
+*/}}
+{{- define "pinot.chart" -}}
+{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
+{{- end -}}
+
+
+{{/*
+Create a default fully qualified pinot controller name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+*/}}
+{{- define "pinot.controller.fullname" -}}
+{{ template "pinot.fullname" . }}-{{ .Values.controller.name }}
+{{- end -}}
+
+
+{{/*
+Create a default fully qualified pinot broker name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+*/}}
+{{- define "pinot.broker.fullname" -}}
+{{ template "pinot.fullname" . }}-{{ .Values.broker.name }}
+{{- end -}}
+
+
+{{/*
+Create a default fully qualified pinot server name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+*/}}
+{{- define "pinot.server.fullname" -}}
+{{ template "pinot.fullname" . }}-{{ .Values.server.name }}
+{{- end -}}
+
diff --git a/kubernetes/examples/gke/helm/templates/broker/configmap.yaml b/kubernetes/examples/gke/helm/templates/broker/configmap.yaml
new file mode 100644
index 0000000..99dd0f7
--- /dev/null
+++ b/kubernetes/examples/gke/helm/templates/broker/configmap.yaml
@@ -0,0 +1,9 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: broker-config
+data:
+ pinot-broker.conf: |-
+ pinot.broker.client.queryPort={{ .Values.broker.port }}
+ pinot.broker.routing.table.builder.class=random
+ pinot.set.instance.id.to.hostname=true
diff --git a/kubernetes/examples/gke/helm/templates/broker/service.yaml b/kubernetes/examples/gke/helm/templates/broker/service.yaml
new file mode 100644
index 0000000..7bad75a
--- /dev/null
+++ b/kubernetes/examples/gke/helm/templates/broker/service.yaml
@@ -0,0 +1,19 @@
+apiVersion: v1
+kind: Service
+metadata:
+ name: {{ include "pinot.broker.fullname" . }}
+ labels:
+ app: {{ include "pinot.name" . }}
+ chart: {{ include "pinot.chart" . }}
+ component: {{ .Values.broker.name }}
+ release: {{ .Release.Name }}
+ heritage: {{ .Release.Service }}
+spec:
+ type: ClusterIP
+ ports:
+ # [pod_name].[service_name].[namespace].svc.cluster.local
+ - port: {{ .Values.broker.service.port }}
+ selector:
+ app: {{ include "pinot.name" . }}
+ release: {{ .Release.Name }}
+ component: {{ .Values.broker.name }}
diff --git a/kubernetes/examples/gke/helm/templates/broker/statefulset.yml b/kubernetes/examples/gke/helm/templates/broker/statefulset.yml
new file mode 100644
index 0000000..d6e7477
--- /dev/null
+++ b/kubernetes/examples/gke/helm/templates/broker/statefulset.yml
@@ -0,0 +1,96 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: {{ include "pinot.broker.fullname" . }}
+ labels:
+ app: {{ include "pinot.name" . }}
+ chart: {{ include "pinot.chart" . }}
+ component: {{ .Values.broker.name }}
+ release: {{ .Release.Name }}
+ heritage: {{ .Release.Service }}
+spec:
+ selector:
+ matchLabels:
+ app: {{ include "pinot.name" . }}
+ release: {{ .Release.Name }}
+ component: {{ .Values.broker.name }}
+ serviceName: broker
+ replicas: {{ .Values.broker.replicaCount }}
+ updateStrategy:
+ type: {{ .Values.broker.updateStrategy.type }}
+ podManagementPolicy: Parallel
+ template:
+ metadata:
+ labels:
+ app: {{ include "pinot.name" . }}
+ release: {{ .Release.Name }}
+ component: {{ .Values.broker.name }}
+ {{- with .Values.broker.podAnnotations }}
+ annotations:
+ {{ toYaml . | indent 8 }}
+ {{- end }}
+ spec:
+ containers:
+ - name: broker
+ image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
+ imagePullPolicy: {{ .Values.image.pullPolicy }}
+ args: [
+ "StartBroker",
+ "-clusterName", "{{ .Values.cluster.name }}",
+ "-zkAddress", "{{ .Values.cluster.zookeeper }}",
+ "-configFileName", "/var/pinot/broker/config/pinot-broker.conf"
+ ]
+ ports:
+ - containerPort: {{ .Values.broker.port }}
+ protocol: TCP
+ volumeMounts:
+ - name: config
+ mountPath: /var/pinot/broker/config
+ livenessProbe:
+ initialDelaySeconds: 60
+ httpGet:
+ path: /health
+ port: {{ .Values.broker.port }}
+ readinessProbe:
+ initialDelaySeconds: 60
+ httpGet:
+ path: /health
+ port: {{ .Values.broker.port }}
+ resources:
+ {{ toYaml .Values.broker.resources | indent 12 }}
+ {{- with .Values.broker.nodeSelector }}
+ nodeSelector:
+ {{ toYaml . | indent 8 }}
+ {{- end }}
+ {{- with .Values.broker.affinity }}
+ affinity:
+ {{ toYaml . | indent 8 }}
+ {{- end }}
+ {{- with .Values.broker.tolerations }}
+ tolerations:
+ {{ toYaml . | indent 8 }}
+ {{- end }}
+ restartPolicy: Always
+ volumes:
+ - name: config
+ configMap:
+ name: broker-config
diff --git a/kubernetes/examples/gke/helm/templates/controller/configmap.yaml b/kubernetes/examples/gke/helm/templates/controller/configmap.yaml
new file mode 100644
index 0000000..f7e33b4
--- /dev/null
+++ b/kubernetes/examples/gke/helm/templates/controller/configmap.yaml
@@ -0,0 +1,13 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: controller-config
+data:
+ pinot-controller.conf: |-
+ controller.helix.cluster.name={{ .Values.cluster.name }}
+ controller.port={{ .Values.controller.port }}
+ controller.vip.host={{ .Values.controller.host }}
+ controller.vip.port={{ .Values.controller.port }}
+ controller.data.dir={{ .Values.controller.data.dir }}
+ controller.zk.str={{ .Values.cluster.zookeeper }}
+ pinot.set.instance.id.to.hostname=true
diff --git a/kubernetes/examples/gke/helm/templates/controller/service.yaml b/kubernetes/examples/gke/helm/templates/controller/service.yaml
new file mode 100644
index 0000000..45a5ec0
--- /dev/null
+++ b/kubernetes/examples/gke/helm/templates/controller/service.yaml
@@ -0,0 +1,19 @@
+apiVersion: v1
+kind: Service
+metadata:
+ name: {{ include "pinot.controller.fullname" . }}
+ labels:
+ app: {{ include "pinot.name" . }}
+ chart: {{ include "pinot.chart" . }}
+ component: {{ .Values.controller.name }}
+ release: {{ .Release.Name }}
+ heritage: {{ .Release.Service }}
+spec:
+ type: ClusterIP
+ ports:
+ # [pod_name].[service_name].[namespace].svc.cluster.local
+ - port: {{ .Values.controller.service.port }}
+ selector:
+ app: {{ include "pinot.name" . }}
+ release: {{ .Release.Name }}
+ component: {{ .Values.controller.name }}
diff --git a/kubernetes/examples/gke/helm/templates/controller/statefulset.yaml b/kubernetes/examples/gke/helm/templates/controller/statefulset.yaml
new file mode 100644
index 0000000..99082fa
--- /dev/null
+++ b/kubernetes/examples/gke/helm/templates/controller/statefulset.yaml
@@ -0,0 +1,101 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: {{ include "pinot.controller.fullname" . }}
+ labels:
+ app: {{ include "pinot.name" . }}
+ chart: {{ include "pinot.chart" . }}
+ component: {{ .Values.controller.name }}
+ release: {{ .Release.Name }}
+ heritage: {{ .Release.Service }}
+spec:
+ selector:
+ matchLabels:
+ app: {{ include "pinot.name" . }}
+ release: {{ .Release.Name }}
+ component: {{ .Values.controller.name }}
+ serviceName: controller
+ replicas: {{ .Values.controller.replicaCount }}
+ updateStrategy:
+ type: {{ .Values.controller.updateStrategy.type }}
+ podManagementPolicy: Parallel
+ template:
+ metadata:
+ labels:
+ app: {{ include "pinot.name" . }}
+ release: {{ .Release.Name }}
+ component: {{ .Values.controller.name }}
+ annotations:
+{{ toYaml .Values.controller.podAnnotations | indent 8 }}
+ spec:
+ nodeSelector:
+{{ toYaml .Values.controller.nodeSelector | indent 8 }}
+ affinity:
+{{ toYaml .Values.controller.affinity | indent 8 }}
+ tolerations:
+{{ toYaml .Values.controller.tolerations | indent 8 }}
+ containers:
+ - name: controller
+ image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
+ imagePullPolicy: {{ .Values.image.pullPolicy }}
+ args: [ "StartController", "-configFileName", "/var/pinot/controller/config/pinot-controller.conf" ]
+ ports:
+ - containerPort: {{ .Values.controller.port }}
+ protocol: TCP
+ volumeMounts:
+ - name: config
+ mountPath: /var/pinot/controller/config
+ - name: data
+ mountPath: "{{ .Values.controller.data.dir }}"
+ resources:
+{{ toYaml .Values.controller.resources | indent 12 }}
+ initContainers:
+ - name: create-zk-root-path
+ image: solsson/kafka:2.1.0@sha256:ac3f06d87d45c7be727863f31e79fbfdcb9c610b51ba9cf03c75a95d602f15e1
+ command: ["bin/zookeeper-shell.sh", "ZooKeeper", "-server", "{{ .Values.cluster.zookeeper }}", "create", "/", "\"\""]
+ restartPolicy: Always
+ volumes:
+ - name: config
+ configMap:
+ name: controller-config
+{{- if not .Values.controller.persistence.enabled }}
+ - name: data
+ emptyDir: {}
+{{- end }}
+{{- if .Values.controller.persistence.enabled }}
+ volumeClaimTemplates:
+ - metadata:
+ name: data
+ spec:
+ accessModes:
+ - {{ .Values.controller.persistence.accessMode | quote }}
+ {{- if .Values.controller.persistence.storageClass }}
+ {{- if (eq "-" .Values.controller.persistence.storageClass) }}
+ storageClassName: ""
+ {{- else }}
+ storageClassName: {{ .Values.controller.persistence.storageClass }}
+ {{- end }}
+ {{- end }}
+ resources:
+ requests:
+ storage: {{ .Values.controller.persistence.size | quote}}
+{{ end }}
diff --git a/kubernetes/examples/gke/helm/templates/server/configmap.yaml b/kubernetes/examples/gke/helm/templates/server/configmap.yaml
new file mode 100644
index 0000000..5f5040c
--- /dev/null
+++ b/kubernetes/examples/gke/helm/templates/server/configmap.yaml
@@ -0,0 +1,11 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: server-config
+data:
+ pinot-server.conf: |-
+ pinot.server.netty.port={{ .Values.server.ports.netty }}
+ pinot.server.adminapi.port={{ .Values.server.ports.admin }}
+ pinot.server.instance.dataDir={{ .Values.server.dataDir }}
+ pinot.server.instance.segmentTarDir={{ .Values.server.segmentTarDir }}
+ pinot.set.instance.id.to.hostname=true
diff --git a/kubernetes/examples/gke/helm/templates/server/service.yaml b/kubernetes/examples/gke/helm/templates/server/service.yaml
new file mode 100644
index 0000000..cdbaa36
--- /dev/null
+++ b/kubernetes/examples/gke/helm/templates/server/service.yaml
@@ -0,0 +1,19 @@
+apiVersion: v1
+kind: Service
+metadata:
+ name: {{ include "pinot.server.fullname" . }}
+ labels:
+ app: {{ include "pinot.name" . }}
+ chart: {{ include "pinot.chart" . }}
+ component: {{ .Values.server.name }}
+ release: {{ .Release.Name }}
+ heritage: {{ .Release.Service }}
+spec:
+ type: ClusterIP
+ ports:
+ # [pod_name].[service_name].[namespace].svc.cluster.local
+ - port: {{ .Values.server.service.port }}
+ selector:
+ app: {{ include "pinot.name" . }}
+ release: {{ .Release.Name }}
+ component: {{ .Values.server.name }}
diff --git a/kubernetes/examples/gke/helm/templates/server/statefulset.yml b/kubernetes/examples/gke/helm/templates/server/statefulset.yml
new file mode 100644
index 0000000..0a63feb
--- /dev/null
+++ b/kubernetes/examples/gke/helm/templates/server/statefulset.yml
@@ -0,0 +1,109 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: {{ include "pinot.server.fullname" . }}
+ labels:
+ app: {{ include "pinot.name" . }}
+ chart: {{ include "pinot.chart" . }}
+ component: {{ .Values.server.name }}
+ release: {{ .Release.Name }}
+ heritage: {{ .Release.Service }}
+spec:
+ selector:
+ matchLabels:
+ app: {{ include "pinot.name" . }}
+ release: {{ .Release.Name }}
+ component: {{ .Values.server.name }}
+ serviceName: server
+ replicas: {{ .Values.server.replicaCount }}
+ updateStrategy:
+ type: {{ .Values.server.updateStrategy.type }}
+ podManagementPolicy: Parallel
+ template:
+ metadata:
+ labels:
+ app: {{ include "pinot.name" . }}
+ release: {{ .Release.Name }}
+ component: {{ .Values.server.name }}
+ annotations:
+{{- with .Values.server.podAnnotations }}
+{{ toYaml . | indent 8 }}
+{{- end }}
+ spec:
+ nodeSelector:
+{{- with .Values.server.nodeSelector }}
+{{ toYaml . | indent 8 }}
+{{- end }}
+ affinity:
+{{- with .Values.server.affinity }}
+{{ toYaml . | indent 8 }}
+{{- end }}
+ tolerations:
+{{- with .Values.server.tolerations }}
+{{ toYaml . | indent 8 }}
+{{- end }}
+ containers:
+ - name: server
+ image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
+ imagePullPolicy: {{ .Values.image.pullPolicy }}
+ args: [
+ "StartServer",
+ "-clusterName", "{{ .Values.cluster.name }}",
+ "-zkAddress", "{{ .Values.cluster.zookeeper }}",
+ "-configFileName", "/var/pinot/server/config/pinot-server.conf"
+ ]
+ ports:
+ - containerPort: {{ .Values.server.ports.netty }}
+ protocol: TCP
+ volumeMounts:
+ - name: config
+ mountPath: /var/pinot/server/config
+ - name: data
+ mountPath: /var/pinot/server/data
+ restartPolicy: Always
+ volumes:
+ - name: config
+ configMap:
+ name: server-config
+ {{- if not .Values.server.persistence.enabled }}
+ - name: data
+ emptyDir: {}
+ {{- end }}
+
+ {{- if .Values.server.persistence.enabled }}
+ volumeClaimTemplates:
+ - metadata:
+ name: data
+ spec:
+ accessModes:
+ - {{ .Values.server.persistence.accessMode | quote }}
+ {{- if .Values.server.persistence.storageClass }}
+ {{- if (eq "-" .Values.server.persistence.storageClass) }}
+ storageClassName: ""
+ {{- else }}
+ storageClassName: {{ .Values.server.persistence.storageClass }}
+ {{- end }}
+ {{- end }}
+ resources:
+ requests:
+ storage: {{ .Values.server.persistence.size }}
+ {{ end }}
diff --git a/kubernetes/examples/gke/helm/values.yaml b/kubernetes/examples/gke/helm/values.yaml
new file mode 100644
index 0000000..1965156
--- /dev/null
+++ b/kubernetes/examples/gke/helm/values.yaml
@@ -0,0 +1,138 @@
+# Default values for Pinot.
+
+image:
+ repository: winedepot/pinot
+ tag: 0.1.13-SNAPSHOT
+ pullPolicy: IfNotPresent
+
+namespace: pinot-quickstart
+
+cluster:
+ name: pinot-quickstart
+ zookeeper: zookeeper:2181/pinot
+
+controller:
+ name: controller
+
+ replicaCount: 1
+
+ persistence:
+ enabled: true
+ accessMode: ReadWriteOnce
+ size: 10G
+ #storageClass: "ssd"
+
+ data:
+ dir: /var/pinot/controller/data
+
+ port: 9000
+
+ service:
+ annotations: {}
+ clusterIP: ""
+ externalIPs: []
+ loadBalancerIP: ""
+ loadBalancerSourceRanges: []
+ type: ClusterIP
+ port: 9000
+ nodePort: ""
+
+ resources: {}
+
+ nodeSelector: {}
+
+ tolerations: []
+
+ affinity: {}
+
+ podAnnotations: {}
+
+ updateStrategy:
+ type: RollingUpdate
+
+broker:
+ name: broker
+
+ replicaCount: 1
+
+ port: 8099
+ service:
+ annotations: {}
+ clusterIP: ""
+ externalIPs: []
+ loadBalancerIP: ""
+ loadBalancerSourceRanges: []
+ type: ClusterIP
+ port: 8099
+ nodePort: ""
+
+ resources: {}
+
+ nodeSelector: {}
+
+ tolerations: []
+
+ affinity: {}
+
+ podAnnotations: {}
+
+ updateStrategy:
+ type: RollingUpdate
+
+server:
+ name: server
+
+ replicaCount: 1
+
+ ports:
+ netty: 8098
+ admin: 8097
+
+ dataDir: /var/pinot/server/data/index
+ segmentTarDir: /var/pinot/server/data/index
+
+ persistence:
+ enabled: true
+ accessMode: ReadWriteOnce
+ size: 10G
+ #storageClass: "ssd"
+
+ service:
+ annotations: {}
+ clusterIP: ""
+ externalIPs: []
+ loadBalancerIP: ""
+ loadBalancerSourceRanges: []
+ type: ClusterIP
+ port: 8098
+ nodePort: ""
+
+ resources: {}
+
+ nodeSelector: {}
+
+ tolerations: []
+
+ affinity: {}
+
+ podAnnotations: {}
+
+ updateStrategy:
+ type: RollingUpdate
+
+# ------------------------------------------------------------------------------
+# Zookeeper:
+# ------------------------------------------------------------------------------
+
+# zkHosts: pinot-zookeeper-headless:2181
+
+zookeeper:
+ enabled: true
+ ## Environmental variables to set in Zookeeper
+ ##
+ env:
+ ## The JVM heap size to allocate to Zookeeper
+ ZK_HEAP_SIZE: "512M"
+ ## Configure Zookeeper persistence
+ persistence:
+ enabled: true
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org