You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by xi...@apache.org on 2020/03/17 22:19:04 UTC
[incubator-pinot] branch update_k8s_helm_setup created (now 2ac19af)
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a change to branch update_k8s_helm_setup
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
at 2ac19af Fixing the missing fields in k8s helm template and reduce hardware & jvm requirements to start up pinot stack
This branch includes the following new commits:
new 2ac19af Fixing the missing fields in k8s helm template and reduce hardware & jvm requirements to start up pinot stack
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[incubator-pinot] 01/01: Fixing the missing fields in k8s helm
template and reduce hardware & jvm requirements to start up pinot stack
Posted by xi...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a commit to branch update_k8s_helm_setup
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit 2ac19afedb6e52902b1c2d6c586c0cf180d0b73d
Author: Xiang Fu <fx...@gmail.com>
AuthorDate: Tue Mar 17 15:18:43 2020 -0700
Fixing the missing fields in k8s helm template and reduce hardware & jvm requirements to start up pinot stack
---
kubernetes/helm/README.md | 18 +++++++++---------
kubernetes/helm/templates/broker/statefulset.yml | 2 +-
kubernetes/helm/templates/server/statefulset.yml | 10 ++++++----
kubernetes/helm/values.yaml | 19 +++++++++++--------
4 files changed, 27 insertions(+), 22 deletions(-)
diff --git a/kubernetes/helm/README.md b/kubernetes/helm/README.md
index b248864..211d8be 100644
--- a/kubernetes/helm/README.md
+++ b/kubernetes/helm/README.md
@@ -245,14 +245,14 @@ kubectl get all -n pinot-quickstart
```bash
helm repo add incubator http://storage.googleapis.com/kubernetes-charts-incubator
-helm install -n pinot-quickstart kafka incubator/kafka
+helm install -n pinot-quickstart kafka incubator/kafka --set replicas=1
```
- For helm v2.12.1
```bash
helm repo add incubator http://storage.googleapis.com/kubernetes-charts-incubator
-helm install --namespace "pinot-quickstart" --name kafka incubator/kafka
+helm install --namespace "pinot-quickstart" --name kafka incubator/kafka --set replicas=1
```
#### Create Kafka topic
@@ -291,7 +291,7 @@ following configurable parameters:
|------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------|
| `controller.name` | Name of Pinot Controller | `controller` |
| `controller.port` | Pinot controller port | `9000` |
-| `controller.replicaCount` | Pinot controller replicas | `2` |
+| `controller.replicaCount` | Pinot controller replicas | `1` |
| `controller.data.dir` | Pinot controller data directory, should be same as `controller.persistence.mountPath` or a sub directory of it | `/var/pinot/controller/data` |
| `controller.vip.host` | Pinot Vip host | `pinot-controller` |
| `controller.vip.port` | Pinot Vip port | `9000` |
@@ -300,7 +300,7 @@ following configurable parameters:
| `controller.persistence.size` | Size of data volume | `1G` |
| `controller.persistence.mountPath` | Mount path of controller data volume | `/var/pinot/controller/data` |
| `controller.persistence.storageClass` | Storage class of backing PVC | `""` |
-| `controller.jvmOpts` | Pinot Controller JVM Options | `-Xms4G -Xmx4G` |
+| `controller.jvmOpts` | Pinot Controller JVM Options | `-Xms256M -Xmx1G` |
| `controller.log4j2ConfFile` | Pinot Controller log4j2 configuration file | `/opt/pinot/conf/pinot-controller-log4j2.xml` |
| `controller.pluginsDir` | Pinot Controller plugins directory | `/opt/pinot/plugins` |
| `controller.service.port` | Service Port | `9000` |
@@ -316,8 +316,8 @@ following configurable parameters:
|------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------|
| `broker.name` | Name of Pinot Broker | `broker` |
| `broker.port` | Pinot broker port | `8099` |
-| `broker.replicaCount` | Pinot broker replicas | `2` |
-| `broker.jvmOpts` | Pinot Broker JVM Options | `-Xms4G -Xmx4G` |
+| `broker.replicaCount` | Pinot broker replicas | `1` |
+| `broker.jvmOpts` | Pinot Broker JVM Options | `-Xms256M -Xmx1G` |
| `broker.log4j2ConfFile` | Pinot Broker log4j2 configuration file | `/opt/pinot/conf/pinot-broker-log4j2.xml` |
| `broker.pluginsDir` | Pinot Broker plugins directory | `/opt/pinot/plugins` |
| `broker.service.port` | Service Port | `8099` |
@@ -335,7 +335,7 @@ following configurable parameters:
| `server.name` | Name of Pinot Server | `server` |
| `server.port.netty` | Pinot server netty port | `8098` |
| `server.port.admin` | Pinot server admin port | `8097` |
-| `server.replicaCount` | Pinot server replicas | `2` |
+| `server.replicaCount` | Pinot server replicas | `1` |
| `server.dataDir` | Pinot server data directory, should be same as `server.persistence.mountPath` or a sub directory of it | `/var/pinot/server/data/index` |
| `server.segmentTarDir` | Pinot server segment directory, should be same as `server.persistence.mountPath` or a sub directory of it | `/var/pinot/server/data/segments` |
| `server.persistence.enabled` | Use a PVC to persist Pinot Server data | `true` |
@@ -343,7 +343,7 @@ following configurable parameters:
| `server.persistence.size` | Size of data volume | `4G` |
| `server.persistence.mountPath` | Mount path of server data volume | `/var/pinot/server/data` |
| `server.persistence.storageClass` | Storage class of backing PVC | `""` |
-| `server.jvmOpts` | Pinot Server JVM Options | `-Xms4G -Xmx4G -XX:MaxDirectMemorySize=10g` |
+| `server.jvmOpts` | Pinot Server JVM Options | `-Xms512M -Xmx1G` |
| `server.log4j2ConfFile` | Pinot Server log4j2 configuration file | `/opt/pinot/conf/pinot-server-log4j2.xml` |
| `server.pluginsDir` | Pinot Server plugins directory | `/opt/pinot/plugins` |
| `server.service.port` | Service Port | `8098` |
@@ -356,7 +356,7 @@ following configurable parameters:
|------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------|
| `zookeeper.enabled` | If True, installs Zookeeper Chart | `true` |
| `zookeeper.resources` | Zookeeper resource requests and limits | `{}` |
-| `zookeeper.env` | Environmental variables provided to Zookeeper Zookeeper | `{ZK_HEAP_SIZE: "1G"}` |
+| `zookeeper.env` | Environmental variables provided to Zookeeper Zookeeper | `{ZK_HEAP_SIZE: "256M"}` |
| `zookeeper.storage` | Zookeeper Persistent volume size | `2Gi` |
| `zookeeper.image.PullPolicy` | Zookeeper Container pull policy | `IfNotPresent` |
| `zookeeper.url` | URL of Zookeeper Cluster (unneeded if installing Zookeeper Chart) | `""` |
diff --git a/kubernetes/helm/templates/broker/statefulset.yml b/kubernetes/helm/templates/broker/statefulset.yml
index 2a6fa15..4c5fc1e 100644
--- a/kubernetes/helm/templates/broker/statefulset.yml
+++ b/kubernetes/helm/templates/broker/statefulset.yml
@@ -83,7 +83,7 @@ spec:
path: /health
port: {{ .Values.broker.port }}
resources:
- {{ toYaml .Values.broker.resources | indent 12 }}
+{{ toYaml .Values.broker.resources | indent 12 }}
restartPolicy: Always
volumes:
- name: config
diff --git a/kubernetes/helm/templates/server/statefulset.yml b/kubernetes/helm/templates/server/statefulset.yml
index 2f57075..ebcb74d 100644
--- a/kubernetes/helm/templates/server/statefulset.yml
+++ b/kubernetes/helm/templates/server/statefulset.yml
@@ -45,14 +45,14 @@ spec:
release: {{ .Release.Name }}
component: {{ .Values.server.name }}
annotations:
-{{ toYaml .Values.controller.podAnnotations | indent 8 }}
+{{ toYaml .Values.server.podAnnotations | indent 8 }}
spec:
nodeSelector:
-{{ toYaml .Values.controller.nodeSelector | indent 8 }}
+{{ toYaml .Values.server.nodeSelector | indent 8 }}
affinity:
-{{ toYaml .Values.controller.affinity | indent 8 }}
+{{ toYaml .Values.server.affinity | indent 8 }}
tolerations:
-{{ toYaml .Values.controller.tolerations | indent 8 }}
+{{ toYaml .Values.server.tolerations | indent 8 }}
containers:
- name: server
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
@@ -76,6 +76,8 @@ spec:
mountPath: /var/pinot/server/config
- name: data
mountPath: "{{ .Values.server.persistence.mountPath }}"
+ resources:
+{{ toYaml .Values.server.resources | indent 12 }}
restartPolicy: Always
volumes:
- name: config
diff --git a/kubernetes/helm/values.yaml b/kubernetes/helm/values.yaml
index e25e8f3..5f567a7 100644
--- a/kubernetes/helm/values.yaml
+++ b/kubernetes/helm/values.yaml
@@ -30,7 +30,7 @@ cluster:
controller:
name: controller
port: 9000
- replicaCount: 2
+ replicaCount: 1
persistence:
enabled: true
@@ -47,7 +47,7 @@ controller:
host: pinot-controller
port: 9000
- jvmOpts: "-Xms4G -Xmx4G"
+ jvmOpts: "-Xms256M -Xmx1G"
log4j2ConfFile: /opt/pinot/conf/pinot-controller-log4j2.xml
pluginsDir: /opt/pinot/plugins
@@ -85,9 +85,9 @@ broker:
port: 8099
- replicaCount: 2
+ replicaCount: 1
- jvmOpts: "-Xms4G -Xmx4G"
+ jvmOpts: "-Xms256M -Xmx1G"
log4j2ConfFile: /opt/pinot/conf/pinot-broker-log4j2.xml
pluginsDir: /opt/pinot/plugins
@@ -130,7 +130,7 @@ server:
netty: 8098
admin: 8097
- replicaCount: 2
+ replicaCount: 1
dataDir: /var/pinot/server/data/index
segmentTarDir: /var/pinot/server/data/segment
@@ -143,7 +143,7 @@ server:
storageClass: ""
#storageClass: "ssd"
- jvmOpts: "-Xms4G -Xmx4G -XX:MaxDirectMemorySize=10g"
+ jvmOpts: "-Xms512M -Xmx1G"
log4j2ConfFile: /opt/pinot/conf/pinot-server-log4j2.xml
pluginsDir: /opt/pinot/plugins
@@ -182,12 +182,15 @@ zookeeper:
## Configure Zookeeper resource requests and limits
## ref: http://kubernetes.io/docs/user-guide/compute-resources/
- resources: ~
+ resources: {}
+
+ ## Replicas
+ replicaCount: 1
## Environmental variables to set in Zookeeper
env:
## The JVM heap size to allocate to Zookeeper
- ZK_HEAP_SIZE: "1G"
+ ZK_HEAP_SIZE: "256M"
persistence:
enabled: true
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org