You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kyuubi.apache.org by ya...@apache.org on 2021/10/20 10:16:36 UTC

[incubator-kyuubi] branch master updated: [KYUUBI #1255] [KYUUBI#1253] Support use helm to deploy kyuubi server on k8s

This is an automated email from the ASF dual-hosted git repository.

yao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-kyuubi.git


The following commit(s) were added to refs/heads/master by this push:
     new 23d8219  [KYUUBI #1255] [KYUUBI#1253] Support use helm to deploy kyuubi server on k8s
23d8219 is described below

commit 23d821950e1670a6dc1841887cc95d6bd40a5792
Author: hongdongdong <ho...@cmss.chinamobile.com>
AuthorDate: Wed Oct 20 18:16:28 2021 +0800

    [KYUUBI #1255] [KYUUBI#1253] Support use helm to deploy kyuubi server on k8s
    
    <!--
    Thanks for sending a pull request!
    
    Here are some tips for you:
      1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
      2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
      3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
    -->
    
    ### _Why are the changes needed?_
    <!--
    Please clarify why the changes are needed. For instance,
      1. If you add a feature, you can talk about the use case of it.
      2. If you fix a bug, you can clarify why it is a bug.
    -->
    We usally use helm to manager application yaml on k8s, we can use helm to delopy kyuubi server too.
    Usages:
    ```
    helm install kyuubi ${kyuubi_helm_url} -n kyuubi
    ```
    ```
    helm uninstall kyuubi -n kyuubi
    ```
    
    ### _How was this patch tested?_
    - [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
    
    - [ ] Add screenshots for manual tests if appropriate
    
    - [X] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
    
    Closes #1255 from hddong/add-helm.
    
    Closes #1255
    
    e197de34 [hongdongdong] Add doc
    d0f74bbb [hongdongdong] rm hel tgz
    e9a0a2e4 [hongdongdong] update and fix
    f666d331 [hongdongdong] [KYUUBI#1253] Support use helm to deploy kyuubi server on k8s
    
    Authored-by: hongdongdong <ho...@cmss.chinamobile.com>
    Signed-off-by: Kent Yao <ya...@apache.org>
---
 docker/helm/Chart.yaml                       |  46 +++++++++++
 docker/helm/templates/NOTES.txt              |  21 +++++
 docker/helm/templates/_helpers.tpl           |  48 ++++++++++++
 docker/helm/templates/kyuubi-configmap.yaml  |  47 ++++++++++++
 docker/helm/templates/kyuubi-deployment.yaml |  55 +++++++++++++
 docker/helm/templates/kyuubi-service.yaml    |  30 ++++++++
 docker/helm/values.yaml                      |  44 +++++++++++
 docs/quick_start/index.rst                   |   1 +
 docs/quick_start/quick_start_with_helm.md    | 111 +++++++++++++++++++++++++++
 9 files changed, 403 insertions(+)

diff --git a/docker/helm/Chart.yaml b/docker/helm/Chart.yaml
new file mode 100644
index 0000000..787b0c7
--- /dev/null
+++ b/docker/helm/Chart.yaml
@@ -0,0 +1,46 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+apiVersion: v2
+name: kyuubi
+description: A Helm chart for kyuubi server
+
+# A chart can be either an 'application' or a 'library' chart.
+#
+# Application charts are a collection of templates that can be packaged into versioned archives
+# to be deployed.
+#
+# Library charts provide useful utilities or functions for the chart developer. They're included as
+# a dependency of application charts to inject those utilities and functions into the rendering
+# pipeline. Library charts do not define any templates and therefore cannot be deployed.
+type: application
+
+# This is the chart version. This version number should be incremented each time you make changes
+# to the chart and its templates, including the app version.
+# Versions are expected to follow Semantic Versioning (https://semver.org/)
+version: 0.1.0
+
+# This is the version number of the application being deployed. This version number should be
+# incremented each time you make changes to the application. Versions are not expected to
+# follow Semantic Versioning. They should reflect the version the application is using.
+# It is recommended to use it with quotes.
+appVersion: "1.4.0"
+
+icon: https://github.com/apache/incubator-kyuubi/blob/master/docs/imgs/kyuubi_logo.png
+
+sources:
+  - https://github.com/apache/incubator-kyuubi/tree/master/docker
\ No newline at end of file
diff --git a/docker/helm/templates/NOTES.txt b/docker/helm/templates/NOTES.txt
new file mode 100644
index 0000000..44a35b6
--- /dev/null
+++ b/docker/helm/templates/NOTES.txt
@@ -0,0 +1,21 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+Get kyuubi expose URL by running these commands:
+  export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "kyuubi.fullname" . }}-nodeport)
+  export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
+  echo $NODE_IP:$NODE_PORT
\ No newline at end of file
diff --git a/docker/helm/templates/_helpers.tpl b/docker/helm/templates/_helpers.tpl
new file mode 100644
index 0000000..684c1f3
--- /dev/null
+++ b/docker/helm/templates/_helpers.tpl
@@ -0,0 +1,48 @@
+{{/*
+  Licensed to the Apache Software Foundation (ASF) under one or more
+  contributor license agreements.  See the NOTICE file distributed with
+  this work for additional information regarding copyright ownership.
+  The ASF licenses this file to You under the Apache License, Version 2.0
+  (the "License"); you may not use this file except in compliance with
+  the License.  You may obtain a copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+*/}}
+
+{{/*
+Expand the name of the chart.
+*/}}
+{{- define "kyuubi.name" -}}
+{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
+{{- end }}
+
+{{/*
+Create a default fully qualified app name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+If release name contains chart name it will be used as a full name.
+*/}}
+{{- define "kyuubi.fullname" -}}
+{{- if .Values.fullnameOverride }}
+{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- $name := default .Chart.Name .Values.nameOverride }}
+{{- if contains $name .Release.Name }}
+{{- .Release.Name | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
+{{- end }}
+{{- end }}
+{{- end }}
+
+{{/*
+Create chart name and version as used by the chart label.
+*/}}
+{{- define "kyuubi.chart" -}}
+{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
+{{- end }}
\ No newline at end of file
diff --git a/docker/helm/templates/kyuubi-configmap.yaml b/docker/helm/templates/kyuubi-configmap.yaml
new file mode 100644
index 0000000..9a653b9
--- /dev/null
+++ b/docker/helm/templates/kyuubi-configmap.yaml
@@ -0,0 +1,47 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: kyuubi-defaults
+data:
+  kyuubi-defaults.conf: |
+    #
+    # Licensed to the Apache Software Foundation (ASF) under one or more
+    # contributor license agreements.  See the NOTICE file distributed with
+    # this work for additional information regarding copyright ownership.
+    # The ASF licenses this file to You under the Apache License, Version 2.0
+    # (the "License"); you may not use this file except in compliance with
+    # the License.  You may obtain a copy of the License at
+    #
+    #    http://www.apache.org/licenses/LICENSE-2.0
+    #
+    # Unless required by applicable law or agreed to in writing, software
+    # distributed under the License is distributed on an "AS IS" BASIS,
+    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+    # See the License for the specific language governing permissions and
+    # limitations under the License.
+    #
+
+    ## Kyuubi Configurations
+
+    #
+    # kyuubi.authentication           NONE
+    #
+
+    # Details in https://kyuubi.apache.org/docs/latest/deployment/settings.html
\ No newline at end of file
diff --git a/docker/helm/templates/kyuubi-deployment.yaml b/docker/helm/templates/kyuubi-deployment.yaml
new file mode 100644
index 0000000..f3f715d
--- /dev/null
+++ b/docker/helm/templates/kyuubi-deployment.yaml
@@ -0,0 +1,55 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: kyuubi-server
+  labels:
+    app: {{ template "kyuubi.name" . }}
+    release: {{ .Release.Name }}
+    heritage: {{ .Release.Service }}
+spec:
+  replicas: {{ .Values.replicaCount }}
+  selector:
+    matchLabels:
+      app: {{ template "kyuubi.name" . }}
+      release: {{ .Release.Name }}
+  template:
+    metadata:
+      labels:
+        app: {{ template "kyuubi.name" . }}
+        release: {{ .Release.Name }}
+    spec:
+      containers:
+        - name: kyuubi-server
+          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
+          imagePullPolicy: {{ .Values.image.pullPolicy }}
+          env:
+            - name: KYUUBI_JAVA_OPTS
+              value: -Dkyuubi.frontend.bind.host={{ .Values.server.bind.host }} -Dkyuubi.frontend.bind.port={{ .Values.server.bind.port }}
+          ports:
+            - name: frontend-port
+              containerPort: {{ .Values.server.bind.port }}
+              protocol: TCP
+          volumeMounts:
+            - name: kyuubi-defaults
+              mountPath: {{ .Values.server.conf.mountPath }}
+      volumes:
+        - name: kyuubi-defaults
+          configMap:
+            name: kyuubi-defaults
\ No newline at end of file
diff --git a/docker/helm/templates/kyuubi-service.yaml b/docker/helm/templates/kyuubi-service.yaml
new file mode 100644
index 0000000..396b082
--- /dev/null
+++ b/docker/helm/templates/kyuubi-service.yaml
@@ -0,0 +1,30 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+apiVersion: v1
+kind: Service
+metadata:
+  name: kyuubi-nodeport
+spec:
+  ports:
+    - nodePort: {{ .Values.service.port }}
+      port: {{ .Values.server.bind.port }}
+      protocol: TCP
+  type: NodePort
+  selector:
+    app: {{ template "kyuubi.name" . }}
+    release: {{ .Release.Name }}
\ No newline at end of file
diff --git a/docker/helm/values.yaml b/docker/helm/values.yaml
new file mode 100644
index 0000000..d203177
--- /dev/null
+++ b/docker/helm/values.yaml
@@ -0,0 +1,44 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# Default values for kyuubi.
+# This is a YAML-formatted file.
+# Declare variables to be passed into your templates.
+
+# Kyuubi server numbers
+replicaCount: 2
+
+image:
+  repository: apache/kyuubi
+  pullPolicy: Always
+  # Overrides the image tag whose default is the chart appVersion.
+  tag: "master-snapshot"
+
+server:
+  bind:
+    host: 0.0.0.0
+    port: 10009
+  conf:
+    mountPath: /opt/kyuubi/conf
+
+service:
+  type: NodePort
+  # The default port limit of kubernetes is 30000-32767
+  # to change:
+  #   vim kube-apiserver.yaml (usually under path: /etc/kubernetes/manifests/)
+  #   add or change line 'service-node-port-range=1-32767' under kube-apiserver
+  port: 30009
\ No newline at end of file
diff --git a/docs/quick_start/index.rst b/docs/quick_start/index.rst
index b536d3e..64c5f56 100644
--- a/docs/quick_start/index.rst
+++ b/docs/quick_start/index.rst
@@ -26,6 +26,7 @@ In this section, you will learn how to setup and interact with kyuubi quickly.
     :numbered: 3
 
     quick_start
+    quick_start_with_helm
     quick_start_with_dbeaver
     quick_start_with_beeline
     quick_start_with_jdbc
diff --git a/docs/quick_start/quick_start_with_helm.md b/docs/quick_start/quick_start_with_helm.md
new file mode 100644
index 0000000..fd3dfcb
--- /dev/null
+++ b/docs/quick_start/quick_start_with_helm.md
@@ -0,0 +1,111 @@
+<!--
+ - Licensed to the Apache Software Foundation (ASF) under one or more
+ - contributor license agreements.  See the NOTICE file distributed with
+ - this work for additional information regarding copyright ownership.
+ - The ASF licenses this file to You under the Apache License, Version 2.0
+ - (the "License"); you may not use this file except in compliance with
+ - the License.  You may obtain a copy of the License at
+ -
+ -   http://www.apache.org/licenses/LICENSE-2.0
+ -
+ - Unless required by applicable law or agreed to in writing, software
+ - distributed under the License is distributed on an "AS IS" BASIS,
+ - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ - See the License for the specific language governing permissions and
+ - limitations under the License.
+ -->
+
+<div align=center>
+
+![](../imgs/kyuubi_logo.png)
+
+</div>
+
+# Getting Started With Kyuubi on kubernetes
+
+## Running kyuubi with helm
+
+[Helm](https://helm.sh/) is the package manager for Kubernetes,it can be used to find, share, and use software built for Kubernetes.
+
+### Get helm and Install
+
+Please go to [Install Helm](https://helm.sh/docs/intro/install/) page to get and install an appropriate release version for yourself.
+
+### Get Kyuubi Started
+
+#### [Optional] Create namespace on kubernetes
+```bash
+cretate ns kyuubi
+```
+
+#### Get kyuubi started
+```bash
+helm install kyuubi-helm ${KYUUBI_HOME}/docker/helm -n ${namespace_name}
+```
+It will print variables and the way to get kyuubi expose ip and port
+```bash
+NAME: kyuubi-helm
+LAST DEPLOYED: Wed Oct 20 15:22:47 2021
+NAMESPACE: kyuubi
+STATUS: deployed
+REVISION: 1
+TEST SUITE: None
+NOTES:
+Get kyuubi expose URL by running these commands:
+  export NODE_PORT=$(kubectl get --namespace kyuubi -o jsonpath="{.spec.ports[0].nodePort}" services kyuubi-helm-nodeport)
+  export NODE_IP=$(kubectl get nodes --namespace kyuubi -o jsonpath="{.items[0].status.addresses[0].address}")
+  echo $NODE_IP:$NODE_PORT
+```
+
+#### Using hive beeline  
+[Using Hive Beeline](https://kyuubi.readthedocs.io/en/latest/quick_start/quick_start.html#using-hive-beeline) to opening a connection.
+
+#### Remove kyuubi
+```bash
+helm uninstall kyuubi-helm -n ${namespace_name}
+```
+
+#### Edit server config
+
+Modify `values.yaml` under `${KYUUBI_HOME}/docker/helm`
+```yaml
+# Kyuubi server numbers
+replicaCount: 2
+
+image:
+  repository: apache/kyuubi
+  pullPolicy: Always
+  # Overrides the image tag whose default is the chart appVersion.
+  tag: "master-snapshot"
+
+server:
+  bind:
+    host: 0.0.0.0
+    port: 10009
+  conf:
+    mountPath: /opt/kyuubi/conf
+
+service:
+  type: NodePort
+  # The default port limit of kubernetes is 30000-32767
+  # to change:
+  #   vim kube-apiserver.yaml (usually under path: /etc/kubernetes/manifests/)
+  #   add or change line 'service-node-port-range=1-32767' under kube-apiserver
+  port: 30009
+```
+
+#### Get server log  
+List all server pods:
+```bash
+kubectl get po -n ${namespace_name}
+```
+The server pods will print:
+```text
+NAME                             READY   STATUS    RESTARTS   AGE
+kyuubi-server-585d8944c5-m7j5s   1/1     Running   0          30m
+kyuubi-server-32sdsa1245-2d2sj   1/1     Running   0          30m
+```
+then, use pod name to get logs
+```bash
+kubectl -n ${namespace_name} logs kyuubi-server-585d8944c5-m7j5s
+```