You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@submarine.apache.org by pi...@apache.org on 2022/12/06 04:37:21 UTC
[submarine] branch master updated: SUBMARINE-1350. Optimize helm template
This is an automated email from the ASF dual-hosted git repository.
pingsutw pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/submarine.git
The following commit(s) were added to refs/heads/master by this push:
new fd692e2e SUBMARINE-1350. Optimize helm template
fd692e2e is described below
commit fd692e2ea82fcfbdba5fd1baa38d0223b0f95c44
Author: cdmikechen <cd...@apache.org>
AuthorDate: Tue Dec 6 08:07:34 2022 +0800
SUBMARINE-1350. Optimize helm template
### What is this PR for?
Additional configuration of some helm templates to allow users to better follow the subamrine
### What type of PR is it?
Improvement
### Todos
* [x] - Add imagePullSecrets
* [x] - Add description and seldon-core istio gateway config
* [x] - Add README
* [x] - Fix quickstart description
### What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-1350
### How should this be tested?
NA
### Screenshots (if appropriate)
NA
### Questions:
* Do the license files need updating? No
* Are there breaking changes for older versions? No
* Does this need new documentation? Yes
Author: cdmikechen <cd...@apache.org>
Signed-off-by: Kevin <pi...@apache.org>
Closes #1021 from cdmikechen/helm-chart-template and squashes the following commits:
e11f62f6 [cdmikechen] Fix role and resources
1a42afdb [cdmikechen] Fix a typo
9104924f [cdmikechen] Add some descrpiton in readme
d9c35e9c [cdmikechen] change some descriptions and add seldon-core-operator.istio.gateway
92d9a6db [cdmikechen] fix submarine
037388f9 [cdmikechen] add some config descrpitions
a5d37dbe [cdmikechen] update istio gateway label selector and readme
7f69033f [cdmikechen] Changes to the steps and description of the installation
74f74461 [cdmikechen] Add description and seldon-core istio gateway config
27c74c4c [cdmikechen] add imagePullSecrets
---
helm-charts/submarine/Chart.yaml | 8 +-
helm-charts/submarine/README.md | 115 +++++++++++++++++++++
.../charts/notebook-controller/Chart.yaml | 2 +-
.../notebook-controller/templates/deployment.yaml | 10 +-
.../charts/notebook-controller/values.yaml | 15 +++
.../submarine/charts/training-operator/Chart.yaml | 4 +-
.../training-operator/templates/cluster-role.yaml | 5 +
.../training-operator/templates/deployment.yaml | 14 ++-
.../submarine/charts/training-operator/values.yaml | 16 +++
helm-charts/submarine/templates/istio-gateway.yaml | 5 +-
.../submarine/templates/seldon-gateway.yaml | 21 ++--
.../submarine/templates/submarine-operator.yaml | 12 +++
helm-charts/submarine/values.yaml | 27 ++++-
submarine-serve/installation/seldon-secret.yaml | 4 +-
website/docs/gettingStarted/quickstart.md | 13 ++-
15 files changed, 243 insertions(+), 28 deletions(-)
diff --git a/helm-charts/submarine/Chart.yaml b/helm-charts/submarine/Chart.yaml
index e7c71ac0..080462b9 100644
--- a/helm-charts/submarine/Chart.yaml
+++ b/helm-charts/submarine/Chart.yaml
@@ -23,10 +23,12 @@ version: 0.8.0-SNAPSHOT
icon: https://submarine.apache.org/assets/themes/submarine/img/submarine_white_logo.png
dependencies:
- name: training-operator
- version: "0.7.0"
+ version: "0.8.0"
+ condition: training-operator.enabled
- name: notebook-controller
- version: "0.1.0"
+ version: "0.8.0"
+ condition: notebook-controller.enabled
- name: seldon-core-operator
version: "1.10.0"
repository: https://storage.googleapis.com/seldon-charts
- condition: seldon-core-operator.enabled
\ No newline at end of file
+ condition: seldon-core-operator.enabled
diff --git a/helm-charts/submarine/README.md b/helm-charts/submarine/README.md
new file mode 100644
index 00000000..456b5ab1
--- /dev/null
+++ b/helm-charts/submarine/README.md
@@ -0,0 +1,115 @@
+<!---
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# Submarine Operator Deployment Guide
+
+- Installs the Cloud Native Machine Learning Platform [Apache Submarine](https://submarine.apache.org/)
+
+## Debugging the Chart
+
+To debug the chart with the release name `submarine`:
+
+```shell
+# lint
+helm lint ./helm-charts/submarine
+# dry-run command
+helm install --dry-run --debug submarine ./helm-charts/submarine -n submarine
+# or template command
+helm template --debug submarine ./helm-charts/submarine -n submarine
+```
+
+## Installing the Chart
+
+To install the chart with the release name `submarine`:
+
+Submarine requires istio to provide the ingress, we assume you have istio installed on K8s.
+If not, you can refer to https://istio.io/latest/docs/setup/install/
+
+```shell
+# Create namespace
+export NAMESPACE=submarine
+kubectl create namespace ${NAMESPACE}
+
+# Add istio injection
+kubectl label namespace ${NAMESPACE} istio-injection=enabled
+# If istio support `meshConfig.discoverySelectors`, you will need to add this label:
+kubectl label namespace ${NAMESPACE} istio-discovery=enabled
+
+# We have also integrated seldon-core install by helm, thus we need to update our dependency.
+helm dependency update ./helm-charts/submarine
+
+# Install submarine operator in namespace submarine
+helm install submarine ./helm-charts/submarine --set seldon-core-operator.istio.gateway=${NAMESPACE}/seldon-gateway -n ${NAMESPACE}
+```
+
+## Upgrading the Chart
+
+To upgrade the chart with the release name `submarine`:
+
+```shell
+helm upgrade submarine ./helm-charts/submarine -n submarine
+```
+
+## Uninstalling the Chart
+
+To uninstall/delete the `subamrine` deployment:
+
+```shell
+helm uninstall submarine -n submarine
+```
+
+## Upgrading an existing Release to a new major version
+
+A major chart version change (like v0.8.0 -> v1.0.0) indicates that there is an
+incompatible breaking change needing manual actions.
+
+### To 0.8.0
+
+This version requires Helm >= 3.1.0.
+This version is a major change, we migrated `traefik` to `istio` and upgraded the `operator`. You need to back up the
+database and redeploy.
+
+## Configuration
+
+The following table lists the configurable parameters of the MySQL chart and their default values.
+
+| Parameter | Description | Default |
+|----------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------|
+| `name` | Submarine operator deployment name. | `submarine-operator` |
+| `replicas` | Number of operators | `1` |
+| `image` | Submarine operator deployment image | `apache/submarine:operator-0.8.0-SNAPSHOT` |
+| `imagePullSecrets` | Image pull secrets | `[]` |
+| `dev` | Tell helm to install submarine-operator or not in dev mode | `false` |
+| `storageClass.reclaimPolicy` | Determine the action after the persistent volume is released | `Delete` |
+| `storageClass.volumeBindingMode` | Control when volume binding and dynamically provisioning should occur | `Immediate` |
+| `storageClass.provisioner` | Determine what volume plugin is used for provisioning PVs | `k8s.io/minikube-hostpath` |
+| `storageClass.parameters` | Describe volumes belonging to the storage class | `{}` |
+| `clusterType` | k8s cluster type. can be: kubernetes or openshift | `kubernetes` |
+| `podSecurityPolicy.create` | Specifies whether a PodSecurityPolicy should be created, this configuration enables the database/minio/server to set securityContext.runAsUser | `true` |
+| `istio.enabled` | Use istio to expose the service | `true` |
+| `istio.gatewaySelector` | Gateway label selector | `istio: ingressgateway` |
+| `training-operator.enabled` | If we need to deploye a kubeflow training operator in this helm | `true` |
+| `training-operator.image.pullPolicy` | Training operator image pull policy | `IfNotPresent` |
+| `training-operator.image.registry` | Training operator image registry | `public.ecr.aws` |
+| `training-operator.image.repository` | Training operator image repository | `j1r0q0g6/training/training-operator` |
+| `training-operator.image.tag` | Training operator image tag | `760ac1171dd30039a7363ffa03c77454bd714da5` |
+| `training-operator.image.imagePullSecrets` | Training operator image pull Secrets | `[]` |
+| `notebook-controller.enabled` | If we need to deploye a kubeflow notebook controller in this helm | `true` |
+| `notebook-controller.image.pullPolicy` | Notebook controller image pull policy | `IfNotPresent` |
+| `notebook-controller.image.registry` | Notebook controller image registry | `docker.io` |
+| `notebook-controller.image.repository` | Notebook controller image repository | `apache/submarine` |
+| `notebook-controller.image.tag` | Notebook controller image tag | `notebook-controller-v1.4` |
+| `notebook-controller.image.imagePullSecrets` | Notebook controller image pull Secrets | `[]` |
+| `seldon-core-operator.enabled` | If we need to deploye a seldon core operator in this helm | `true` |
+| `seldon-core-operator.istio.gateway` | Seldon istio gateway, if helm is deployed in another namespace, this will need to be modified as well `${namespace}/seldon-gateway` | `submarine/seldon-gateway` |
diff --git a/helm-charts/submarine/charts/notebook-controller/Chart.yaml b/helm-charts/submarine/charts/notebook-controller/Chart.yaml
index 6a2e6d34..6b8c5979 100644
--- a/helm-charts/submarine/charts/notebook-controller/Chart.yaml
+++ b/helm-charts/submarine/charts/notebook-controller/Chart.yaml
@@ -30,7 +30,7 @@ type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
-version: 0.1.0
+version: 0.8.0
# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application.
diff --git a/helm-charts/submarine/charts/notebook-controller/templates/deployment.yaml b/helm-charts/submarine/charts/notebook-controller/templates/deployment.yaml
index e62ad35e..1e9c22cf 100644
--- a/helm-charts/submarine/charts/notebook-controller/templates/deployment.yaml
+++ b/helm-charts/submarine/charts/notebook-controller/templates/deployment.yaml
@@ -31,12 +31,18 @@ spec:
labels:
app: notebook-controller
spec:
+ {{- if .Values.image.imagePullSecrets }}
+ imagePullSecrets:
+ {{- range .Values.image.imagePullSecrets }}
+ - name: {{ . }}
+ {{- end }}
+ {{- end }}
containers:
- name: manager
- image: apache/submarine:notebook-controller-v1.4
+ image: '{{ .Values.image.registry }}/{{ .Values.image.repository }}:{{ .Values.image.tag }}'
command:
- /manager
- imagePullPolicy: IfNotPresent
+ imagePullPolicy: {{ .Values.image.pullPolicy | default "IfNotPresent" | quote }}
env:
- name: USE_ISTIO
value: "false"
diff --git a/helm-charts/submarine/charts/notebook-controller/values.yaml b/helm-charts/submarine/charts/notebook-controller/values.yaml
index cce3acad..54eda75a 100644
--- a/helm-charts/submarine/charts/notebook-controller/values.yaml
+++ b/helm-charts/submarine/charts/notebook-controller/values.yaml
@@ -14,3 +14,18 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
+
+image:
+ # notebook controller image pull policy
+ pullPolicy: IfNotPresent
+ # notebook controller image registry
+ registry: docker.io
+ # notebook controller image repository
+ repository: apache/submarine
+ # notebook controller image tag
+ tag: notebook-controller-v1.4
+ # ImagePullSecrets for notebook-controller ServiceAccount, list of secrets in the same namespace
+ # to use for pulling any images in pods that reference this ServiceAccount.
+ # ImagePullSecrets will be added to the corresponding Deployment(StatefulSet) objects.
+ # Must be set for any cluster configured with private docker registry.
+ imagePullSecrets: []
diff --git a/helm-charts/submarine/charts/training-operator/Chart.yaml b/helm-charts/submarine/charts/training-operator/Chart.yaml
index 0efb733d..2f4f79de 100644
--- a/helm-charts/submarine/charts/training-operator/Chart.yaml
+++ b/helm-charts/submarine/charts/training-operator/Chart.yaml
@@ -31,8 +31,8 @@ type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
-version: 0.7.0
+version: 0.8.0
# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application.
-appVersion: 1.4.0
+appVersion: 1.3.0
diff --git a/helm-charts/submarine/charts/training-operator/templates/cluster-role.yaml b/helm-charts/submarine/charts/training-operator/templates/cluster-role.yaml
index f3dfd5db..a424ca37 100644
--- a/helm-charts/submarine/charts/training-operator/templates/cluster-role.yaml
+++ b/helm-charts/submarine/charts/training-operator/templates/cluster-role.yaml
@@ -18,6 +18,11 @@ rules:
- pytorchjobs/status
- mxjobs/status
- xgboostjobs/status
+ - mpijobs/finalizers
+ - tfjobs/finalizers
+ - pytorchjobs/finalizers
+ - mxjobs/finalizers
+ - xgboostjobs/finalizers
verbs:
- create
- delete
diff --git a/helm-charts/submarine/charts/training-operator/templates/deployment.yaml b/helm-charts/submarine/charts/training-operator/templates/deployment.yaml
index 218e0fa6..6d8deff4 100644
--- a/helm-charts/submarine/charts/training-operator/templates/deployment.yaml
+++ b/helm-charts/submarine/charts/training-operator/templates/deployment.yaml
@@ -33,10 +33,15 @@ spec:
annotations:
sidecar.istio.io/inject: "false"
spec:
+ {{- if .Values.image.imagePullSecrets }}
+ imagePullSecrets:
+ {{- range .Values.image.imagePullSecrets }}
+ - name: {{ . }}
+ {{- end }}
+ {{- end }}
containers:
- command:
- /manager
- image: kubeflow/training-operator
name: training-operator
ports:
- containerPort: 8080
@@ -51,7 +56,8 @@ spec:
fieldPath: metadata.name
securityContext:
allowPrivilegeEscalation: false
- image: public.ecr.aws/j1r0q0g6/training/training-operator:760ac1171dd30039a7363ffa03c77454bd714da5
+ image: '{{ .Values.image.registry }}/{{ .Values.image.repository }}:{{ .Values.image.tag }}'
+ imagePullPolicy: {{ .Values.image.pullPolicy | default "IfNotPresent" | quote }}
livenessProbe:
httpGet:
path: /healthz
@@ -68,8 +74,8 @@ spec:
timeoutSeconds: 10
resources:
limits:
- cpu: 100m
- memory: 30Mi
+ cpu: 500m
+ memory: 500Mi
requests:
cpu: 100m
memory: 20Mi
diff --git a/helm-charts/submarine/charts/training-operator/values.yaml b/helm-charts/submarine/charts/training-operator/values.yaml
index cce3acad..af3da219 100644
--- a/helm-charts/submarine/charts/training-operator/values.yaml
+++ b/helm-charts/submarine/charts/training-operator/values.yaml
@@ -14,3 +14,19 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
+
+image:
+ # training operator image pull policy
+ pullPolicy: IfNotPresent
+ # training operator image registry
+ registry: public.ecr.aws
+ # training operator image repository
+ repository: j1r0q0g6/training/training-operator
+ # training operator image tag
+ # https://github.com/kubeflow/training-operator/blob/v1.3-branch/manifests/overlays/standalone/kustomization.yaml
+ tag: 760ac1171dd30039a7363ffa03c77454bd714da5
+ # ImagePullSecrets for training-operator ServiceAccount, list of secrets in the same namespace
+ # to use for pulling any images in pods that reference this ServiceAccount.
+ # ImagePullSecrets will be added to the corresponding Deployment(StatefulSet) objects.
+ # Must be set for any cluster configured with private docker registry.
+ imagePullSecrets: []
diff --git a/helm-charts/submarine/templates/istio-gateway.yaml b/helm-charts/submarine/templates/istio-gateway.yaml
index e78ea6e4..052762bb 100644
--- a/helm-charts/submarine/templates/istio-gateway.yaml
+++ b/helm-charts/submarine/templates/istio-gateway.yaml
@@ -20,9 +20,12 @@ apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: submarine-gateway
+ namespace: {{ .Release.Namespace }}
spec:
+ {{- with .Values.istio.gatewaySelector }}
selector:
- istio: {{ .Values.istio.gatewaySelector }}
+ {{ toYaml . | indent 2 }}
+ {{- end }}
servers:
- port:
number: 80
diff --git a/helm-charts/submarine/templates/seldon-gateway.yaml b/helm-charts/submarine/templates/seldon-gateway.yaml
index 3c0bbc63..91fb5c4b 100644
--- a/helm-charts/submarine/templates/seldon-gateway.yaml
+++ b/helm-charts/submarine/templates/seldon-gateway.yaml
@@ -6,26 +6,31 @@
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
-# http://www.apache.org/licenses/LICENSE-2.0
+# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
+#
+{{ if .Values.istio.enabled }}
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: seldon-gateway
namespace: {{ .Release.Namespace }}
spec:
+ {{- with .Values.istio.gatewaySelector }}
selector:
- istio: ingressgateway # use istio default controller
+ {{ toYaml . | indent 2 }}
+ {{- end }}
servers:
- - port:
- number: 80
- name: http
- protocol: HTTP
- hosts:
- - "*"
+ - port:
+ number: 80
+ name: http
+ protocol: HTTP
+ hosts:
+ - "*"
+{{ end }}
\ No newline at end of file
diff --git a/helm-charts/submarine/templates/submarine-operator.yaml b/helm-charts/submarine/templates/submarine-operator.yaml
index 5a7b2bef..a763d4fa 100644
--- a/helm-charts/submarine/templates/submarine-operator.yaml
+++ b/helm-charts/submarine/templates/submarine-operator.yaml
@@ -37,6 +37,12 @@ spec:
annotations:
timestamp: {{ now | quote }}
spec:
+ {{- if .Values.imagePullSecrets }}
+ imagePullSecrets:
+ {{- range .Values.imagePullSecrets }}
+ - name: {{ . }}
+ {{- end }}
+ {{- end }}
containers:
- image: "{{ .Values.image }}"
name: "{{ .Values.name }}"
@@ -47,6 +53,12 @@ spec:
value: "{{ .Values.podSecurityPolicy.create }}"
- name: SUBMARINE_CLUSTER_TYPE
value: {{ .Values.clusterType }}
+ - name: SUBMARINE_LEADER_ELECT
+ {{- if gt .Values.replicas 1.0 }}
+ value: "true"
+ {{- else }}
+ value: "false"
+ {{- end }}
serviceAccountName: submarine-operator
status: {}
{{ end }}
diff --git a/helm-charts/submarine/values.yaml b/helm-charts/submarine/values.yaml
index 7a045c70..b151a099 100644
--- a/helm-charts/submarine/values.yaml
+++ b/helm-charts/submarine/values.yaml
@@ -18,6 +18,10 @@
name: submarine-operator
replicas: 1
image: apache/submarine:operator-0.8.0-SNAPSHOT
+# ImagePullSecrets for submarine operator, list of secrets in the same namespace
+# to use for pulling any images in pods that reference this ServiceAccount.
+# Must be set with private docker registry.
+imagePullSecrets: []
# dev is to tell helm to install submarine-operator or not
dev: false
# storageClass is for dynamically creating persistent volumes
@@ -44,12 +48,33 @@ podSecurityPolicy:
# Istio configuration
istio:
enabled: true
- gatewaySelector: ingressgateway
+ # Use Istio default gateway implementation,
+ # you can find the specified gateway service with this labels by the command:
+ # kubectl get service -n istio-system -l istio=ingressgateway
+ gatewaySelector:
+ istio: ingressgateway
+
+# training operator
+training-operator:
+ # If you have already deployed a kubeflow training operator,
+ # you can change this configuration to `false` to prevent conflicts
+ enabled: true
+
+# notebook controller
+notebook-controller:
+ # If you have already deployed a kubeflow notebook controller,
+ # you can change this configuration to `false` to prevent conflicts
+ enabled: true
# seldon-core-operator configuration
seldon-core-operator:
+ # If you have already deployed a seldon core operator,
+ # you can change this configuration to `false` to prevent conflicts
enabled: true
istio:
enabled: true
+ # We temporarily set the namespace to submarine.
+ # If helm is deployed in another namespace, this will need to be modified as well `${namespace}/seldon-gateway`
+ gateway: submarine/seldon-gateway
executor:
defaultEnvSecretRefName: submarine-serve-secret 2> /dev/null
diff --git a/submarine-serve/installation/seldon-secret.yaml b/submarine-serve/installation/seldon-secret.yaml
index 64aed2f4..9d02dfed 100644
--- a/submarine-serve/installation/seldon-secret.yaml
+++ b/submarine-serve/installation/seldon-secret.yaml
@@ -18,7 +18,7 @@ apiVersion: v1
kind: Secret
metadata:
name: submarine-serve-secret
- namespace: submarine-user-test
+ # namespace: submarine-user-test
type: Opaque
stringData:
RCLONE_CONFIG_S3_TYPE: s3
@@ -26,4 +26,4 @@ stringData:
RCLONE_CONFIG_S3_ENV_AUTH: "false"
RCLONE_CONFIG_S3_ACCESS_KEY_ID: submarine_minio
RCLONE_CONFIG_S3_SECRET_ACCESS_KEY: submarine_minio
- RCLONE_CONFIG_S3_ENDPOINT: http://submarine-minio-service:9000
\ No newline at end of file
+ RCLONE_CONFIG_S3_ENDPOINT: http://submarine-minio-service:9000
diff --git a/website/docs/gettingStarted/quickstart.md b/website/docs/gettingStarted/quickstart.md
index d0b58472..9cccbd04 100644
--- a/website/docs/gettingStarted/quickstart.md
+++ b/website/docs/gettingStarted/quickstart.md
@@ -38,10 +38,15 @@ This document gives you a quick view on the basic usage of Submarine platform. Y
2. Start minikube cluster and install Istio
```bash
+# You can go to https://minikube.sigs.k8s.io/docs/start/ and follow the tutorial to install minikube.
+# Then you can start kubernetes with minikube:
minikube start --vm-driver=docker --cpus 8 --memory 8192 --kubernetes-version v1.21.2
-istioctl install -y
# Or if you want to support Pod Security Policy (https://minikube.sigs.k8s.io/docs/tutorials/using_psp), you can use the following command to start cluster
-minikube start --extra-config=apiserver.enable-admission-plugins=PodSecurityPolicy --addons=pod-security-policy --vm-driver=docker --cpus 8 --memory 4096 --kubernetes-version v1.21.2
+minikube start --extra-config=apiserver.enable-admission-plugins=PodSecurityPolicy --addons=pod-security-policy --vm-driver=docker --cpus 8 --memory 8192 --kubernetes-version v1.21.2
+# You can go to the https://github.com/istio/istio/releases/ to download the istioctl for your k8s version
+# e.g. we can execute the following command to download the istio version adapted to k8s 1.21.2
+# wget https://github.com/istio/istio/releases/download/1.13.9/istio-1.13.9-linux-amd64.tar.gz
+istioctl install -y
```
### Launch submarine in the cluster
@@ -76,10 +81,10 @@ helm install submarine ./helm-charts/submarine -n submarine
kubectl apply -f submarine-cloud-v2/artifacts/examples/example-submarine.yaml -n submarine-user-test
```
-5. Install submarine serve package istio
+5. Install submarine serve dependent minio secret key file
```bash
-./submarine-serve/installation/install.sh
+kubectl apply -f ./submarine-serve/installation/seldon-secret.yaml -n submarine-user-test
```
### Ensure submarine is ready
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@submarine.apache.org
For additional commands, e-mail: dev-help@submarine.apache.org