You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@submarine.apache.org by ch...@apache.org on 2022/05/08 11:17:55 UTC
[submarine] branch master updated: SUBMARINE-1099. Replace Traefik with Istio
This is an automated email from the ASF dual-hosted git repository.
chishengliu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/submarine.git
The following commit(s) were added to refs/heads/master by this push:
new 91e0ace5 SUBMARINE-1099. Replace Traefik with Istio
91e0ace5 is described below
commit 91e0ace506ce95d723b90f0e8569c63d67f58061
Author: MortalHappiness <b0...@ntu.edu.tw>
AuthorDate: Tue Mar 29 20:02:44 2022 +0800
SUBMARINE-1099. Replace Traefik with Istio
### What is this PR for?
Replace Traefik with Istio.
### What type of PR is it?
[Improvement]
### Todos
* [x] - Ensure mlflow and tensorboard can be accessed
* [x] - Ensure notebook can be accessed
### What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-1099
### How should this be tested?
### Screenshots (if appropriate)
### Questions:
* Do the license files need updating? No
* Are there breaking changes for older versions? Yes
* Does this need new documentation? Yes
Author: MortalHappiness <b0...@ntu.edu.tw>
Signed-off-by: Chi-Sheng Liu <ch...@apache.org>
Closes #894 from MortalHappiness/SUBMARINE-1099 and squashes the following commits:
262f235f [MortalHappiness] Fix github action failed
c72a9fb2 [MortalHappiness] Reduce IstioOperator sidecar resource requests
2258d3bc [MortalHappiness] Update github actions
93c72101 [MortalHappiness] Fix operator bug
b9ba6e12 [MortalHappiness] Fix python-sdk github action
277e5c6e [MortalHappiness] Increase wait interval in start-submarine.sh
58c6d12c [MortalHappiness] Fix istio install error in github actions
11a295f7 [MortalHappiness] Fix operator rbac
b8b1fc7e [MortalHappiness] Fix operator rbac
3c7d284f [MortalHappiness] Update start-submarine.sh
df398cbf [MortalHappiness] Fix operator image build error
5ef3c279 [MortalHappiness] Fix license error
fdbc4e97 [MortalHappiness] Update github actions
6e328710 [MortalHappiness] Update docs
9e992b70 [MortalHappiness] Fix notebook route
0a2894f9 [MortalHappiness] Fix tensorboard and mlflow route
c588b339 [MortalHappiness] Replace traefik with istio but api fails
---
.github/scripts/start-submarine.sh | 33 +-
.github/workflows/master.yml | 26 +-
.github/workflows/python.yml | 9 +-
dev-support/docker-images/operator/Dockerfile | 3 +-
helm-charts/submarine/Chart.yaml | 2 -
helm-charts/submarine/charts/traefik/.helmignore | 40 ---
helm-charts/submarine/charts/traefik/Chart.yaml | 30 --
.../charts/traefik/crds/ingressroute-tcp.yaml | 29 --
.../charts/traefik/crds/ingressroute-udp.yaml | 30 --
.../submarine/charts/traefik/crds/middlewares.yaml | 29 --
.../submarine/charts/traefik/crds/tls-options.yaml | 29 --
.../submarine/charts/traefik/crds/tls-stores.yaml | 30 --
.../charts/traefik/crds/traefik-services.yaml | 29 --
.../charts/traefik/templates/_helpers.tpl | 71 -----
.../templates/dashboard-hook-ingressroute.yaml | 45 ---
.../charts/traefik/templates/deployment.yaml | 260 ----------------
.../submarine/charts/traefik/templates/hpa.yaml | 37 ---
.../traefik/templates/pod-disruption-budget.yaml | 39 ---
.../submarine/charts/traefik/templates/pvc.yaml | 42 ---
.../templates/rbac/cluster-role-binding.yaml | 35 ---
.../traefik/templates/rbac/cluster-role.yaml | 69 -----
.../traefik/templates/rbac/role-binding.yaml | 35 ---
.../charts/traefik/templates/rbac/role.yaml | 66 ----
.../traefik/templates/rbac/service-account.yaml | 31 --
.../charts/traefik/templates/service.yaml | 126 --------
helm-charts/submarine/charts/traefik/values.yaml | 336 ---------------------
.../istio-gateway.yaml} | 24 +-
helm-charts/submarine/templates/rbac.yaml | 6 +
helm-charts/submarine/values.yaml | 7 +-
submarine-cloud-v2/README.md | 101 -------
.../artifacts/submarine/submarine-ingress.yaml | 67 +++-
.../artifacts/submarine/submarine-minio.yaml | 57 ++--
.../artifacts/submarine/submarine-mlflow.yaml | 19 +-
.../artifacts/submarine/submarine-server.yaml | 1 +
.../artifacts/submarine/submarine-tensorboard.yaml | 57 ++--
submarine-cloud-v2/docs/developer-guide.md | 47 +++
submarine-cloud-v2/go.mod | 11 +-
submarine-cloud-v2/go.sum | 271 ++++++++++++++---
submarine-cloud-v2/main.go | 42 ++-
.../submarine/v1alpha1/zz_generated.deepcopy.go | 1 +
submarine-cloud-v2/pkg/controller/controller.go | 20 +-
.../pkg/controller/controller_builder.go | 8 +-
.../pkg/controller/controller_builder_config.go | 47 ++-
.../pkg/controller/controller_event_handlers.go | 4 +-
submarine-cloud-v2/pkg/controller/parser.go | 12 +
.../pkg/controller/submarine_ingress.go | 25 +-
.../pkg/controller/submarine_minio.go | 36 ---
.../pkg/controller/submarine_mlflow.go | 35 ---
.../pkg/controller/submarine_tensorboard.go | 36 ---
.../serve/istio/IstioHTTPDestination.java | 16 +-
.../submarine/serve/istio/IstioHTTPRoute.java | 5 +
.../submarine/serve/istio/IstioVirtualService.java | 49 +--
.../serve/istio/IstioVirtualServiceSpec.java | 71 +++++
.../submarine/server/api/experiment/Info.java | 9 +-
.../server/api/experiment/MlflowInfo.java | 15 +-
.../server/api/experiment/TensorboardInfo.java | 15 +-
.../server/submitter/k8s/K8sSubmitter.java | 93 ++++--
.../server/submitter/k8s/model/AgentPod.java | 6 +-
website/docs/devDocs/Development.md | 2 +-
website/docs/gettingStarted/quickstart.md | 86 +++---
60 files changed, 801 insertions(+), 2011 deletions(-)
diff --git a/.github/scripts/start-submarine.sh b/.github/scripts/start-submarine.sh
index b58ef4da..c1cb9bd9 100644
--- a/.github/scripts/start-submarine.sh
+++ b/.github/scripts/start-submarine.sh
@@ -17,7 +17,8 @@
#
wait_interval=5
-wait_timeout=900
+wait_timeout=2000
+submarine_user_namespace=default
wait_times=$((wait_timeout / wait_interval))
@@ -25,31 +26,35 @@ wait_times=$((wait_timeout / wait_interval))
sudo ln -s /etc/apparmor.d/usr.sbin.mysqld /etc/apparmor.d/disable/
sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.mysqld
-helm install --wait --set storageClass.provisioner=rancher.io/local-path --set storageClass.volumeBindingMode=WaitForFirstConsumer submarine ./helm-charts/submarine
-kubectl apply -f ./submarine-cloud-v2/artifacts/examples/example-submarine.yaml
+kubectl create namespace submarine
+kubectl create namespace "$submarine_user_namespace"
+kubectl label namespace submarine istio-injection=enabled
+kubectl label namespace "$submarine_user_namespace" istio-injection=enabled
+helm install --wait --set storageClass.provisioner=rancher.io/local-path --set storageClass.volumeBindingMode=WaitForFirstConsumer submarine ./helm-charts/submarine -n submarine
+kubectl apply -f ./submarine-cloud-v2/artifacts/examples/example-submarine.yaml -n "$submarine_user_namespace"
# Polling waiting for the submarine to be in the RUNNING state
for ((i=0;i<$wait_times;++i)); do
- state=$(kubectl get submarine -o=jsonpath='{.items[0].status.submarineState.state}')
+ state=$(kubectl get submarine -n "$submarine_user_namespace" -o=jsonpath='{.items[0].status.submarineState.state}')
if [[ "$state" == "RUNNING" ]]; then
echo "Submarine is running!"
- kubectl describe submarine
- kubectl get all
- kubectl port-forward svc/submarine-database 3306:3306 &
- kubectl port-forward svc/submarine-server 8080:8080 &
- kubectl port-forward svc/submarine-minio-service 9000:9000 &
- kubectl port-forward svc/submarine-mlflow-service 5001:5000 &
+ kubectl describe submarine -n "$submarine_user_namespace"
+ kubectl get all -n "$submarine_user_namespace"
+ kubectl port-forward -n "$submarine_user_namespace" svc/submarine-database 3306:3306 &
+ kubectl port-forward -n "$submarine_user_namespace" svc/submarine-server 8080:8080 &
+ kubectl port-forward -n "$submarine_user_namespace" svc/submarine-minio-service 9000:9000 &
+ kubectl port-forward -n "$submarine_user_namespace" svc/submarine-mlflow-service 5001:5000 &
exit 0
elif [[ "$state" == "FAILED" ]]; then
echo "Submarine failed!" 1>&2
- kubectl describe submarine
- kubectl get all
+ kubectl describe submarine -n "$submarine_user_namespace"
+ kubectl get all -n "$submarine_user_namespace"
exit 1
else
sleep $wait_interval
fi
done
echo "Timeout limit reached!" 1>&2
-kubectl describe submarine
-kubectl get all
+kubectl describe submarine -n "$submarine_user_namespace"
+kubectl get all -n "$submarine_user_namespace"
exit 1
diff --git a/.github/workflows/master.yml b/.github/workflows/master.yml
index a37f1815..e80ab9b1 100644
--- a/.github/workflows/master.yml
+++ b/.github/workflows/master.yml
@@ -41,10 +41,10 @@ jobs:
with:
fetch-depth: 0
path: ${{ env.working-directory }}
- - name: Setup Golang 1.16.2
+ - name: Setup Golang 1.17.2
uses: actions/setup-go@v2
with:
- go-version: "1.16.2"
+ go-version: "1.17.2"
- name: Check golang version
working-directory: ${{ env.working-directory }}/submarine-cloud-v2
run: go version
@@ -74,10 +74,10 @@ jobs:
uses: stCarolas/setup-maven@v4
with:
maven-version: 3.6.3
- - name: Setup Golang 1.16.2
+ - name: Setup Golang 1.17.2
uses: actions/setup-go@v2
with:
- go-version: "1.16.2"
+ go-version: "1.17.2"
- uses: actions/cache@v2
with:
path: ~/.m2/repository
@@ -93,6 +93,13 @@ jobs:
kind version
- name: Create kind cluster
run: kind create cluster --config ./.github/config/kind-config-kind.yaml --wait 3m --image kindest/node:${{ matrix.k8s-version }}
+ - name: Download Istio 1.13.0
+ run: |
+ curl -L https://istio.io/downloadIstio | ISTIO_VERSION=1.13.0 sh -
+ cd istio-1.13.0
+ echo "$PWD/bin" >> $GITHUB_PATH
+ - name: Install Istio
+ run: istioctl install -y --set values.global.proxy.resources.requests.cpu=10m
- name: Show K8s cluster information
run: |
kubectl cluster-info
@@ -214,6 +221,13 @@ jobs:
kind version
- name: Create kind cluster
run: kind create cluster --config ./.github/config/kind-config-kind.yaml --wait 3m --image kindest/node:${{ matrix.k8s-version }}
+ - name: Download Istio 1.13.0
+ run: |
+ curl -L https://istio.io/downloadIstio | ISTIO_VERSION=1.13.0 sh -
+ cd istio-1.13.0
+ echo "$PWD/bin" >> $GITHUB_PATH
+ - name: Install Istio
+ run: istioctl install -y --set values.global.proxy.resources.requests.cpu=10m
- name: Show K8s cluster information
run: |
kubectl cluster-info
@@ -449,7 +463,7 @@ jobs:
fetch-depth: 50
- uses: actions/setup-node@v3
with:
- node-version: '14'
+ node-version: "14"
- name: Check version
run: node --version
- name: Install dependencies
@@ -525,7 +539,7 @@ jobs:
go-version: "1.16.2"
- uses: actions/setup-node@v3
with:
- node-version: '14'
+ node-version: "14"
- name: Check version
run: |
mvn --version
diff --git a/.github/workflows/python.yml b/.github/workflows/python.yml
index f2369ff1..ca88e49f 100644
--- a/.github/workflows/python.yml
+++ b/.github/workflows/python.yml
@@ -83,6 +83,13 @@ jobs:
java -version
- name: Create kind cluster
run: kind create cluster --config ./.github/config/kind-config-kind.yaml --wait 3m --image kindest/node:${KUBERNETES_VERSION}
+ - name: Download Istio 1.13.0
+ run: |
+ curl -L https://istio.io/downloadIstio | ISTIO_VERSION=1.13.0 sh -
+ cd istio-1.13.0
+ echo "$PWD/bin" >> $GITHUB_PATH
+ - name: Install Istio
+ run: istioctl install -y --set values.global.proxy.resources.requests.cpu=10m
- name: Show K8s cluster information
run: |
kubectl cluster-info
@@ -125,4 +132,4 @@ jobs:
kubectl describe nodes
kubectl get pods
kubectl -n default get events --sort-by='{.lastTimestamp}'
- if: ${{ failure() }}
\ No newline at end of file
+ if: ${{ failure() }}
diff --git a/dev-support/docker-images/operator/Dockerfile b/dev-support/docker-images/operator/Dockerfile
index 3f2795a5..029bdedd 100644
--- a/dev-support/docker-images/operator/Dockerfile
+++ b/dev-support/docker-images/operator/Dockerfile
@@ -13,7 +13,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-FROM golang:1.16.2 AS build-image
+FROM golang:1.17.2 AS build-image
+MAINTAINER Apache Software Foundation <de...@submarine.apache.org>
ADD tmp/submarine-cloud-v2 /usr/src
diff --git a/helm-charts/submarine/Chart.yaml b/helm-charts/submarine/Chart.yaml
index ed831ac0..b70b9f74 100644
--- a/helm-charts/submarine/Chart.yaml
+++ b/helm-charts/submarine/Chart.yaml
@@ -26,5 +26,3 @@ dependencies:
version: "0.1.0"
- name: notebook-controller
version: "0.1.0"
- - name: traefik
- version: "9.1.0"
diff --git a/helm-charts/submarine/charts/traefik/.helmignore b/helm-charts/submarine/charts/traefik/.helmignore
deleted file mode 100644
index cab781ad..00000000
--- a/helm-charts/submarine/charts/traefik/.helmignore
+++ /dev/null
@@ -1,40 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-# Patterns to ignore when building packages.
-# This supports shell glob matching, relative path matching, and
-# negation (prefixed with !). Only one pattern per line.
-.DS_Store
-# Common VCS dirs
-.git/
-.gitignore
-.bzr/
-.bzrignore
-.hg/
-.hgignore
-.svn/
-# Common backup files
-*.swp
-*.bak
-*.tmp
-*.orig
-*~
-# Various IDEs
-.project
-.idea/
-*.tmproj
-.vscode/
diff --git a/helm-charts/submarine/charts/traefik/Chart.yaml b/helm-charts/submarine/charts/traefik/Chart.yaml
deleted file mode 100644
index 5ec6bc56..00000000
--- a/helm-charts/submarine/charts/traefik/Chart.yaml
+++ /dev/null
@@ -1,30 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-apiVersion: v2
-name: traefik
-description: A Traefik based Kubernetes ingress controller
-type: application
-version: 9.1.0
-appVersion: 2.2.8
-keywords:
- - traefik
- - ingress
-home: https://traefik.io/
-sources:
- - https://github.com/containous/traefik
-icon: https://raw.githubusercontent.com/containous/traefik/v2.2/docs/content/assets/img/traefik.logo.png
diff --git a/helm-charts/submarine/charts/traefik/crds/ingressroute-tcp.yaml b/helm-charts/submarine/charts/traefik/crds/ingressroute-tcp.yaml
deleted file mode 100644
index 7dbaceae..00000000
--- a/helm-charts/submarine/charts/traefik/crds/ingressroute-tcp.yaml
+++ /dev/null
@@ -1,29 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
- name: ingressroutetcps.traefik.containo.us
-spec:
- group: traefik.containo.us
- version: v1alpha1
- names:
- kind: IngressRouteTCP
- plural: ingressroutetcps
- singular: ingressroutetcp
- scope: Namespaced
diff --git a/helm-charts/submarine/charts/traefik/crds/ingressroute-udp.yaml b/helm-charts/submarine/charts/traefik/crds/ingressroute-udp.yaml
deleted file mode 100644
index 4efa15da..00000000
--- a/helm-charts/submarine/charts/traefik/crds/ingressroute-udp.yaml
+++ /dev/null
@@ -1,30 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
- name: ingressrouteudps.traefik.containo.us
-
-spec:
- group: traefik.containo.us
- version: v1alpha1
- names:
- kind: IngressRouteUDP
- plural: ingressrouteudps
- singular: ingressrouteudp
- scope: Namespaced
diff --git a/helm-charts/submarine/charts/traefik/crds/middlewares.yaml b/helm-charts/submarine/charts/traefik/crds/middlewares.yaml
deleted file mode 100644
index 8b19370d..00000000
--- a/helm-charts/submarine/charts/traefik/crds/middlewares.yaml
+++ /dev/null
@@ -1,29 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
- name: middlewares.traefik.containo.us
-spec:
- group: traefik.containo.us
- version: v1alpha1
- names:
- kind: Middleware
- plural: middlewares
- singular: middleware
- scope: Namespaced
diff --git a/helm-charts/submarine/charts/traefik/crds/tls-options.yaml b/helm-charts/submarine/charts/traefik/crds/tls-options.yaml
deleted file mode 100644
index 4bc26976..00000000
--- a/helm-charts/submarine/charts/traefik/crds/tls-options.yaml
+++ /dev/null
@@ -1,29 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
- name: tlsoptions.traefik.containo.us
-spec:
- group: traefik.containo.us
- version: v1alpha1
- names:
- kind: TLSOption
- plural: tlsoptions
- singular: tlsoption
- scope: Namespaced
diff --git a/helm-charts/submarine/charts/traefik/crds/tls-stores.yaml b/helm-charts/submarine/charts/traefik/crds/tls-stores.yaml
deleted file mode 100644
index 72834546..00000000
--- a/helm-charts/submarine/charts/traefik/crds/tls-stores.yaml
+++ /dev/null
@@ -1,30 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
- name: tlsstores.traefik.containo.us
-
-spec:
- group: traefik.containo.us
- version: v1alpha1
- names:
- kind: TLSStore
- plural: tlsstores
- singular: tlsstore
- scope: Namespaced
diff --git a/helm-charts/submarine/charts/traefik/crds/traefik-services.yaml b/helm-charts/submarine/charts/traefik/crds/traefik-services.yaml
deleted file mode 100644
index 7eb8ab9a..00000000
--- a/helm-charts/submarine/charts/traefik/crds/traefik-services.yaml
+++ /dev/null
@@ -1,29 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
-metadata:
- name: traefikservices.traefik.containo.us
-spec:
- group: traefik.containo.us
- version: v1alpha1
- names:
- kind: TraefikService
- plural: traefikservices
- singular: traefikservice
- scope: Namespaced
diff --git a/helm-charts/submarine/charts/traefik/templates/_helpers.tpl b/helm-charts/submarine/charts/traefik/templates/_helpers.tpl
deleted file mode 100644
index d599823e..00000000
--- a/helm-charts/submarine/charts/traefik/templates/_helpers.tpl
+++ /dev/null
@@ -1,71 +0,0 @@
-{{/*
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-*/}}
-
-{{/* vim: set filetype=mustache: */}}
-
-{{/*
-Expand the name of the chart.
-*/}}
-{{- define "traefik.name" -}}
-{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
-{{- end -}}
-
-{{/*
-Create chart name and version as used by the chart label.
-*/}}
-{{- define "traefik.chart" -}}
-{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
-{{- end -}}
-
-{{/*
-Create a default fully qualified app name.
-We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
-If release name contains chart name it will be used as a full name.
-*/}}
-{{- define "traefik.fullname" -}}
-{{- if .Values.fullnameOverride -}}
-{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
-{{- else -}}
-{{- $name := default .Chart.Name .Values.nameOverride -}}
-{{- if contains $name .Release.Name -}}
-{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
-{{- else -}}
-{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
-{{- end -}}
-{{- end -}}
-{{- end -}}
-
-{{/*
-The name of the service account to use
-*/}}
-{{- define "traefik.serviceAccountName" -}}
-{{- default (include "traefik.fullname" .) .Values.serviceAccount.name -}}
-{{- end -}}
-
-{{/*
-Construct the path for the providers.kubernetesingress.ingressendpoint.publishedservice.
-By convention this will simply use the <namespace>/<service-name> to match the name of the
-service generated.
-Users can provide an override for an explicit service they want bound via `.Values.providers.kubernetesIngress.publishedService.pathOverride`
-*/}}
-{{- define "providers.kubernetesIngress.publishedServicePath" -}}
-{{- $defServiceName := printf "%s/%s" .Release.Namespace (include "traefik.fullname" .) -}}
-{{- $servicePath := default $defServiceName .Values.providers.kubernetesIngress.publishedService.pathOverride }}
-{{- print $servicePath | trimSuffix "-" -}}
-{{- end -}}
diff --git a/helm-charts/submarine/charts/traefik/templates/dashboard-hook-ingressroute.yaml b/helm-charts/submarine/charts/traefik/templates/dashboard-hook-ingressroute.yaml
deleted file mode 100644
index 9565b937..00000000
--- a/helm-charts/submarine/charts/traefik/templates/dashboard-hook-ingressroute.yaml
+++ /dev/null
@@ -1,45 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-{{- if .Values.ingressRoute.dashboard.enabled -}}
-apiVersion: traefik.containo.us/v1alpha1
-kind: IngressRoute
-metadata:
- name: {{ template "traefik.fullname" . }}-dashboard
- annotations:
- helm.sh/hook: "post-install,post-upgrade"
- {{- with .Values.ingressRoute.dashboard.annotations }}
- {{- toYaml . | nindent 4 }}
- {{- end }}
- labels:
- app.kubernetes.io/name: {{ template "traefik.name" . }}
- helm.sh/chart: {{ template "traefik.chart" . }}
- app.kubernetes.io/managed-by: {{ .Release.Service }}
- app.kubernetes.io/instance: {{ .Release.Name }}
- {{- with .Values.ingressRoute.dashboard.labels }}
- {{- toYaml . | nindent 4 }}
- {{- end }}
-spec:
- entryPoints:
- - traefik
- routes:
- - match: PathPrefix(`/dashboard`) || PathPrefix(`/api`)
- kind: Rule
- services:
- - name: api@internal
- kind: TraefikService
-{{- end -}}
diff --git a/helm-charts/submarine/charts/traefik/templates/deployment.yaml b/helm-charts/submarine/charts/traefik/templates/deployment.yaml
deleted file mode 100644
index 0c6e65d0..00000000
--- a/helm-charts/submarine/charts/traefik/templates/deployment.yaml
+++ /dev/null
@@ -1,260 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-{{- if .Values.deployment.enabled -}}
- {{- if gt (int .Values.deployment.replicas) 1 -}}
- {{- with .Values.additionalArguments -}}
- {{- range . -}}
- {{- if contains ".acme." . -}}
- {{- fail (printf "You can not enabled acme if you set more than one traefik replica") -}}
- {{- end -}}
- {{- end -}}
- {{- end -}}
- {{- end -}}
-
----
-apiVersion: apps/v1
-kind: Deployment
-metadata:
- name: {{ template "traefik.fullname" . }}
- labels:
- app.kubernetes.io/name: {{ template "traefik.name" . }}
- helm.sh/chart: {{ template "traefik.chart" . }}
- app.kubernetes.io/managed-by: {{ .Release.Service }}
- app.kubernetes.io/instance: {{ .Release.Name }}
- annotations:
- {{- with .Values.deployment.annotations }}
- {{- toYaml . | nindent 4 }}
- {{- end }}
-spec:
- {{- if not .Values.autoscaling.enabled }}
- replicas: {{ default 1 .Values.deployment.replicas }}
- {{- end }}
- selector:
- matchLabels:
- app.kubernetes.io/name: {{ template "traefik.name" . }}
- app.kubernetes.io/instance: {{ .Release.Name }}
- strategy:
- type: RollingUpdate
- rollingUpdate:
- {{- with .Values.rollingUpdate }}
- {{- toYaml . | nindent 6 }}
- {{- end }}
- template:
- metadata:
- annotations:
- {{- with .Values.deployment.podAnnotations }}
- {{- toYaml . | nindent 8 }}
- {{- end }}
- labels:
- app.kubernetes.io/name: {{ template "traefik.name" . }}
- helm.sh/chart: {{ template "traefik.chart" . }}
- app.kubernetes.io/managed-by: {{ .Release.Service }}
- app.kubernetes.io/instance: {{ .Release.Name }}
- spec:
- serviceAccountName: {{ include "traefik.serviceAccountName" . }}
- terminationGracePeriodSeconds: 60
- hostNetwork: {{ .Values.hostNetwork }}
- {{- with .Values.deployment.dnsPolicy }}
- dnsPolicy: {{ . }}
- {{- end }}
- {{- with .Values.deployment.initContainers }}
- initContainers:
- {{- toYaml . | nindent 6 }}
- {{- end }}
- containers:
- - image: {{ .Values.image.name }}:{{ .Values.image.tag }}
- imagePullPolicy: {{ .Values.image.pullPolicy }}
- name: {{ template "traefik.fullname" . }}
- resources:
- {{- with .Values.resources }}
- {{- toYaml . | nindent 10 }}
- {{- end }}
- readinessProbe:
- httpGet:
- path: /ping
- port: {{ .Values.ports.traefik.port }}
- failureThreshold: 1
- initialDelaySeconds: 10
- periodSeconds: 10
- successThreshold: 1
- timeoutSeconds: 2
- livenessProbe:
- httpGet:
- path: /ping
- port: {{ .Values.ports.traefik.port }}
- failureThreshold: 3
- initialDelaySeconds: 10
- periodSeconds: 10
- successThreshold: 1
- timeoutSeconds: 2
- ports:
- {{- range $name, $config := .Values.ports }}
- {{- if $config }}
- - name: {{ $name | quote }}
- containerPort: {{ $config.port }}
- {{- if $config.hostPort }}
- hostPort: {{ $config.hostPort }}
- {{- end }}
- {{- if $config.hostIP }}
- hostIP: {{ $config.hostIP }}
- {{- end }}
- protocol: {{ default "TCP" $config.protocol | quote }}
- {{- end }}
- {{- end }}
- {{- with .Values.securityContext }}
- securityContext:
- {{- toYaml . | nindent 10 }}
- {{- end }}
- volumeMounts:
- - name: data
- mountPath: {{ .Values.persistence.path }}
- {{- if .Values.persistence.subPath }}
- subPath: {{ .Values.persistence.subPath }}
- {{- end }}
- - name: tmp
- mountPath: /tmp
- {{- range .Values.volumes }}
- - name: {{ .name }}
- mountPath: {{ .mountPath }}
- readOnly: true
- {{- end }}
- args:
- {{- with .Values.globalArguments }}
- {{- range . }}
- - {{ . | quote }}
- {{- end }}
- {{- end }}
- {{- range $name, $config := .Values.ports }}
- {{- if $config }}
- - "--entryPoints.{{$name}}.address=:{{ $config.port }}/{{ default "tcp" $config.protocol | lower }}"
- {{- end }}
- {{- end }}
- - "--api.dashboard=true"
- - "--ping=true"
- {{- if .Values.providers.kubernetesCRD.enabled }}
- - "--providers.kubernetescrd"
- {{- end }}
- {{- if .Values.providers.kubernetesIngress.enabled }}
- - "--providers.kubernetesingress"
- {{- if and .Values.service.enabled .Values.providers.kubernetesIngress.publishedService.enabled }}
- - "--providers.kubernetesingress.ingressendpoint.publishedservice={{ template "providers.kubernetesIngress.publishedServicePath" . }}"
- {{- end }}
- {{- end }}
- {{- if and .Values.rbac.enabled .Values.rbac.namespaced }}
- - "--providers.kubernetescrd.namespaces={{ .Release.Namespace }}"
- - "--providers.kubernetesingress.namespaces={{ .Release.Namespace }}"
- {{- end }}
- {{- range $entrypoint, $config := $.Values.ports }}
- {{- if $config.redirectTo }}
- {{- $toPort := index $.Values.ports $config.redirectTo }}
- - "--entrypoints.{{ $entrypoint }}.http.redirections.entryPoint.to=:{{ $toPort.exposedPort }}"
- - "--entrypoints.{{ $entrypoint }}.http.redirections.entryPoint.scheme=https"
- {{- end }}
- {{- end }}
- {{- with .Values.logs }}
- {{- if .general.format }}
- - "--log.format={{ .general.format }}"
- {{- end }}
- {{- if ne .general.level "ERROR" }}
- - "--log.level={{ .general.level | upper }}"
- {{- end }}
- {{- if .access.enabled }}
- - "--accesslog=true"
- {{- if .access.format }}
- - "--accesslog.format={{ .access.format }}"
- {{- end }}
- {{- if .access.bufferingsize }}
- - "--accesslog.bufferingsize={{ .access.bufferingsize }}"
- {{- end }}
- {{- if .access.filters }}
- {{- if .access.filters.statuscodes }}
- - "--accesslog.filters.statuscodes={{ .access.filters.statuscodes }}"
- {{- end }}
- {{- if .access.filters.retryattempts }}
- - "--accesslog.filters.retryattempts"
- {{- end }}
- {{- if .access.filters.minduration }}
- - "--accesslog.filters.minduration={{ .access.filters.minduration }}"
- {{- end }}
- {{- end }}
- - "--accesslog.fields.defaultmode={{ .access.fields.general.defaultmode }}"
- {{- range $fieldname, $fieldaction := .access.fields.general.names }}
- - "--accesslog.fields.names.{{ $fieldname }}={{ $fieldaction }}"
- {{- end }}
- - "--accesslog.fields.headers.defaultmode={{ .access.fields.headers.defaultmode }}"
- {{- range $fieldname, $fieldaction := .access.fields.headers.names }}
- - "--accesslog.fields.headers.names.{{ $fieldname }}={{ $fieldaction }}"
- {{- end }}
- {{- end }}
- {{- end }}
- {{- with .Values.additionalArguments }}
- {{- range . }}
- - {{ . | quote }}
- {{- end }}
- {{- end }}
- {{- with .Values.env }}
- env:
- {{- toYaml . | nindent 10 }}
- {{- end }}
- {{- with .Values.envFrom }}
- envFrom:
- {{- toYaml . | nindent 10 }}
- {{- end }}
- {{- if .Values.deployment.additionalContainers }}
- {{- toYaml .Values.deployment.additionalContainers | nindent 6 }}
- {{- end }}
- volumes:
- - name: data
- {{- if .Values.persistence.enabled }}
- persistentVolumeClaim:
- claimName: {{ default (include "traefik.fullname" .) .Values.persistence.existingClaim }}
- {{- else }}
- emptyDir: {}
- {{- end }}
- - name: tmp
- emptyDir: {}
- {{- range .Values.volumes }}
- - name: {{ .name }}
- {{- if eq .type "secret" }}
- secret:
- secretName: {{ .name }}
- {{- else if eq .type "configMap" }}
- configMap:
- name: {{ .name }}
- {{- end }}
- {{- end }}
- {{- with .Values.affinity }}
- affinity:
- {{- toYaml . | nindent 8 }}
- {{- end }}
- {{- with .Values.tolerations }}
- tolerations:
- {{- toYaml . | nindent 8 }}
- {{- end }}
- {{- with .Values.nodeSelector }}
- nodeSelector:
- {{- toYaml . | nindent 8 }}
- {{- end }}
- {{- if .Values.priorityClassName }}
- priorityClassName: {{ .Values.priorityClassName }}
- {{- end }}
- {{- with .Values.podSecurityContext }}
- securityContext:
- {{- toYaml . | nindent 8 }}
- {{- end }}
-{{- end -}}
diff --git a/helm-charts/submarine/charts/traefik/templates/hpa.yaml b/helm-charts/submarine/charts/traefik/templates/hpa.yaml
deleted file mode 100644
index 9ecdd16f..00000000
--- a/helm-charts/submarine/charts/traefik/templates/hpa.yaml
+++ /dev/null
@@ -1,37 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-{{- if .Values.autoscaling.enabled }}
-apiVersion: autoscaling/v2beta1
-kind: HorizontalPodAutoscaler
-metadata:
- name: {{ template "traefik.fullname" . }}
- labels:
- app.kubernetes.io/name: {{ template "traefik.name" . }}
- helm.sh/chart: {{ template "traefik.chart" . }}
- app.kubernetes.io/managed-by: {{ .Release.Service }}
- app.kubernetes.io/instance: {{ .Release.Name }}
-spec:
- scaleTargetRef:
- apiVersion: apps/v1
- kind: Deployment
- name: {{ template "traefik.fullname" . }}
- minReplicas: {{ .Values.autoscaling.minReplicas }}
- maxReplicas: {{ .Values.autoscaling.maxReplicas }}
- metrics:
-{{ toYaml .Values.autoscaling.metrics | indent 4 }}
-{{- end }}
diff --git a/helm-charts/submarine/charts/traefik/templates/pod-disruption-budget.yaml b/helm-charts/submarine/charts/traefik/templates/pod-disruption-budget.yaml
deleted file mode 100644
index aa460e91..00000000
--- a/helm-charts/submarine/charts/traefik/templates/pod-disruption-budget.yaml
+++ /dev/null
@@ -1,39 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-{{- if .Values.podDisruptionBudget.enabled -}}
-apiVersion: policy/v1beta1
-kind: PodDisruptionBudget
-metadata:
- name: {{ template "traefik.fullname" . }}
- labels:
- app.kubernetes.io/name: {{ template "traefik.name" . }}
- helm.sh/chart: {{ template "traefik.chart" . }}
- app.kubernetes.io/managed-by: {{ .Release.Service }}
- app.kubernetes.io/instance: {{ .Release.Name }}
-spec:
- selector:
- matchLabels:
- app.kubernetes.io/name: {{ template "traefik.name" . }}
- app.kubernetes.io/instance: {{ .Release.Name }}
- {{- if .Values.podDisruptionBudget.minAvailable }}
- minAvailable: {{ .Values.podDisruptionBudget.minAvailable | int }}
- {{- end }}
- {{- if .Values.podDisruptionBudget.maxUnavailable }}
- maxUnavailable: {{ .Values.podDisruptionBudget.maxUnavailable | int }}
- {{- end }}
-{{- end -}}
diff --git a/helm-charts/submarine/charts/traefik/templates/pvc.yaml b/helm-charts/submarine/charts/traefik/templates/pvc.yaml
deleted file mode 100644
index 0993ea10..00000000
--- a/helm-charts/submarine/charts/traefik/templates/pvc.yaml
+++ /dev/null
@@ -1,42 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-{{- if and .Values.persistence.enabled (not .Values.persistence.existingClaim) -}}
-apiVersion: v1
-kind: PersistentVolumeClaim
-metadata:
- name: {{ template "traefik.fullname" . }}
- annotations:
- {{- with .Values.persistence.annotations }}
- {{ toYaml . | indent 4 }}
- {{- end }}
- labels:
- app.kubernetes.io/name: {{ template "traefik.name" . }}
- helm.sh/chart: {{ template "traefik.chart" . }}
- app.kubernetes.io/managed-by: {{ .Release.Service }}
- app.kubernetes.io/instance: {{ .Release.Name }}
-spec:
- accessModes:
- - {{ .Values.persistence.accessMode | quote }}
- resources:
- requests:
- storage: {{ .Values.persistence.size | quote }}
- {{- if .Values.persistence.storageClass }}
- storageClassName: {{ .Values.persistence.storageClass | quote }}
- {{- end }}
-{{- end -}}
-
diff --git a/helm-charts/submarine/charts/traefik/templates/rbac/cluster-role-binding.yaml b/helm-charts/submarine/charts/traefik/templates/rbac/cluster-role-binding.yaml
deleted file mode 100644
index d1c5c958..00000000
--- a/helm-charts/submarine/charts/traefik/templates/rbac/cluster-role-binding.yaml
+++ /dev/null
@@ -1,35 +0,0 @@
-{{- if and .Values.rbac.enabled (not .Values.rbac.namespaced) }}
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-kind: ClusterRoleBinding
-apiVersion: rbac.authorization.k8s.io/v1
-metadata:
- name: {{ template "traefik.fullname" . }}
- labels:
- app.kubernetes.io/name: {{ template "traefik.name" . }}
- helm.sh/chart: {{ template "traefik.chart" . }}
- app.kubernetes.io/managed-by: {{ .Release.Service }}
- app.kubernetes.io/instance: {{ .Release.Name }}
-roleRef:
- apiGroup: rbac.authorization.k8s.io
- kind: ClusterRole
- name: {{ template "traefik.fullname" . }}
-subjects:
- - kind: ServiceAccount
- name: {{ include "traefik.serviceAccountName" . }}
- namespace: {{ .Release.Namespace }}
-{{- end -}}
diff --git a/helm-charts/submarine/charts/traefik/templates/rbac/cluster-role.yaml b/helm-charts/submarine/charts/traefik/templates/rbac/cluster-role.yaml
deleted file mode 100644
index 5d044add..00000000
--- a/helm-charts/submarine/charts/traefik/templates/rbac/cluster-role.yaml
+++ /dev/null
@@ -1,69 +0,0 @@
-{{- if and .Values.rbac.enabled (not .Values.rbac.namespaced) -}}
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-kind: ClusterRole
-apiVersion: rbac.authorization.k8s.io/v1
-metadata:
- name: {{ template "traefik.fullname" . }}
- labels:
- app.kubernetes.io/name: {{ template "traefik.name" . }}
- helm.sh/chart: {{ template "traefik.chart" . }}
- app.kubernetes.io/managed-by: {{ .Release.Service }}
- app.kubernetes.io/instance: {{ .Release.Name }}
-rules:
- - apiGroups:
- - ""
- resources:
- - services
- - endpoints
- - secrets
- verbs:
- - get
- - list
- - watch
- - apiGroups:
- - extensions
- resources:
- - ingresses
- verbs:
- - get
- - list
- - watch
- - apiGroups:
- - extensions
- resources:
- - ingresses/status
- verbs:
- - update
- - apiGroups:
- - traefik.containo.us
- resources:
- - ingressroutes
- - ingressroutetcps
- - ingressrouteudps
- - middlewares
- - tlsoptions
- - tlsstores
- - traefikservices
- verbs:
- - get
- - list
- - watch
- - create
- - update
- - delete
-{{- end -}}
diff --git a/helm-charts/submarine/charts/traefik/templates/rbac/role-binding.yaml b/helm-charts/submarine/charts/traefik/templates/rbac/role-binding.yaml
deleted file mode 100644
index d3d6996a..00000000
--- a/helm-charts/submarine/charts/traefik/templates/rbac/role-binding.yaml
+++ /dev/null
@@ -1,35 +0,0 @@
-{{- if and .Values.rbac.enabled .Values.rbac.namespaced }}
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-kind: RoleBinding
-apiVersion: rbac.authorization.k8s.io/v1
-metadata:
- name: {{ template "traefik.fullname" . }}
- labels:
- app.kubernetes.io/name: {{ template "traefik.name" . }}
- helm.sh/chart: {{ template "traefik.chart" . }}
- app.kubernetes.io/managed-by: {{ .Release.Service }}
- app.kubernetes.io/instance: {{ .Release.Name }}
-roleRef:
- apiGroup: rbac.authorization.k8s.io
- kind: Role
- name: {{ template "traefik.fullname" . }}
-subjects:
- - kind: ServiceAccount
- name: {{ include "traefik.serviceAccountName" . }}
- namespace: {{ .Release.Namespace }}
-{{- end -}}
diff --git a/helm-charts/submarine/charts/traefik/templates/rbac/role.yaml b/helm-charts/submarine/charts/traefik/templates/rbac/role.yaml
deleted file mode 100644
index c0775f40..00000000
--- a/helm-charts/submarine/charts/traefik/templates/rbac/role.yaml
+++ /dev/null
@@ -1,66 +0,0 @@
-{{- if and .Values.rbac.enabled .Values.rbac.namespaced }}
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-kind: Role
-apiVersion: rbac.authorization.k8s.io/v1
-metadata:
- name: {{ template "traefik.fullname" . }}
- labels:
- app.kubernetes.io/name: {{ template "traefik.name" . }}
- helm.sh/chart: {{ template "traefik.chart" . }}
- app.kubernetes.io/managed-by: {{ .Release.Service }}
- app.kubernetes.io/instance: {{ .Release.Name }}
-rules:
- - apiGroups:
- - ""
- resources:
- - services
- - endpoints
- - secrets
- verbs:
- - get
- - list
- - watch
- - apiGroups:
- - extensions
- resources:
- - ingresses
- verbs:
- - get
- - list
- - watch
- - apiGroups:
- - extensions
- resources:
- - ingresses/status
- verbs:
- - update
- - apiGroups:
- - traefik.containo.us
- resources:
- - ingressroutes
- - ingressroutetcps
- - ingressrouteudps
- - middlewares
- - tlsoptions
- - tlsstores
- - traefikservices
- verbs:
- - get
- - list
- - watch
-{{- end -}}
diff --git a/helm-charts/submarine/charts/traefik/templates/rbac/service-account.yaml b/helm-charts/submarine/charts/traefik/templates/rbac/service-account.yaml
deleted file mode 100644
index 1df71159..00000000
--- a/helm-charts/submarine/charts/traefik/templates/rbac/service-account.yaml
+++ /dev/null
@@ -1,31 +0,0 @@
-{{- if not .Values.serviceAccount.name -}}
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-kind: ServiceAccount
-apiVersion: v1
-metadata:
- name: {{ include "traefik.serviceAccountName" . }}
- labels:
- app.kubernetes.io/name: {{ template "traefik.name" . }}
- helm.sh/chart: {{ template "traefik.chart" . }}
- app.kubernetes.io/managed-by: {{ .Release.Service }}
- app.kubernetes.io/instance: {{ .Release.Name }}
- annotations:
- {{- with .Values.serviceAccountAnnotations }}
- {{- toYaml . | nindent 4 }}
- {{- end }}
-{{- end -}}
diff --git a/helm-charts/submarine/charts/traefik/templates/service.yaml b/helm-charts/submarine/charts/traefik/templates/service.yaml
deleted file mode 100644
index 40972e45..00000000
--- a/helm-charts/submarine/charts/traefik/templates/service.yaml
+++ /dev/null
@@ -1,126 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-{{- if .Values.service.enabled -}}
-
-{{ $tcpPorts := dict }}
-{{ $udpPorts := dict }}
-{{- range $name, $config := .Values.ports }}
- {{- if eq (toString $config.protocol) "UDP" }}
- {{ $_ := set $udpPorts $name $config }}
- {{- else }}
- {{ $_ := set $tcpPorts $name $config }}
- {{- end }}
-{{- end }}
-
-apiVersion: v1
-kind: List
-items:
-{{- if $tcpPorts }}
- - apiVersion: v1
- kind: Service
- metadata:
- name: {{ template "traefik.fullname" . }}
- labels:
- app.kubernetes.io/name: {{ template "traefik.name" . }}
- helm.sh/chart: {{ template "traefik.chart" . }}
- app.kubernetes.io/managed-by: {{ .Release.Service }}
- app.kubernetes.io/instance: {{ .Release.Name }}
- annotations:
- {{- with .Values.service.annotations }}
- {{- toYaml . | nindent 8 }}
- {{- end }}
- spec:
- {{- $type := default "LoadBalancer" .Values.service.type }}
- type: {{ $type }}
- {{- with .Values.service.spec }}
- {{- toYaml . | nindent 6 }}
- {{- end }}
- selector:
- app.kubernetes.io/name: {{ template "traefik.name" . }}
- app.kubernetes.io/instance: {{ .Release.Name }}
- ports:
- {{- range $name, $config := $tcpPorts }}
- {{- if $config.expose }}
- - port: {{ default $config.port $config.exposedPort }}
- name: {{ $name }}
- targetPort: {{ $name | quote }}
- protocol: {{ default "TCP" $config.protocol | quote }}
- {{- if $config.nodePort }}
- nodePort: {{ $config.nodePort }}
- {{- end }}
- {{- end }}
- {{- end }}
- {{- if eq $type "LoadBalancer" }}
- {{- with .Values.service.loadBalancerSourceRanges }}
- loadBalancerSourceRanges:
- {{- toYaml . | nindent 6 }}
- {{- end -}}
- {{- end -}}
- {{- with .Values.service.externalIPs }}
- externalIPs:
- {{- toYaml . | nindent 6 }}
- {{- end -}}
-{{- end }}
-
-{{- if $udpPorts }}
- - apiVersion: v1
- kind: Service
- metadata:
- name: {{ template "traefik.fullname" . }}-udp
- labels:
- app.kubernetes.io/name: {{ template "traefik.name" . }}
- helm.sh/chart: {{ template "traefik.chart" . }}
- app.kubernetes.io/managed-by: {{ .Release.Service }}
- app.kubernetes.io/instance: {{ .Release.Name }}
- annotations:
- {{- with .Values.service.annotations }}
- {{- toYaml . | nindent 8 }}
- {{- end }}
- spec:
- {{- $type := default "LoadBalancer" .Values.service.type }}
- type: {{ $type }}
- {{- with .Values.service.spec }}
- {{- toYaml . | nindent 6 }}
- {{- end }}
- selector:
- app.kubernetes.io/name: {{ template "traefik.name" . }}
- app.kubernetes.io/instance: {{ .Release.Name }}
- ports:
- {{- range $name, $config := $udpPorts }}
- {{- if $config.expose }}
- - port: {{ default $config.port $config.exposedPort }}
- name: {{ $name }}
- targetPort: {{ $name | quote }}
- protocol: {{ default "UDP" $config.protocol | quote }}
- {{- if $config.nodePort }}
- nodePort: {{ $config.nodePort }}
- {{- end }}
- {{- end }}
- {{- end }}
- {{- if eq $type "LoadBalancer" }}
- {{- with .Values.service.loadBalancerSourceRanges }}
- loadBalancerSourceRanges:
- {{- toYaml . | nindent 6 }}
- {{- end -}}
- {{- end -}}
- {{- with .Values.service.externalIPs }}
- externalIPs:
- {{- toYaml . | nindent 6 }}
- {{- end -}}
-{{- end }}
-{{- end -}}
diff --git a/helm-charts/submarine/charts/traefik/values.yaml b/helm-charts/submarine/charts/traefik/values.yaml
deleted file mode 100644
index 17d164b1..00000000
--- a/helm-charts/submarine/charts/traefik/values.yaml
+++ /dev/null
@@ -1,336 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-
-# Default values for Traefik
-image:
- name: traefik
- tag: 2.2.8
- pullPolicy: IfNotPresent
-
-#
-# Configure the deployment
-#
-deployment:
- enabled: true
- # Number of pods of the deployment
- replicas: 1
- # Additional deployment annotations (e.g. for jaeger-operator sidecar injection)
- annotations: {}
- # Additional pod annotations (e.g. for mesh injection or prometheus scraping)
- podAnnotations: {}
- # Additional containers (e.g. for metric offloading sidecars)
- additionalContainers: []
- # Additional initContainers (e.g. for setting file permission as shown below)
- initContainers: []
- # The "volume-permissions" init container is required if you run into permission issues.
- # Related issue: https://github.com/containous/traefik/issues/6972
- # - name: volume-permissions
- # image: busybox:1.31.1
- # command: ["sh", "-c", "chmod -Rv 600 /data/*"]
- # volumeMounts:
- # - name: data
- # mountPath: /data
- # Custom pod DNS policy. Apply if `hostNetwork: true`
- # dnsPolicy: ClusterFirstWithHostNet
-
-# Pod disruption budget
-podDisruptionBudget:
- enabled: false
- # maxUnavailable: 1
- # minAvailable: 0
-
-# Create an IngressRoute for the dashboard
-ingressRoute:
- dashboard:
- enabled: false
- # Additional ingressRoute annotations (e.g. for kubernetes.io/ingress.class)
- annotations: {}
- # Additional ingressRoute labels (e.g. for filtering IngressRoute by custom labels)
- labels: {}
-
-rollingUpdate:
- maxUnavailable: 1
- maxSurge: 1
-
-
-#
-# Configure providers
-#
-providers:
- kubernetesCRD:
- enabled: true
- kubernetesIngress:
- enabled: true
- # IP used for Kubernetes Ingress endpoints
- publishedService:
- enabled: false
- # Published Kubernetes Service to copy status from. Format: namespace/servicename
- # By default this Traefik service
- # pathOverride: ""
-
-#
-# Add volumes to the traefik pod.
-# This can be used to mount a cert pair or a configmap that holds a config.toml file.
-# After the volume has been mounted, add the configs into traefik by using the `additionalArguments` list below, eg:
-# additionalArguments:
-# - "--providers.file.filename=/config/dynamic.toml"
-volumes: []
-# - name: public-cert
-# mountPath: "/certs"
-# type: secret
-# - name: configs
-# mountPath: "/config"
-# type: configMap
-
-# Logs
-# https://docs.traefik.io/observability/logs/
-logs:
- # Traefik logs concern everything that happens to Traefik itself (startup, configuration, events, shutdown, and so on).
- general:
- # By default, the logs use a text format (common), but you can
- # also ask for the json format in the format option
- # format: json
- # By default, the level is set to ERROR. Alternative logging levels are DEBUG, PANIC, FATAL, ERROR, WARN, and INFO.
- level: ERROR
- access:
- # To enable access logs
- enabled: false
- # By default, logs are written using the Common Log Format (CLF).
- # To write logs in JSON, use json in the format option.
- # If the given format is unsupported, the default (CLF) is used instead.
- # format: json
- # To write the logs in an asynchronous fashion, specify a bufferingSize option.
- # This option represents the number of log lines Traefik will keep in memory before writing
- # them to the selected output. In some cases, this option can greatly help performances.
- # bufferingSize: 100
- # Filtering https://docs.traefik.io/observability/access-logs/#filtering
- filters: {}
- # statuscodes: "200,300-302"
- # retryattempts: true
- # minduration: 10ms
- # Fields
- # https://docs.traefik.io/observability/access-logs/#limiting-the-fieldsincluding-headers
- fields:
- general:
- defaultmode: keep
- names: {}
- # Examples:
- # ClientUsername: drop
- headers:
- defaultmode: drop
- names: {}
- # Examples:
- # User-Agent: redact
- # Authorization: drop
- # Content-Type: keep
-
-globalArguments:
- - "--global.checknewversion"
- - "--global.sendanonymoususage"
-
-#
-# Configure Traefik static configuration
-# Additional arguments to be passed at Traefik's binary
-# All available options available on https://docs.traefik.io/reference/static-configuration/cli/
-## Use curly braces to pass values: `helm install --set="additionalArguments={--providers.kubernetesingress.ingressclass=traefik-internal,--log.level=DEBUG}"`
-additionalArguments: []
-# - "--providers.kubernetesingress.ingressclass=traefik-internal"
-# - "--log.level=DEBUG"
-
-# Environment variables to be passed to Traefik's binary
-env: []
-# - name: SOME_VAR
-# value: some-var-value
-# - name: SOME_VAR_FROM_CONFIG_MAP
-# valueFrom:
-# configMapRef:
-# name: configmap-name
-# key: config-key
-# - name: SOME_SECRET
-# valueFrom:
-# secretKeyRef:
-# name: secret-name
-# key: secret-key
-
-envFrom: []
-# - configMapRef:
-# name: config-map-name
-# - secretRef:
-# name: secret-name
-
-# Configure ports
-ports:
- # The name of this one can't be changed as it is used for the readiness and
- # liveness probes, but you can adjust its config to your liking
- traefik:
- port: 9000
- # Use hostPort if set.
- # hostPort: 9000
- #
- # Use hostIP if set. If not set, Kubernetes will default to 0.0.0.0, which
- # means it's listening on all your interfaces and all your IPs. You may want
- # to set this value if you need traefik to listen on specific interface
- # only.
- # hostIP: 192.168.100.10
-
- # Defines whether the port is exposed if service.type is LoadBalancer or
- # NodePort.
- #
- # You SHOULD NOT expose the traefik port on production deployments.
- # If you want to access it from outside of your cluster,
- # use `kubectl proxy` or create a secure ingress
- expose: false
- # The exposed port for this service
- exposedPort: 9000
- # The port protocol (TCP/UDP)
- protocol: TCP
- web:
- port: 8000
- # hostPort: 8000
- expose: true
- exposedPort: 80
- # The port protocol (TCP/UDP)
- protocol: TCP
- # Use nodeport if set. This is useful if you have configured Traefik in a
- # LoadBalancer
- nodePort: 32080
- # Port Redirections
- # Added in 2.2, you can make permanent redirects via entrypoints.
- # https://docs.traefik.io/routing/entrypoints/#redirection
- # redirectTo: websecure
- websecure:
- port: 8443
- # hostPort: 8443
- expose: true
- exposedPort: 443
- # The port protocol (TCP/UDP)
- protocol: TCP
- # nodePort: 32443
-
-# Options for the main traefik service, where the entrypoints traffic comes
-# from.
-service:
- enabled: true
- type: NodePort
- # Additional annotations (e.g. for cloud provider specific config)
- annotations: {}
- # Additional entries here will be added to the service spec. Cannot contains
- # type, selector or ports entries.
- spec: {}
- # externalTrafficPolicy: Cluster
- # loadBalancerIP: "1.2.3.4"
- # clusterIP: "2.3.4.5"
- loadBalancerSourceRanges: []
- # - 192.168.0.1/32
- # - 172.16.0.0/16
- externalIPs: []
- # - 1.2.3.4
-
-## Create HorizontalPodAutoscaler object.
-##
-autoscaling:
- enabled: false
-# minReplicas: 1
-# maxReplicas: 10
-# metrics:
-# - type: Resource
-# resource:
-# name: cpu
-# targetAverageUtilization: 60
-# - type: Resource
-# resource:
-# name: memory
-# targetAverageUtilization: 60
-
-# Enable persistence using Persistent Volume Claims
-# ref: http://kubernetes.io/docs/user-guide/persistent-volumes/
-# After the pvc has been mounted, add the configs into traefik by using the `additionalArguments` list below, eg:
-# additionalArguments:
-# - "--certificatesresolvers.le.acme.storage=/data/acme.json"
-# It will persist TLS certificates.
-persistence:
- enabled: false
-# existingClaim: ""
- accessMode: ReadWriteOnce
- size: 128Mi
- # storageClass: ""
- path: /data
- annotations: {}
- # subPath: "" # only mount a subpath of the Volume into the pod
-
-# If hostNetwork is true, runs traefik in the host network namespace
-# To prevent unschedulabel pods due to port collisions, if hostNetwork=true
-# and replicas>1, a pod anti-affinity is recommended and will be set if the
-# affinity is left as default.
-hostNetwork: false
-
-# Whether Role Based Access Control objects like roles and rolebindings should be created
-rbac:
- enabled: true
-
- # If set to false, installs ClusterRole and ClusterRoleBinding so Traefik can be used across namespaces.
- # If set to true, installs namespace-specific Role and RoleBinding and requires provider configuration be set to that same namespace
- namespaced: false
-
-# The service account the pods will use to interact with the Kubernetes API
-serviceAccount:
- # If set, an existing service account is used
- # If not set, a service account is created automatically using the fullname template
- name: ""
-
-# Additional serviceAccount annotations (e.g. for oidc authentication)
-serviceAccountAnnotations: {}
-
-resources: {}
- # requests:
- # cpu: "100m"
- # memory: "50Mi"
- # limits:
- # cpu: "300m"
- # memory: "150Mi"
-affinity: {}
-# # This example pod anti-affinity forces the scheduler to put traefik pods
-# # on nodes where no other traefik pods are scheduled.
-# # It should be used when hostNetwork: true to prevent port conflicts
-# podAntiAffinity:
-# requiredDuringSchedulingIgnoredDuringExecution:
-# - labelSelector:
-# matchExpressions:
-# - key: app
-# operator: In
-# values:
-# - {{ template "traefik.name" . }}
-# topologyKey: failure-domain.beta.kubernetes.io/zone
-nodeSelector: {}
-tolerations: []
-
-# Pods can have priority.
-# Priority indicates the importance of a Pod relative to other Pods.
-priorityClassName: ""
-
-# Set the container security context
-# To run the container with ports below 1024 this will need to be adjust to run as root
-securityContext:
- capabilities:
- drop: [ALL]
- readOnlyRootFilesystem: true
- runAsGroup: 65532
- runAsNonRoot: true
- runAsUser: 65532
-
-podSecurityContext:
- fsGroup: 65532
diff --git a/helm-charts/submarine/charts/traefik/crds/ingressroute.yaml b/helm-charts/submarine/templates/istio-gateway.yaml
similarity index 73%
rename from helm-charts/submarine/charts/traefik/crds/ingressroute.yaml
rename to helm-charts/submarine/templates/istio-gateway.yaml
index d42e4576..e78ea6e4 100644
--- a/helm-charts/submarine/charts/traefik/crds/ingressroute.yaml
+++ b/helm-charts/submarine/templates/istio-gateway.yaml
@@ -15,15 +15,19 @@
# limitations under the License.
#
-apiVersion: apiextensions.k8s.io/v1beta1
-kind: CustomResourceDefinition
+{{ if .Values.istio.enabled }}
+apiVersion: networking.istio.io/v1alpha3
+kind: Gateway
metadata:
- name: ingressroutes.traefik.containo.us
+ name: submarine-gateway
spec:
- group: traefik.containo.us
- version: v1alpha1
- names:
- kind: IngressRoute
- plural: ingressroutes
- singular: ingressroute
- scope: Namespaced
+ selector:
+ istio: {{ .Values.istio.gatewaySelector }}
+ servers:
+ - port:
+ number: 80
+ name: http
+ protocol: HTTP
+ hosts:
+ - "*"
+{{ end }}
diff --git a/helm-charts/submarine/templates/rbac.yaml b/helm-charts/submarine/templates/rbac.yaml
index 2c854efb..9bba6547 100644
--- a/helm-charts/submarine/templates/rbac.yaml
+++ b/helm-charts/submarine/templates/rbac.yaml
@@ -40,6 +40,12 @@ rules:
- traefikservices
verbs:
- "*"
+ - apiGroups:
+ - networking.istio.io
+ resources:
+ - virtualservices
+ verbs:
+ - "*"
- apiGroups:
- kubeflow.org
resources:
diff --git a/helm-charts/submarine/values.yaml b/helm-charts/submarine/values.yaml
index 55956485..56c02090 100644
--- a/helm-charts/submarine/values.yaml
+++ b/helm-charts/submarine/values.yaml
@@ -29,7 +29,7 @@ storageClass:
# provisioner is to determine what volume plugin is used for provisioning PVs
provisioner: k8s.io/minikube-hostpath
# parameters describe volumes belonging to the storage class
- parameters:
+ parameters: {}
# k8s cluster type. can be: kubernetes or openshift
clusterType: kubernetes
@@ -40,3 +40,8 @@ podSecurityPolicy:
# Specifies whether a PodSecurityPolicy should be created,
# This configuration enables the database/minio/server to set securityContext.runAsUser
create: true
+
+# Istio configuration
+istio:
+ enabled: true
+ gatewaySelector: ingressgateway
diff --git a/submarine-cloud-v2/README.md b/submarine-cloud-v2/README.md
index 4e4ff176..0b29ce36 100644
--- a/submarine-cloud-v2/README.md
+++ b/submarine-cloud-v2/README.md
@@ -19,107 +19,6 @@
`submarine-cloud-v2`, i.e. **submarine-operator**, implements the operator for Submarine application. The operator provides a new option for users to deploy the Submarine service to their Kubernetes clusters. The **submarine-operator** can fix some errors automatically. However, if the Submarine service is deployed with Helm, the errors need to be fixed by human operators.
-# Getting Started
-
-- In this section, we provide two methods, including **out-of-cluter** method and **in-cluster** method, for you to deploy your **submarine-operator**. In addition, the out-of-cluster method is convenient for operator developers. On the other hand, the in-cluster method is suitable for production.
-
-Golang version: `1.16.2`
-
-## Initialization
-
-```bash
-# Install dependencies
-go mod vendor
-# Run the cluster
-minikube start --vm-driver=docker --cpus 8 --memory 4096 --kubernetes-version v1.21.2
-# Or if you want to support Pod Security Policy (https://minikube.sigs.k8s.io/docs/tutorials/using_psp), you can use the following command to start cluster
-minikube start --extra-config=apiserver.enable-admission-plugins=PodSecurityPolicy --addons=pod-security-policy --vm-driver=docker --cpus 8 --memory 4096 --kubernetes-version v1.21.2
-```
-
-## Set up storage class fields
-
-One can set up storage class fields in `values.yaml` or using helm with `--set`. We've set up minikube's provisioner for storage class as default.
-
-For example, if you are using kind in local, please add `--set storageClass.provisioner=rancher.io/local-path --set storageClass.volumeBindingMode=WaitForFirstConsumer` to helm install command.
-
-Documentation for storage class: https://kubernetes.io/docs/concepts/storage/storage-classes/
-
-## Run operator out-of-cluster
-
-```bash
-# Step1: Install helm chart dependencies
-helm install --set dev=true submarine ../helm-charts/submarine/
-
-# Step2: Build & Run "submarine-operator"
-make
-./submarine-operator
-
-# Step3: Deploy a Submarine
-kubectl create ns submarine-user-test
-kubectl apply -n submarine-user-test -f artifacts/examples/example-submarine.yaml
-
-# Step4: Exposing Service
-# Method1 -- use minikube ip
-minikube ip # you'll get the IP address of minikube, ex: 192.168.49.2
-
-# Method2 -- use port-forwarding
-kubectl port-forward --address 0.0.0.0 service/submarine-traefik 32080:80
-
-# Step5: View Workbench
-# http://{minikube ip}:32080 (from Method 1), ex: http://192.168.49.2:32080
-# or http://127.0.0.1:32080 (from Method 2).
-
-# Step6: Delete Submarine
-# By deleting the submarine custom resource, the operator will do the following things:
-# (1) Remove all relevant Helm chart releases
-# (2) Remove all resources in the namespace "submariner-user-test"
-# (3) Remove all non-namespaced resources (Ex: PersistentVolume) created by client-go API
-# (4) **Note:** The namespace "submarine-user-test" will not be deleted
-kubectl delete submarine example-submarine -n submarine-user-test
-
-# Step6: Stop the operator
-# Press ctrl+c to stop the operator
-
-# Step7: Uninstall helm chart dependencies
-helm delete submarine
-```
-
-## Run operator in-cluster
-
-```bash
-# Step1: Install submarine-operator
-helm install submarine ../helm-charts/submarine/
-
-# Step2: Deploy a submarine
-kubectl create ns submarine-user-test
-kubectl apply -n submarine-user-test -f artifacts/examples/example-submarine.yaml
-
-# Step3: Inspect the logs of submarine-operator
-kubectl logs -f $(kubectl get pods --output=name | grep submarine-operator)
-
-# Step4: Exposing Service
-# Method1 -- use minikube ip
-minikube ip # you'll get the IP address of minikube, ex: 192.168.49.2
-
-# Method2 -- use port-forwarding
-kubectl port-forward --address 0.0.0.0 service/submarine-traefik 32080:80
-
-# Step5: View Workbench
-# http://{minikube ip}:32080 (from Method 1), ex: http://192.168.49.2:32080
-# or http://127.0.0.1:32080 (from Method 2).
-
-# Step6: Delete Submarine
-# By deleting the submarine custom resource, the operator will do the following things:
-# (1) Remove all relevant Helm chart releases
-# (2) Remove all resources in the namespace "submariner-user-test"
-# (3) Remove all non-namespaced resources (Ex: PersistentVolume) created by client-go API
-# (4) **Note:** The namespace "submarine-user-test" will not be deleted
-kubectl delete submarine example-submarine -n submarine-user-test
-
-# Step7: Delete the submarine-operator
-helm delete submarine
-```
-
# Development
Please check out the [Developer Guide](./docs/developer-guide.md).
diff --git a/submarine-cloud-v2/artifacts/submarine/submarine-ingress.yaml b/submarine-cloud-v2/artifacts/submarine/submarine-ingress.yaml
index 0f6b5830..73f4c842 100644
--- a/submarine-cloud-v2/artifacts/submarine/submarine-ingress.yaml
+++ b/submarine-cloud-v2/artifacts/submarine/submarine-ingress.yaml
@@ -1,15 +1,56 @@
----
-# Source: submarine/templates/submarine-ingress.yaml
-apiVersion: extensions/v1beta1
-kind: Ingress
-metadata:
- name: submarine-server-ingress
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+apiVersion: networking.istio.io/v1alpha3
+kind: VirtualService
+metadata:
+ name: submarine-virtual-service
spec:
- rules:
- - http:
- paths:
- - path: /
- backend:
- serviceName: submarine-server
- servicePort: 8080
+ hosts:
+ - "*"
+ gateways:
+ - submarine/submarine-gateway
+ http:
+ - match:
+ - uri:
+ prefix: /tensorboard
+ route:
+ - destination:
+ host: submarine-tensorboard-service
+ port:
+ number: 8080
+ - match:
+ - uri:
+ prefix: /mlflow
+ route:
+ - destination:
+ host: submarine-mlflow-service
+ port:
+ number: 5000
+ - match:
+ - uri:
+ prefix: /minio
+ route:
+ - destination:
+ host: submarine-minio-service
+ port:
+ number: 9000
+ - route:
+ - destination:
+ host: submarine-server
+ port:
+ number: 8080
diff --git a/submarine-cloud-v2/artifacts/submarine/submarine-minio.yaml b/submarine-cloud-v2/artifacts/submarine/submarine-minio.yaml
index 799fc096..a9b006bc 100644
--- a/submarine-cloud-v2/artifacts/submarine/submarine-minio.yaml
+++ b/submarine-cloud-v2/artifacts/submarine/submarine-minio.yaml
@@ -39,9 +39,10 @@ spec:
selector:
app: submarine-minio
ports:
- - protocol: TCP
- port: 9000
- targetPort: 9000
+ - protocol: TCP
+ port: 9000
+ targetPort: 9000
+ name: http
---
# Source: submarine/templates/submarine-minio.yaml
apiVersion: apps/v1
@@ -59,40 +60,24 @@ spec:
spec:
serviceAccountName: "submarine-storage"
containers:
- - name: submarine-minio-container
- image: minio/minio:RELEASE.2021-02-14T04-01-33Z
- imagePullPolicy: IfNotPresent
- args:
- - server
- - /data
- env:
- - name: MINIO_ACCESS_KEY
- value: "submarine_minio"
- - name: MINIO_SECRET_KEY
- value: "submarine_minio"
- ports:
- - containerPort: 9000
- volumeMounts:
- - mountPath: "/data"
- name: "volume"
- subPath: "submarine-minio"
+ - name: submarine-minio-container
+ image: minio/minio:RELEASE.2021-02-14T04-01-33Z
+ imagePullPolicy: IfNotPresent
+ args:
+ - server
+ - /data
+ env:
+ - name: MINIO_ACCESS_KEY
+ value: "submarine_minio"
+ - name: MINIO_SECRET_KEY
+ value: "submarine_minio"
+ ports:
+ - containerPort: 9000
+ volumeMounts:
+ - mountPath: "/data"
+ name: "volume"
+ subPath: "submarine-minio"
volumes:
- name: "volume"
persistentVolumeClaim:
claimName: "submarine-minio-pvc"
----
-# Source: submarine/templates/submarine-minio.yaml
-apiVersion: traefik.containo.us/v1alpha1
-kind: IngressRoute
-metadata:
- name: submarine-minio-ingressroute
-spec:
- entryPoints:
- - web
- routes:
- - kind: Rule
- match: "PathPrefix(`/minio`)"
- services:
- - kind: Service
- name: submarine-minio-service
- port: 9000
diff --git a/submarine-cloud-v2/artifacts/submarine/submarine-mlflow.yaml b/submarine-cloud-v2/artifacts/submarine/submarine-mlflow.yaml
index 66add3a5..c9ab92cc 100644
--- a/submarine-cloud-v2/artifacts/submarine/submarine-mlflow.yaml
+++ b/submarine-cloud-v2/artifacts/submarine/submarine-mlflow.yaml
@@ -43,6 +43,7 @@ spec:
- protocol: TCP
port: 5000
targetPort: 5000
+ name: http
---
# Source: submarine/templates/submarine-mlflow.yaml
apiVersion: apps/v1
@@ -108,20 +109,4 @@ spec:
persistentVolumeClaim:
claimName: "submarine-mlflow-pvc"
- name: mc-config-vol
- emptyDir: { }
----
-# Source: submarine/templates/submarine-mlflow.yaml
-apiVersion: traefik.containo.us/v1alpha1
-kind: IngressRoute
-metadata:
- name: submarine-mlflow-ingressroute
-spec:
- entryPoints:
- - web
- routes:
- - kind: Rule
- match: "PathPrefix(`/mlflow`)"
- services:
- - kind: Service
- name: submarine-mlflow-service
- port: 5000
+ emptyDir: {}
\ No newline at end of file
diff --git a/submarine-cloud-v2/artifacts/submarine/submarine-server.yaml b/submarine-cloud-v2/artifacts/submarine/submarine-server.yaml
index f01ce03f..50cb0137 100644
--- a/submarine-cloud-v2/artifacts/submarine/submarine-server.yaml
+++ b/submarine-cloud-v2/artifacts/submarine/submarine-server.yaml
@@ -40,6 +40,7 @@ spec:
- port: 8080
targetPort: 8080
protocol: TCP
+ name: http
selector:
app: "submarine-server"
---
diff --git a/submarine-cloud-v2/artifacts/submarine/submarine-tensorboard.yaml b/submarine-cloud-v2/artifacts/submarine/submarine-tensorboard.yaml
index b49776b5..486c4038 100644
--- a/submarine-cloud-v2/artifacts/submarine/submarine-tensorboard.yaml
+++ b/submarine-cloud-v2/artifacts/submarine/submarine-tensorboard.yaml
@@ -38,9 +38,10 @@ spec:
selector:
app: submarine-tensorboard
ports:
- - protocol: TCP
- port: 8080
- targetPort: 6006
+ - protocol: TCP
+ port: 8080
+ targetPort: 6006
+ name: http
---
# Source: submarine/templates/submarine-tensorboard.yaml
apiVersion: apps/v1
@@ -58,40 +59,24 @@ spec:
spec:
serviceAccountName: "submarine-storage"
containers:
- - name: submarine-tensorboard-container
- image: tensorflow/tensorflow:1.11.0
- command:
- - "tensorboard"
- - "--logdir=/logs"
- - "--path_prefix=/tensorboard"
- imagePullPolicy: IfNotPresent
- ports:
- - containerPort: 6006
- volumeMounts:
- - mountPath: "/logs"
- name: "volume"
- subPath: "submarine-tensorboard"
- readinessProbe:
- tcpSocket:
- port: 6006
- periodSeconds: 10
+ - name: submarine-tensorboard-container
+ image: tensorflow/tensorflow:1.11.0
+ command:
+ - "tensorboard"
+ - "--logdir=/logs"
+ - "--path_prefix=/tensorboard"
+ imagePullPolicy: IfNotPresent
+ ports:
+ - containerPort: 6006
+ volumeMounts:
+ - mountPath: "/logs"
+ name: "volume"
+ subPath: "submarine-tensorboard"
+ readinessProbe:
+ tcpSocket:
+ port: 6006
+ periodSeconds: 10
volumes:
- name: "volume"
persistentVolumeClaim:
claimName: "submarine-tensorboard-pvc"
----
-# Source: submarine/templates/submarine-tensorboard.yaml
-apiVersion: traefik.containo.us/v1alpha1
-kind: IngressRoute
-metadata:
- name: submarine-tensorboard-ingressroute
-spec:
- entryPoints:
- - web
- routes:
- - kind: Rule
- match: "PathPrefix(`/tensorboard`)"
- services:
- - kind: Service
- name: submarine-tensorboard-service
- port: 8080
diff --git a/submarine-cloud-v2/docs/developer-guide.md b/submarine-cloud-v2/docs/developer-guide.md
index 93677715..2e0b3212 100644
--- a/submarine-cloud-v2/docs/developer-guide.md
+++ b/submarine-cloud-v2/docs/developer-guide.md
@@ -17,6 +17,43 @@
# Developer Guide
+Golang version: `1.17`
+
+## Prerequisites
+
+First finish the prerequisites specified in the [QuickStart](quickstart) section on the submarine website.
+
+Next, install golang dependencies.
+
+```bash
+go mod vendor
+```
+
+## Run operator in-cluster
+
+If you follow the [QuickStart][quickstart] section on the submarine website, you are running operator in-cluster.
+
+## Run operator out-of-cluster
+
+Running operator out-of-cluster is very handy for development
+
+```bash
+# Step1: Install helm chart dependencies, --set dev=true option will not install the operator deployment to the cluster
+helm install --set dev=true submarine ../helm-charts/submarine/ -n submarine
+
+# Step2: Build & Run "submarine-operator"
+make
+./submarine-operator
+
+# Step3: Deploy a Submarine
+kubectl create ns submarine-user-test
+kubectl apply -n submarine-user-test -f artifacts/examples/example-submarine.yaml
+```
+
+If you follow the above steps, you can view the submarine workbench via the same approach specified in the [QuickStart][quickstart] section on the submarine website.
+
+Whenever you change the operator code, simply shutdown the operator and recompile the operator using `make` and re-run again.
+
## Coding Style
For `go` files, please use [gofmt](https://golang.org/pkg/cmd/gofmt/) to format the code.
@@ -31,6 +68,14 @@ We use [code-generator](https://github.com/kubernetes/code-generator) to generat
Everytime when you change the codes in `submarine-cloud-v2/pkg/apis`, you must run `make api` to re-generate the API.
+## Set up storage class fields
+
+One can set up storage class fields in `values.yaml` or using helm with `--set`. We've set up minikube's provisioner for storage class as default.
+
+For example, if you are using kind in local, please add `--set storageClass.provisioner=rancher.io/local-path --set storageClass.volumeBindingMode=WaitForFirstConsumer` to helm install command.
+
+Documentation for storage class: https://kubernetes.io/docs/concepts/storage/storage-classes/
+
## Create New Custom Resources
To modify the parameters of submarine resource, go to the location `submarine/submarine-cloud-v2/artifacts` to edit specific `yaml` files. In this case, you won't need to modify the code for operator.
@@ -110,3 +155,5 @@ Examples:
./hack/build_image.sh all # build all images
./hack/build_image.sh server # only build the server image
```
+
+[quickstart]: https://submarine.apache.org/docs/next/gettingStarted/quickstart
diff --git a/submarine-cloud-v2/go.mod b/submarine-cloud-v2/go.mod
index 7b2e070b..e187ec3d 100644
--- a/submarine-cloud-v2/go.mod
+++ b/submarine-cloud-v2/go.mod
@@ -7,12 +7,13 @@ require (
github.com/pkg/errors v0.9.1
github.com/stretchr/testify v1.7.0
github.com/traefik/traefik/v2 v2.4.8
- golang.org/x/sync v0.0.0-20201207232520-09787c993a3a
- k8s.io/api v0.20.4
- k8s.io/apimachinery v0.20.4
- k8s.io/client-go v0.20.4
+ golang.org/x/sync v0.0.0-20210220032951-036812b2e83c
+ istio.io/client-go v1.13.0
+ k8s.io/api v0.23.1
+ k8s.io/apimachinery v0.23.1
+ k8s.io/client-go v0.23.1
k8s.io/code-generator v0.21.2
- k8s.io/klog/v2 v2.8.0
+ k8s.io/klog/v2 v2.30.0
)
replace (
diff --git a/submarine-cloud-v2/go.sum b/submarine-cloud-v2/go.sum
index da9889e0..fe48cb50 100644
--- a/submarine-cloud-v2/go.sum
+++ b/submarine-cloud-v2/go.sum
@@ -9,18 +9,33 @@ cloud.google.com/go v0.50.0/go.mod h1:r9sluTvynVuxRIOHXQEHMFffphuXHOMZMycpNR5e6T
cloud.google.com/go v0.52.0/go.mod h1:pXajvRH/6o3+F9jDHZWQ5PbGhn+o8w9qiu/CffaVdO4=
cloud.google.com/go v0.53.0/go.mod h1:fp/UouUEsRkN6ryDKNW/Upv/JBKnv6WDthjR6+vze6M=
cloud.google.com/go v0.54.0/go.mod h1:1rq2OEkV3YMf6n/9ZvGWI3GWw0VoqH/1x2nd8Is/bPc=
+cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKVk=
+cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs=
+cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc=
+cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY=
+cloud.google.com/go v0.72.0/go.mod h1:M+5Vjvlc2wnp6tjzE102Dw08nGShTscUx2nZMufOKPI=
+cloud.google.com/go v0.74.0/go.mod h1:VV1xSbzvo+9QJOxLDaJfTjx5e+MePCpCWwvftOeQmWk=
+cloud.google.com/go v0.78.0/go.mod h1:QjdrLG0uq+YwhjoVOLsS1t7TW8fs36kLs4XO5R5ECHg=
+cloud.google.com/go v0.79.0/go.mod h1:3bzgcEeQlzbuEAYu4mrWhKqWjmpprinYgKJLgKHnbb8=
+cloud.google.com/go v0.81.0/go.mod h1:mk/AM35KwGk/Nm2YSeZbxXdrNK3KZOYHmLkOqC2V6E0=
cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=
cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc=
+cloud.google.com/go/bigquery v1.5.0/go.mod h1:snEHRnqQbz117VIFhE8bmtwIDY80NLUZUMb4Nv6dBIg=
+cloud.google.com/go/bigquery v1.7.0/go.mod h1://okPTzCYNXSlb24MZs83e2Do+h+VXtc4gLoIoXIAPc=
+cloud.google.com/go/bigquery v1.8.0/go.mod h1:J5hqkt3O0uAFnINi6JXValWIb1v0goeZM77hZzJN/fQ=
cloud.google.com/go/datastore v1.0.0/go.mod h1:LXYbyblFSglQ5pkeyhO+Qmw7ukd3C+pD7TKLgZqpHYE=
cloud.google.com/go/datastore v1.1.0/go.mod h1:umbIZjpQpHh4hmRpGhH4tLFup+FVzqBi1b3c64qFpCk=
cloud.google.com/go/firestore v1.1.0/go.mod h1:ulACoGHTpvq5r8rxGJ4ddJZBZqakUQqClKRT5SZwBmk=
cloud.google.com/go/pubsub v1.0.1/go.mod h1:R0Gpsv3s54REJCy4fxDixWD93lHJMoZTyQ2kNxGRt3I=
cloud.google.com/go/pubsub v1.1.0/go.mod h1:EwwdRX2sKPjnvnqCa270oGRyludottCI76h+R3AArQw=
cloud.google.com/go/pubsub v1.2.0/go.mod h1:jhfEVHT8odbXTkndysNHCcx0awwzvfOlguIAii9o8iA=
+cloud.google.com/go/pubsub v1.3.1/go.mod h1:i+ucay31+CNRpDW4Lu78I4xXG+O1r/MAHgjpRVR+TSU=
cloud.google.com/go/storage v1.0.0/go.mod h1:IhtSnM/ZTZV8YYJWCY8RULGVqBDmpoyjwiyrjsg+URw=
cloud.google.com/go/storage v1.5.0/go.mod h1:tpKbwo567HUNpVclU5sGELwQWBDZ8gh0ZeosJ0Rtdos=
cloud.google.com/go/storage v1.6.0/go.mod h1:N7U0C8pVQ/+NIKOBQyamJIeKQKkZ+mxpohlUTyfDhBk=
+cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs=
+cloud.google.com/go/storage v1.10.0/go.mod h1:FLPqc6j+Ki4BU591ie1oL6qBQGu2Bl/tZ9ullr3+Kg0=
contrib.go.opencensus.io/exporter/ocagent v0.4.12/go.mod h1:450APlNTSR6FrvC3CTRqYosuDstRB9un7SOx2k/9ckA=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
github.com/Azure/azure-sdk-for-go v16.2.1+incompatible/go.mod h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
@@ -31,10 +46,12 @@ github.com/Azure/go-autorest v14.2.0+incompatible/go.mod h1:r+4oMnoxhatjLLJ6zxSW
github.com/Azure/go-autorest/autorest v0.1.0/go.mod h1:AKyIcETwSUFxIcs/Wnq/C+kwCtlEYGUVd7FPNb2slmg=
github.com/Azure/go-autorest/autorest v0.9.0/go.mod h1:xyHB1BMZT0cuDHU7I0+g046+BFDTQ8rEZB0s4Yfa6bI=
github.com/Azure/go-autorest/autorest v0.11.1/go.mod h1:JFgpikqFJ/MleTTxwepExTKnFUKKszPS8UavbQYUMuw=
+github.com/Azure/go-autorest/autorest v0.11.18/go.mod h1:dSiJPy22c3u0OtOKDNttNgqpNFY/GeWa7GH/Pz56QRA=
github.com/Azure/go-autorest/autorest/adal v0.1.0/go.mod h1:MeS4XhScH55IST095THyTxElntu7WqB7pNbZo8Q5G3E=
github.com/Azure/go-autorest/autorest/adal v0.5.0/go.mod h1:8Z9fGy2MpX0PvDjB1pEgQTmVqjGhiHBW7RJJEciWzS0=
github.com/Azure/go-autorest/autorest/adal v0.9.0/go.mod h1:/c022QCutn2P7uY+/oQWWNcK9YU+MH96NgK+jErpbcg=
github.com/Azure/go-autorest/autorest/adal v0.9.5/go.mod h1:B7KF7jKIeC9Mct5spmyCB/A8CG/sEz1vwIRGv/bbw7A=
+github.com/Azure/go-autorest/autorest/adal v0.9.13/go.mod h1:W/MM4U6nLxnIskrw4UwWzlHfGjwUS50aOsc/I3yuU8M=
github.com/Azure/go-autorest/autorest/azure/auth v0.1.0/go.mod h1:Gf7/i2FUpyb/sGBLIFxTBzrNzBo7aPXXE3ZVeDRwdpM=
github.com/Azure/go-autorest/autorest/azure/cli v0.1.0/go.mod h1:Dk8CUAt/b/PzkfeRsWzVG9Yj3ps8mS8ECztu43rdU8U=
github.com/Azure/go-autorest/autorest/date v0.1.0/go.mod h1:plvfp3oPSKwf2DNjlBjWF/7vwR+cUD/ELuzDCXwHUVA=
@@ -47,6 +64,7 @@ github.com/Azure/go-autorest/autorest/to v0.2.0/go.mod h1:GunWKJp1AEqgMaGLV+iocm
github.com/Azure/go-autorest/autorest/validation v0.1.0/go.mod h1:Ha3z/SqBeaalWQvokg3NZAlQTalVMtOIAs1aGK7G6u8=
github.com/Azure/go-autorest/logger v0.1.0/go.mod h1:oExouG+K6PryycPJfVSxi/koC6LSNgds39diKLz7Vrc=
github.com/Azure/go-autorest/logger v0.2.0/go.mod h1:T9E3cAhj2VqvPOtCYAvby9aBXkZmbF5NWuPV8+WeEW8=
+github.com/Azure/go-autorest/logger v0.2.1/go.mod h1:T9E3cAhj2VqvPOtCYAvby9aBXkZmbF5NWuPV8+WeEW8=
github.com/Azure/go-autorest/tracing v0.1.0/go.mod h1:ROEEAFwXycQw7Sn3DXNtEedEvdeRAgDr0izn4z5Ij88=
github.com/Azure/go-autorest/tracing v0.5.0/go.mod h1:r/s2XiOKccPW3HrqB+W0TQzfbtp2fGCgRFtBroKn4Dk=
github.com/Azure/go-autorest/tracing v0.6.0/go.mod h1:+vhtPC754Xsa23ID7GlGsrdKBpUA79WCAKPPZVC2DeU=
@@ -91,6 +109,7 @@ github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRF
github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/aliyun/alibaba-cloud-sdk-go v1.61.976/go.mod h1:pUKYbK5JQ+1Dfxk80P0qxGqe5dkxDoabbZS7zOcouyA=
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
+github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY=
github.com/apache/thrift v0.12.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ=
github.com/apache/thrift v0.13.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ=
github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o=
@@ -142,6 +161,13 @@ github.com/circonus-labs/circonusllhist v0.1.3/go.mod h1:kMXHVDlOchFAehlya5ePtbp
github.com/clbanning/x2j v0.0.0-20191024224557-825249438eec/go.mod h1:jMjuTZXRI4dUb/I5gc9Hdhagfvm9+RyrPryS/auMzxE=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cloudflare/cloudflare-go v0.14.0/go.mod h1:EnwdgGMaFOruiPZRFSgn+TsQ3hQ7C/YWzIGLeu5c304=
+github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
+github.com/cncf/udpa/go v0.0.0-20200629203442-efcf912fb354/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
+github.com/cncf/udpa/go v0.0.0-20201120205902-5459f2c99403/go.mod h1:WmhPx2Nbnhtbo57+VJT5O0JRkEi1Wbu0z5j0R8u5Hbk=
+github.com/cncf/udpa/go v0.0.0-20210930031921-04548b0d99d4/go.mod h1:6pvJx4me5XPnfI9Z40ddWsdw2W/uZgQLFXToKeRcDiI=
+github.com/cncf/xds/go v0.0.0-20210805033703-aa0b78936158/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
+github.com/cncf/xds/go v0.0.0-20210922020428-25de7278fc84/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
+github.com/cncf/xds/go v0.0.0-20211011173535-cb28da3451f1/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa/go.mod h1:zn76sxSg3SzpJ0PPJaLDCu+Bu0Lg3sKTORVIj19EIF8=
github.com/codahale/hdrhistogram v0.0.0-20161010025455-3a0bb77429bd/go.mod h1:sE/e/2PUdi/liOCUjSTXgM1o87ZssimdTWN964YiIeI=
github.com/codegangsta/negroni v1.0.0/go.mod h1:v0y3T5G7Y1UlFfyxFn/QLRU4a2EuNau2iZY63YTKWo0=
@@ -224,18 +250,25 @@ github.com/emicklei/go-restful v0.0.0-20170410110728-ff4f55a20633/go.mod h1:otzb
github.com/emicklei/go-restful v2.9.5+incompatible h1:spTtZBk5DYEvbxMVutUuTyh1Ao2r4iyvLdACqsl/Ljk=
github.com/emicklei/go-restful v2.9.5+incompatible/go.mod h1:otzb+WCGbkyDHkqmQmT5YD2WR4BBwUdeQoFo8l/7tVs=
github.com/envoyproxy/go-control-plane v0.6.9/go.mod h1:SBwIajubJHhxtWwsL9s8ss4safvEdbitLhGGK48rN6g=
+github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
+github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
+github.com/envoyproxy/go-control-plane v0.9.7/go.mod h1:cwu0lG7PUMfa9snN8LXBig5ynNVH9qI8YYLbd1fK2po=
+github.com/envoyproxy/go-control-plane v0.9.9-0.20201210154907-fd9021fe5dad/go.mod h1:cXg6YxExXjJnVBQHBLXeUAgxn2UodCpnH306RInaBQk=
+github.com/envoyproxy/go-control-plane v0.9.10-0.20210907150352-cf90f659a021/go.mod h1:AFq3mo9L8Lqqiid3OhADV3RfLJnjiw63cSpi+fDTRC0=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/evanphx/json-patch v4.2.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
github.com/evanphx/json-patch v4.5.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
-github.com/evanphx/json-patch v4.9.0+incompatible h1:kLcOMZeuLAJvL2BPWLMIj5oaZQobrkAqrL+WFZwQses=
github.com/evanphx/json-patch v4.9.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
+github.com/evanphx/json-patch v4.12.0+incompatible h1:4onqiflcdA9EOZ4RxV643DvftH5pOlLGNtQ5lPWQu84=
+github.com/evanphx/json-patch v4.12.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
github.com/exoscale/egoscale v0.46.0/go.mod h1:mpEXBpROAa/2i5GC0r33rfxG+TxSEka11g1PIXt9+zc=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
github.com/fatih/structs v1.1.0/go.mod h1:9NiDSp5zOcgEDl+j00MP/WkGVPOlPRLejGD8Ga6PJ7M=
github.com/felixge/httpsnoop v1.0.0/go.mod h1:3+D9sFq0ahK/JeJPhCBUV1xlf4/eIYrUQaxulT0VzX8=
github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568/go.mod h1:xEzjJPgXI435gkrCt3MPfRiAkVrwSbHsst4LCFVfpJc=
github.com/form3tech-oss/jwt-go v3.2.2+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k=
+github.com/form3tech-oss/jwt-go v3.2.3+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k=
github.com/franela/goblin v0.0.0-20200105215937-c9ffbefa60db/go.mod h1:7dvUGVsVBjqR7JHJk0brhHOZYGmfBYOrK0ZhYMEtBr4=
github.com/franela/goreq v0.0.0-20171204163338-bcd34c9993f8/go.mod h1:ZhphrRTfi2rbfLwlschooIH4+wKKDR4Pdxhh+TRoA20=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
@@ -244,6 +277,7 @@ github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4
github.com/gambol99/go-marathon v0.0.0-20180614232016-99a156b96fb2/go.mod h1:GLyXJD41gBO/NPKVPGQbhyyC06eugGy15QEZyUkE2/s=
github.com/garyburd/redigo v0.0.0-20150301180006-535138d7bcd7/go.mod h1:NR3MbYisc3/PwhQ00EMzDiPmrwpPxAn5GI05/YaO1SY=
github.com/getkin/kin-openapi v0.13.0/go.mod h1:WGRs2ZMM1Q8LR1QBEwUxC6RJEfaBcD0s+pcEVXFuAjw=
+github.com/getkin/kin-openapi v0.76.0/go.mod h1:660oXbgy5JFMKreazJaQTw7o+X00qeSyhcnluiMv+Xg=
github.com/ghodss/yaml v0.0.0-20150909031657-73d445a93680/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/globalsign/mgo v0.0.0-20180905125535-1ca0a4f7cbcb/go.mod h1:xkRDCp4j0OGD1HRkm4kmhM+pmpv3AKq5SU7GMg4oO/Q=
@@ -266,8 +300,9 @@ github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG
github.com/go-logr/logr v0.1.0/go.mod h1:ixOQHD9gLJUVQQ2ZOR7zLEifBX6tGkNJF4QyIY7sIas=
github.com/go-logr/logr v0.2.0/go.mod h1:z6/tIYblkpsD+a4lm/fGIIU9mZ+XfAiaFtq7xTgseGU=
github.com/go-logr/logr v0.3.0/go.mod h1:z6/tIYblkpsD+a4lm/fGIIU9mZ+XfAiaFtq7xTgseGU=
-github.com/go-logr/logr v0.4.0 h1:K7/B1jt6fIBQVd4Owv2MqGQClcgf0R266+7C/QjRcLc=
github.com/go-logr/logr v0.4.0/go.mod h1:z6/tIYblkpsD+a4lm/fGIIU9mZ+XfAiaFtq7xTgseGU=
+github.com/go-logr/logr v1.2.0 h1:QK40JKJyMdUDz+h+xvCsru/bJhvG0UxvePV0ufL/AcE=
+github.com/go-logr/logr v1.2.0/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/zapr v0.2.0/go.mod h1:qhKdvif7YF5GI9NWEpyxTSSBdGmzkNguibrdCNVPunU=
github.com/go-openapi/analysis v0.0.0-20180825180245-b006789cd277/go.mod h1:k70tL6pCuVxPJOHXQ+wIac1FUrvNkHolPie/cLEU6hI=
github.com/go-openapi/analysis v0.17.0/go.mod h1:IowGgpVeD0vNm45So8nr+IcQ3pxVtpRoBWb8PVZO0ik=
@@ -281,8 +316,9 @@ github.com/go-openapi/jsonpointer v0.0.0-20160704185906-46af16f9f7b1/go.mod h1:+
github.com/go-openapi/jsonpointer v0.17.0/go.mod h1:cOnomiV+CVVwFLk0A/MExoFMjwdsUdVpsRhURCKh+3M=
github.com/go-openapi/jsonpointer v0.18.0/go.mod h1:cOnomiV+CVVwFLk0A/MExoFMjwdsUdVpsRhURCKh+3M=
github.com/go-openapi/jsonpointer v0.19.2/go.mod h1:3akKfEdA7DF1sugOqz1dVQHBcuDBPKZGEoHC/NkiQRg=
-github.com/go-openapi/jsonpointer v0.19.3 h1:gihV7YNZK1iK6Tgwwsxo2rJbD1GTbdm72325Bq8FI3w=
github.com/go-openapi/jsonpointer v0.19.3/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg=
+github.com/go-openapi/jsonpointer v0.19.5 h1:gZr+CIYByUqjcgeLXnQu2gHYQC9o73G2XUeOFYEICuY=
+github.com/go-openapi/jsonpointer v0.19.5/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg=
github.com/go-openapi/jsonreference v0.0.0-20160704190145-13c6e3589ad9/go.mod h1:W3Z9FmVs9qj+KR4zFKmDPGiLdk1D9Rlm7cyMvf57TTg=
github.com/go-openapi/jsonreference v0.17.0/go.mod h1:g4xxGn04lDIRh0GJb5QlpE3HfopLOL6uZrK/VgnsK9I=
github.com/go-openapi/jsonreference v0.18.0/go.mod h1:g4xxGn04lDIRh0GJb5QlpE3HfopLOL6uZrK/VgnsK9I=
@@ -302,7 +338,6 @@ github.com/go-openapi/spec v0.17.0/go.mod h1:XkF/MOi14NmjsfZ8VtAKf8pIlbZzyoTvZsd
github.com/go-openapi/spec v0.18.0/go.mod h1:XkF/MOi14NmjsfZ8VtAKf8pIlbZzyoTvZsdfssdxcBI=
github.com/go-openapi/spec v0.19.2/go.mod h1:sCxk3jxKgioEJikev4fgkNmwS+3kuYdJtcsZsD5zxMY=
github.com/go-openapi/spec v0.19.3/go.mod h1:FpwSN1ksY1eteniUU7X0N/BgJ7a4WvBFVA8Lj9mJglo=
-github.com/go-openapi/spec v0.19.5 h1:Xm0Ao53uqnk9QE/LlYV5DEU09UAgpliA85QoT9LzqPw=
github.com/go-openapi/spec v0.19.5/go.mod h1:Hm2Jr4jv8G1ciIAo+frC/Ft+rR2kQDh8JHKHb3gWUSk=
github.com/go-openapi/strfmt v0.17.0/go.mod h1:P82hnJI0CXkErkXi8IKjPbNBM6lV6+5pLP5l494TcyU=
github.com/go-openapi/strfmt v0.18.0/go.mod h1:P82hnJI0CXkErkXi8IKjPbNBM6lV6+5pLP5l494TcyU=
@@ -336,19 +371,24 @@ github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903/go.mod h1:cIg4er
github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
-github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e h1:1r7pUrabqp18hOBcwBwiTsbnFeTZHV9eER/QT5JVZxY=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
+github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=
+github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.3.1/go.mod h1:sBzyDLLjw3U8JLTeZvSv8jJB+tU5PVekmnlKIyFUx0Y=
github.com/golang/mock v1.4.0/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
github.com/golang/mock v1.4.1/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
+github.com/golang/mock v1.4.3/go.mod h1:UOMv5ysSaYNkG+OFQykRIcU/QvvxJf3p21QfJ2Bt3cw=
+github.com/golang/mock v1.4.4/go.mod h1:l3mdAwkq5BuhzHwde/uurv3sEJeZMXNpwsxVWU71h+4=
+github.com/golang/mock v1.5.0/go.mod h1:CWnOUgYIOo4TcNZ0wHX3YZCqsaM1I1Jvs6v3mP3KVu8=
github.com/golang/protobuf v0.0.0-20161109072736-4bd1920723d7/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.3.4/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
+github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
@@ -356,21 +396,29 @@ github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:W
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
-github.com/golang/protobuf v1.4.3 h1:JjCZWpVbqXDqFVmTfYWEVTMIYrL/NPdPSCHPJ0T/raM=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
+github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
+github.com/golang/protobuf v1.5.1/go.mod h1:DopwsBzvsk0Fs44TXzsVbJyPhcCPeIwnvohx4u74HPM=
+github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw=
+github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/golangci/lint-1 v0.0.0-20181222135242-d2cdd8c08219/go.mod h1:/X8TswGSh1pIozq4ZwCfxS0WA5JGXguxk94ar/4c87Y=
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
+github.com/google/btree v1.0.1/go.mod h1:xXMiIv4Fb/0kKde4SpL7qlzvu5cMJDRkFDxJfI9uaxA=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
+github.com/google/go-cmp v0.4.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
+github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
-github.com/google/go-cmp v0.5.4 h1:L8R9j+yAqZuZjsqh/z+F1NCffTKKLShY6zXTItVIZ8M=
+github.com/google/go-cmp v0.5.3/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
+github.com/google/go-cmp v0.5.5 h1:Khx7svrCpmxxtHBq5j2mp/xVjsi8hQMfNLvJFAlrGgU=
+github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-github/v28 v28.1.1/go.mod h1:bsqJWQX05omyWVmc00nEUql9mhQyv38lDZ8kPZcQVoM=
github.com/google/go-github/v32 v32.1.0/go.mod h1:rIEpZD9CTDQwDK9GDrtMTycQNA4JU3qBsCizh3q2WCI=
github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
@@ -378,11 +426,19 @@ github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/
github.com/google/gofuzz v1.1.0 h1:Hsa8mG0dQ46ij8Sl2AYJDUv1oA9/d6Vk+3LG99Oe02g=
github.com/google/gofuzz v1.1.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
+github.com/google/martian/v3 v3.0.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
+github.com/google/martian/v3 v3.1.0/go.mod h1:y5Zk1BBys9G+gd6Jrk0W3cC1+ELVxBWuIGO+w/tUAp0=
github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20190515194954-54271f7e092f/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
github.com/google/pprof v0.0.0-20191218002539-d4f498aebedc/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200212024743-f11f1df84d12/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
github.com/google/pprof v0.0.0-20200229191704-1ebb73c60ed3/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
+github.com/google/pprof v0.0.0-20200430221834-fc25d7d30c6d/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
+github.com/google/pprof v0.0.0-20200708004538-1a94d8640e99/go.mod h1:ZgVRPoUq/hfqzAqh7sHMqb3I9Rq5C59dIz2SbBwJ4eM=
+github.com/google/pprof v0.0.0-20201023163331-3e6fc7fc9c4c/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
+github.com/google/pprof v0.0.0-20201203190320-1bf35d6f28c2/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
+github.com/google/pprof v0.0.0-20210122040257-d980be63207e/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
+github.com/google/pprof v0.0.0-20210226084205-cbba55b83ad5/go.mod h1:kpwsk12EmLew5upagYY7GY0pfYCcupk39gWOCRROcvE=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
@@ -393,8 +449,9 @@ github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5m
github.com/googleapis/gnostic v0.0.0-20170729233727-0c5108395e2d/go.mod h1:sJBsCZ4ayReDTBIg8b9dl28c5xFWyhBTVRp3pOg5EKY=
github.com/googleapis/gnostic v0.1.0/go.mod h1:sJBsCZ4ayReDTBIg8b9dl28c5xFWyhBTVRp3pOg5EKY=
github.com/googleapis/gnostic v0.4.1/go.mod h1:LRhVm6pbyptWbWbuZ38d1eyptfvIytN3ir6b65WBswg=
-github.com/googleapis/gnostic v0.5.1 h1:A8Yhf6EtqTv9RMsU6MQTyrtV1TjWlR6xU9BsZIwuTCM=
github.com/googleapis/gnostic v0.5.1/go.mod h1:6U4PtQXGIEt/Z3h5MAT7FNofLnw9vXk2cUuW7uA/OeU=
+github.com/googleapis/gnostic v0.5.5 h1:9fHAtK0uDfpveeqqo1hkEZJcFvYXAiCN3UutL8F9xHw=
+github.com/googleapis/gnostic v0.5.5/go.mod h1:7+EbHbldMins07ALC74bsA81Ovc97DwqyJO1AENw9kA=
github.com/gophercloud/gophercloud v0.1.0/go.mod h1:vxM41WHh5uqHVBMZHzuwNOHh8XEoIEcSTewFxm1c5g8=
github.com/gophercloud/gophercloud v0.15.1-0.20210202035223-633d73521055/go.mod h1:wRtmUelyIIv3CSSDI47aUwbs075O6i+LY+pXsKCBsb4=
github.com/gophercloud/gophercloud v0.16.0/go.mod h1:wRtmUelyIIv3CSSDI47aUwbs075O6i+LY+pXsKCBsb4=
@@ -414,6 +471,7 @@ github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgf
github.com/grpc-ecosystem/grpc-gateway v1.8.5/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
github.com/grpc-ecosystem/grpc-gateway v1.9.0/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
github.com/grpc-ecosystem/grpc-gateway v1.9.5/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
+github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw=
github.com/h2non/parth v0.0.0-20190131123155-b4df798d6542/go.mod h1:Ow0tF8D4Kplbc8s8sSb3V2oUCygFHVp8gC3Dn6U4MNI=
github.com/hashicorp/consul/api v1.1.0/go.mod h1:VmuI/Lkw1nC05EYQWNKwWGbkg+FbDBtguAZLlVdkD9Q=
github.com/hashicorp/consul/api v1.3.0/go.mod h1:MmDNSzIMUjNpY/mQ398R4bk2FnqQLoPndWW5VkKPlCE=
@@ -441,7 +499,6 @@ github.com/hashicorp/go-version v1.2.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09
github.com/hashicorp/go.net v0.0.1/go.mod h1:hjKkEWcCURg++eb33jQU7oqQcI9XDCnUzHA0oac0k90=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
-github.com/hashicorp/golang-lru v0.5.4 h1:YDjusn29QI/Das2iO9M0BHnIbxPeyuCHsjMW+lJfyTc=
github.com/hashicorp/golang-lru v0.5.4/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=
github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64=
@@ -453,6 +510,7 @@ github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpO
github.com/huandu/xstrings v1.3.1/go.mod h1:y5/lhBue+AyNmUVz9RLU9xbLR0o4KIIExikq4ovT0aE=
github.com/hudl/fargo v1.3.0/go.mod h1:y3CKSmjA+wD2gak7sUSXTAoopbhU08POFhmITJgmKTg=
github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
+github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/iij/doapi v0.0.0-20190504054126-0bbf12d6d7df/go.mod h1:QMZY7/J/KSQEhKWFeDesPjMj+wCHReeknARU3wqlyN4=
github.com/imdario/mergo v0.3.5/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
github.com/imdario/mergo v0.3.8/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
@@ -476,8 +534,9 @@ github.com/json-iterator/go v1.1.5/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCV
github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/json-iterator/go v1.1.8/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
-github.com/json-iterator/go v1.1.10 h1:Kz6Cvnvv2wGdaG/V8yMvfkmNiXq9Ya2KUv4rouJJr68=
github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
+github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
+github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
@@ -565,13 +624,15 @@ github.com/mitchellh/mapstructure v1.3.3/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RR
github.com/mitchellh/osext v0.0.0-20151018003038-5e2d6d41470f/go.mod h1:OkQIRizQZAeMln+1tSwduZz7+Af5oFlKirV/MSYes2A=
github.com/mitchellh/reflectwalk v1.0.0/go.mod h1:mSTlrgnPZtwu0c4WaC2kGObEpuNDbx0jmZXqmk4esnw=
github.com/moby/moby v17.12.0-ce-rc1.0.20200618181300-9dc6525e6118+incompatible/go.mod h1:fDXVQ6+S340veQPv35CzDahGBmHsiclFwfEygB/TWMc=
+github.com/moby/spdystream v0.2.0/go.mod h1:f7i0iNDQJ059oMTcWxx8MA/zKFIuD/lY+0GqbN2Wy8c=
github.com/moby/term v0.0.0-20200312100748-672ec06f55cd/go.mod h1:DdlQx2hp0Ss5/fLikoLlEeIYiATotOjgB//nb973jeo=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
-github.com/modern-go/reflect2 v1.0.1 h1:9f412s+6RmYXLWZSEzVVgPGK7C2PphHj5RJrvfx9AWI=
github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
+github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
+github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/morikuni/aec v0.0.0-20170113033406-39771216ff4c/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
github.com/munnerz/goautoneg v0.0.0-20120707110453-a547fc61f48d/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
@@ -607,6 +668,7 @@ github.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+W
github.com/onsi/ginkgo v1.8.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.11.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.12.1/go.mod h1:zj2OWP4+oCPe1qIXoGWkgMRwljMUYCdkwsT2108oapk=
+github.com/onsi/ginkgo v1.14.0/go.mod h1:iSB4RoI2tjJc9BBv4NKIKWKya62Rps+oPG/Lv9klQyY=
github.com/onsi/ginkgo v1.14.1 h1:jMU0WaQrP0a/YAEq8eJmJKjBoMs+pClEr1vDMlM/Do4=
github.com/onsi/ginkgo v1.14.1/go.mod h1:iSB4RoI2tjJc9BBv4NKIKWKya62Rps+oPG/Lv9klQyY=
github.com/onsi/gomega v0.0.0-20170829124025-dcabb60a477c/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA=
@@ -707,6 +769,7 @@ github.com/rainycape/memcache v0.0.0-20150622160815-1031fa0ce2f2/go.mod h1:7tZKc
github.com/rancher/go-rancher-metadata v0.0.0-20200311180630-7f4c936a06ac/go.mod h1:67sLWL17mVlO1HFROaTBmU71NB4R8UNCesFHhg0f6LQ=
github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=
github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg=
+github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=
github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
@@ -811,8 +874,11 @@ github.com/xeipuuv/gojsonschema v0.0.0-20180618132009-1d523034197f/go.mod h1:5yf
github.com/xeipuuv/gojsonschema v1.2.0/go.mod h1:anYRn/JVcOK2ZgGU+IjEV4nwlhoK5sQluxsYJ78Id3Y=
github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=
github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
+github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
+github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
+github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43/go.mod h1:aX5oPXxHm3bOH+xeAttToC8pqch2ScQN/JoXYupl6xs=
github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50/go.mod h1:NUSPSUX/bi6SeDMUh6brw0nXpxHnc96TguQh0+r/ssA=
github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f/go.mod h1:GlGEuHIJweS1mbCqG+7vt2nvWLzLLnRHbXz5JKd/Qbg=
@@ -836,6 +902,10 @@ go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
go.opencensus.io v0.22.0/go.mod h1:+kGneAE2xo2IficOXnaByMWTGM9T73dGwxeWcUqIpI8=
go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
+go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
+go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=
+go.opencensus.io v0.23.0/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E=
+go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI=
go.uber.org/atomic v1.3.2/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
go.uber.org/atomic v1.5.0/go.mod h1:sABNBOSYdrvTF6hTgEIbc7YasKWGhgEQZyfxyTvoXHQ=
@@ -873,8 +943,9 @@ golang.org/x/crypto v0.0.0-20200414173820-0848c9571904/go.mod h1:LzIPMQfyMNhhGPh
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20201016220609-9e8e0b390897/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
-golang.org/x/crypto v0.0.0-20201221181555-eec23a3978ad h1:DN0cp81fZ3njFcrLCytUHRSUkqBjfTo4Tx9RJTWs0EY=
golang.org/x/crypto v0.0.0-20201221181555-eec23a3978ad/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
+golang.org/x/crypto v0.0.0-20210817164053-32db794688a5 h1:HWj/xjIHfjYU5nVXpTM0s39J9CbLn7Cc5a7IC5rwsMQ=
+golang.org/x/crypto v0.0.0-20210817164053-32db794688a5/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@@ -897,6 +968,7 @@ golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHl
golang.org/x/lint v0.0.0-20191125180803-fdd1cda4f05f/go.mod h1:5qLYkcX4OjUUV8bRuDixDT3tpyyb+LUpUlRWLxfhWrs=
golang.org/x/lint v0.0.0-20200130185559-910be7a94367/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
+golang.org/x/lint v0.0.0-20201208152925-83fdc39ff7b5/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/mobile v0.0.0-20190312151609-d3739f865fa6/go.mod h1:z+o9i4GpDbdi3rU15maQ/Ox0txvL9dWGYEHz965HBQE=
golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028/go.mod h1:E/iHnbuqvinMTCcRqshq8CkpyQDoeVncDDYHnLhea+o=
golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
@@ -905,8 +977,11 @@ golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzB
golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
-golang.org/x/mod v0.3.1-0.20200828183125-ce943fd02449 h1:xUIPaMhvROX9dhPvRCenIJtU78+lbEenGbgqB5hfHCQ=
golang.org/x/mod v0.3.1-0.20200828183125-ce943fd02449/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
+golang.org/x/mod v0.4.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
+golang.org/x/mod v0.4.1/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
+golang.org/x/mod v0.4.2 h1:Gz96sIWK3OalVv/I/qNygP42zyoKp3xptRVCWRFEBvo=
+golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/net v0.0.0-20170114055629-f2499483f923/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -943,29 +1018,52 @@ golang.org/x/net v0.0.0-20200222125558-5a598a2470a0/go.mod h1:z5CRVTTTmAJ677TzLL
golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200301022130-244492dfa37a/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200324143707-d3edc9973b7e/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
+golang.org/x/net v0.0.0-20200501053045-e0ff5e5a1de5/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
+golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
+golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200520004742-59133d7f0dd7/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
+golang.org/x/net v0.0.0-20200520182314-0ba52f642ac2/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
+golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
+golang.org/x/net v0.0.0-20200707034311-ab3426394381/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
+golang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
+golang.org/x/net v0.0.0-20201209123823-ac852fbbde11/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
+golang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210220033124-5f55cee0dc0d/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
-golang.org/x/net v0.0.0-20210224082022-3d97a244fca7 h1:OgUuv8lsRpBibGNbSizVwKWlysjaNzmC9gYMhPVfqFM=
golang.org/x/net v0.0.0-20210224082022-3d97a244fca7/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
+golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
+golang.org/x/net v0.0.0-20210316092652-d523dce5a7f4/go.mod h1:RBQZq4jEuRlivfhVLdyRGr576XBO4/greRjx4P4O3yc=
+golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM=
+golang.org/x/net v0.0.0-20210825183410-e898025ed96a/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
+golang.org/x/net v0.0.0-20211209124913-491a49abca63 h1:iocB37TsdFuN6IBRZ+ry36wrkoV51/tl5vOWqkcPGvY=
+golang.org/x/net v0.0.0-20211209124913-491a49abca63/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20191202225959-858c2ad4c8b6/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
-golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d h1:TzXSXBo42m9gQenoE3b9BGiEpg5IG2JkU5FkPIawgtw=
golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
+golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
+golang.org/x/oauth2 v0.0.0-20201109201403-9fd604954f58/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
+golang.org/x/oauth2 v0.0.0-20201208152858-08078c50e5b5/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
+golang.org/x/oauth2 v0.0.0-20210218202405-ba52d332ba99/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
+golang.org/x/oauth2 v0.0.0-20210220000619-9bb904979d93/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
+golang.org/x/oauth2 v0.0.0-20210313182246-cd4f82c27b84/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
+golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f h1:Qmd2pbz05z7z6lm0DrgQVVPuBm92jqujBKMHMOlOQEw=
+golang.org/x/oauth2 v0.0.0-20210819190943-2bc19b11175f/go.mod h1:KelEdhl1UZF7XfJ4dDtk6s++YSgaE7mD/BuKKDLBl4A=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sync v0.0.0-20201207232520-09787c993a3a h1:DcqTD9SDLc+1P/r1EmRBwnVsrOwW+kk2vWf9n+1sGhs=
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20210220032951-036812b2e83c h1:5KslGYwFpkhGh+Q16bwMP3cOontH8FOep7tGV86Y7SQ=
+golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20170830134202-bb24a47a89ea/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180622082034-63fc586f45fe/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@@ -1019,35 +1117,58 @@ golang.org/x/sys v0.0.0-20200212091648-12a6c2dcc1e4/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200331124033-c3d80250170d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200501052902-10377860bb8e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200511232937-7e40ca221e25/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200519105757-fe76b779f299/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200523222454-059865788121/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200909081042-eff7692f9009/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200918174421-af09f7315aff/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201110211018-35f3e6cf4a65/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201112073958-5cba982894dd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20201201145000-ef89a241ccb3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210104204734-6f8348627aad/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210119212857-b64e53b001e4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20210426230700-d19ff857e887 h1:dXfMednGJh/SUUFjTLsWJz3P+TQt9qnR11GgeI3vWKs=
+golang.org/x/sys v0.0.0-20210220050731-9a76102bfb43/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210305230114-8fe3ee5dd75b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210315160823-c6e025ad8005/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210426230700-d19ff857e887/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.0.0-20210831042530-f4d43177bf5e h1:XMgFehsDnnLGtjvjOfqWSUzt0alpTR1RSEuznObga2c=
+golang.org/x/sys v0.0.0-20210831042530-f4d43177bf5e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
-golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1 h1:v+OssWQX+hTHEmOBgwxdZxK4zHq3yOs8F9J7mk0PY8E=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
+golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b h1:9zKuko04nR4gjZ4+DNjHqRlAJqbJETHwiNKDqTfOjfE=
+golang.org/x/term v0.0.0-20210615171337-6886f2dfbf5b/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/text v0.0.0-20160726164857-2910a502d2bf/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
-golang.org/x/text v0.3.4 h1:0YWbFKbhXG/wIiuHDSKpS0Iy7FSA+u45VtBMfQcFTTc=
golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
+golang.org/x/text v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
+golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
-golang.org/x/time v0.0.0-20201208040808-7e3f01d25324 h1:Hir2P/De0WpUhtrKGGjvSb2YxUgyZ7EFOSLIcSSpiwE=
golang.org/x/time v0.0.0-20201208040808-7e3f01d25324/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac h1:7zkz7BUtwNFFqcowJ+RIgu2MaV/MapERkDIy+mwPyjs=
+golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20180828015842-6cd1fcedba52/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
@@ -1093,15 +1214,30 @@ golang.org/x/tools v0.0.0-20200204074204-1cc6d1ef6c74/go.mod h1:TB2adYChydJhpapK
golang.org/x/tools v0.0.0-20200207183749-b753a1ba74fa/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200212150539-ea181f53ac56/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200224181240-023911ca70b2/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
+golang.org/x/tools v0.0.0-20200227222343-706bc42d1f0d/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=
+golang.org/x/tools v0.0.0-20200312045724-11d5b4c81c7d/go.mod h1:o4KQGtdN14AW+yjsvvwRTJJuXz8XRtIHtEnmAXLyFUw=
+golang.org/x/tools v0.0.0-20200331025713-a30bf2db82d4/go.mod h1:Sl4aGygMT6LrqrWclx+PTx3U+LnKx/seiNR+3G19Ar8=
+golang.org/x/tools v0.0.0-20200501065659-ab2804fb9c9d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200505023115-26f46d2f7ef8/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
+golang.org/x/tools v0.0.0-20200512131952-2bc93b1c0c88/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
+golang.org/x/tools v0.0.0-20200515010526-7d3b6ebf133d/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200616133436-c1934b75d054/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200616195046-dc31b401abb5/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
+golang.org/x/tools v0.0.0-20200618134242-20370b0cb4b2/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
+golang.org/x/tools v0.0.0-20200729194436-6467de6f59a7/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
+golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
+golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE=
+golang.org/x/tools v0.0.0-20201110124207-079ba7bd75cd/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
+golang.org/x/tools v0.0.0-20201201161351-ac6f37ff4c2a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
+golang.org/x/tools v0.0.0-20201208233053-a543418bbed2/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
+golang.org/x/tools v0.0.0-20210105154028-b0ab187a4818/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
-golang.org/x/tools v0.1.0 h1:po9/4sTYwZU9lPhi1tOrb4hCv3qrhiQ77LZfGa2OjwY=
golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0=
+golang.org/x/tools v0.1.5 h1:ouewzE6p+/VEB31YYnTbEJdi8pFqKp4P4n85vwo3DHA=
+golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -1119,15 +1255,27 @@ google.golang.org/api v0.14.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsb
google.golang.org/api v0.15.0/go.mod h1:iLdEw5Ide6rF15KTC1Kkl0iskquN2gFfn9o9XIsbkAI=
google.golang.org/api v0.17.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.18.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
+google.golang.org/api v0.19.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
google.golang.org/api v0.20.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
+google.golang.org/api v0.22.0/go.mod h1:BwFmGc8tA3vsd7r/7kR8DY7iEEGSU04BFxCo5jP/sfE=
+google.golang.org/api v0.24.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
+google.golang.org/api v0.28.0/go.mod h1:lIXQywCXRcnZPGlsd8NbLnOjtAoL6em04bJ9+z0MncE=
+google.golang.org/api v0.29.0/go.mod h1:Lcubydp8VUV7KeIHD9z2Bys/sm/vGKnG1UHuDBSrHWM=
+google.golang.org/api v0.30.0/go.mod h1:QGmEvQ87FHZNiUVJkT14jQNYJ4ZJjdRF23ZXz5138Fc=
+google.golang.org/api v0.35.0/go.mod h1:/XrVsuzM0rZmrsbjJutiuftIzeuTQcEeaYcSk/mQ1dg=
+google.golang.org/api v0.36.0/go.mod h1:+z5ficQTmoYpPn8LCUNVpK5I7hwkpjbcgqA7I34qYtE=
+google.golang.org/api v0.40.0/go.mod h1:fYKFpnQN0DsDSKRVRcQSDQNtqWPfM9i+zNPxepjRCQ8=
+google.golang.org/api v0.41.0/go.mod h1:RkxM5lITDfTzmyKFPt+wGrCJbVfniCr2ool8kTBzRTU=
+google.golang.org/api v0.43.0/go.mod h1:nQsDGjRXMo4lvh5hP0TKqF244gqhGcr/YSIykhUk/94=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.2.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.6.1/go.mod h1:i06prIuMbXzDqacNJfV5OdTW448YApPu5ww/cMBSeb0=
google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
-google.golang.org/appengine v1.6.6 h1:lMO5rYAqUxkmaj76jAkRUvt5JZgFymx/+Q5Mzfivuhc=
google.golang.org/appengine v1.6.6/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
+google.golang.org/appengine v1.6.7 h1:FZR1q0exgwxzPzp/aF+VccGrSfxfPpkBqjIIEq3ru6c=
+google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/cloud v0.0.0-20151119220103-975617b05ea8/go.mod h1:0H1ncTHf11KCFhTc/+EFRbzSCOZx+VUbRMk55Yv5MYk=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
@@ -1147,9 +1295,31 @@ google.golang.org/genproto v0.0.0-20200122232147-0452cf42e150/go.mod h1:n3cpQtvx
google.golang.org/genproto v0.0.0-20200204135345-fa8e72b47b90/go.mod h1:GmwEX6Z4W5gMy59cAlVYjN9JhxgbQH6Gn+gFDQe2lzA=
google.golang.org/genproto v0.0.0-20200212174721-66ed5ce911ce/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
+google.golang.org/genproto v0.0.0-20200228133532-8c2c7df3a383/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
google.golang.org/genproto v0.0.0-20200305110556-506484158171/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
+google.golang.org/genproto v0.0.0-20200312145019-da6875a35672/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
+google.golang.org/genproto v0.0.0-20200331122359-1ee6d9798940/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
+google.golang.org/genproto v0.0.0-20200430143042-b979b6f78d84/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
+google.golang.org/genproto v0.0.0-20200511104702-f5ebc3bea380/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
+google.golang.org/genproto v0.0.0-20200513103714-09dca8ec2884/go.mod h1:55QSHmfGQM9UVYDPBsyGGes0y52j32PQ3BqQfXhyH3c=
+google.golang.org/genproto v0.0.0-20200515170657-fc4c6c6a6587/go.mod h1:YsZOwe1myG/8QRHRsmBRE1LrgQY60beZKjly0O1fX9U=
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
+google.golang.org/genproto v0.0.0-20200618031413-b414f8b61790/go.mod h1:jDfRM7FcilCzHH/e9qn6dsT145K34l5v+OpcnNgKAAA=
+google.golang.org/genproto v0.0.0-20200729003335-053ba62fc06f/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20201019141844-1ed22bb0c154/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20201109203340-2640f1f9cdfb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201110150050-8816d57aaa9a/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20201201144952-b05cb90ed32e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20201210142538-e3217bee35cc/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20201214200347-8c77b98c765d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20210222152913-aa3ee6e6a81c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20210303154014-9728d6b83eeb/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20210310155132-4ce2db91004e/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20210319143718-93e7006c17a6/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
+google.golang.org/genproto v0.0.0-20210402141018-6c239bbf2bb1/go.mod h1:9lPAdzaEmUacj36I+k7YKbEc5CXzPIeORRgDAUOu28A=
google.golang.org/grpc v0.0.0-20160317175043-d3ddb4469d5a/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmEdcZw=
google.golang.org/grpc v1.17.0/go.mod h1:6QZJwpn2B+Zp71q/5VxRsJ6NXXVCE5NRUHRo+f3cWCs=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
@@ -1161,9 +1331,22 @@ google.golang.org/grpc v1.21.1/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ij
google.golang.org/grpc v1.22.1/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.23.1/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
+google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
+google.golang.org/grpc v1.28.0/go.mod h1:rpkK4SK4GF4Ach/+MFLZUBavHOvF2JJB5uozKKal+60=
+google.golang.org/grpc v1.29.1/go.mod h1:itym6AZVZYACWQqET3MqgPpjcuV5QH3BxFS3IjizoKk=
+google.golang.org/grpc v1.30.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
+google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
+google.golang.org/grpc v1.31.1/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
+google.golang.org/grpc v1.33.1/go.mod h1:fr5YgcSWrqhRRxogOsw7RzIpsmvOZ6IcH4kBYTpR3n0=
+google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
+google.golang.org/grpc v1.34.0/go.mod h1:WotjhfgOW/POjDeRt8vscBtXq+2VjORFy659qA51WJ8=
+google.golang.org/grpc v1.35.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=
+google.golang.org/grpc v1.36.0/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=
+google.golang.org/grpc v1.36.1/go.mod h1:qjiiYl8FncCW8feJPdyg3v6XW24KsRHe+dy9BAGRRjU=
+google.golang.org/grpc v1.42.0/go.mod h1:k+4IHHFw41K8+bbowsex27ge2rCb65oeWqe4jJ590SU=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
@@ -1173,8 +1356,11 @@ google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGjtUeSXeh4=
-google.golang.org/protobuf v1.25.0 h1:Ejskq+SyPohKW+1uil0JJMtmHCgJPJ/qWTxr8qp+R4c=
google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
+google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
+google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
+google.golang.org/protobuf v1.27.1 h1:SnqbnDw1V7RiZcXPx5MEeqPv2s79L9i7BJUlG/+RurQ=
+google.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
gopkg.in/DataDog/dd-trace-go.v1 v1.19.0/go.mod h1:DVp8HmDh8PuTu2Z0fVVlBsyWaC++fzwVCaGWylTe3tg=
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
@@ -1233,26 +1419,35 @@ honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWh
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
+honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k=
howett.net/plist v0.0.0-20181124034731-591f970eefbb/go.mod h1:vMygbs4qMhSZSc4lCUl2OEE+rDiIIJAIdR4m7MiMcm0=
+istio.io/api v0.0.0-20220208030606-6253688c0c91 h1:qvatjbu3y8lfjqnlTg/nr4vnyvbgT2uyMFiPVB0A7GA=
+istio.io/api v0.0.0-20220208030606-6253688c0c91/go.mod h1:RVwW5wv5bZYrmHvhYtyp0EQvM2VN3Gw4tLpUrynDCH0=
+istio.io/client-go v1.13.0 h1:j0+6JTN84V7v9UfXohHtW6zYWjC5bpADBIA39Z/x64U=
+istio.io/client-go v1.13.0/go.mod h1:LS9ufOL5Vqp0MP/HF8Ken6+gwi+rZloVQqlx9/8ek2A=
+istio.io/gogo-genproto v0.0.0-20211208193508-5ab4acc9eb1e h1:z2WI3y55w0K3c6hmarcp5EcOiP4vVpTBXA8nYstP+cE=
+istio.io/gogo-genproto v0.0.0-20211208193508-5ab4acc9eb1e/go.mod h1:vJDAniIqryf/z///fgZqVPKJ7N2lBk7Gg8DCTB7oCfU=
k8s.io/api v0.18.2/go.mod h1:SJCWI7OLzhZSvbY7U8zwNl9UA4o1fizoug34OV/2r78=
k8s.io/api v0.20.1/go.mod h1:KqwcCVogGxQY3nBlRpwt+wpAMF/KjaCc7RpywacvqUo=
k8s.io/api v0.20.2/go.mod h1:d7n6Ehyzx+S+cE3VhTGfVNNqtGc/oL9DCdYYahlurV8=
-k8s.io/api v0.20.4 h1:xZjKidCirayzX6tHONRQyTNDVIR55TYVqgATqo6ZULY=
-k8s.io/api v0.20.4/go.mod h1:++lNL1AJMkDymriNniQsWRkMDzRaX2Y/POTUi8yvqYQ=
+k8s.io/api v0.23.0/go.mod h1:8wmDdLBHBNxtOIytwLstXt5E9PddnZb0GaMcqsvDBpg=
+k8s.io/api v0.23.1 h1:ncu/qfBfUoClqwkTGbeRqqOqBCRoUAflMuOaOD7J0c8=
+k8s.io/api v0.23.1/go.mod h1:WfXnOnwSqNtG62Y1CdjoMxh7r7u9QXGCkA1u0na2jgo=
k8s.io/apiextensions-apiserver v0.18.2/go.mod h1:q3faSnRGmYimiocj6cHQ1I3WpLqmDgJFlKL37fC4ZvY=
k8s.io/apiextensions-apiserver v0.20.1/go.mod h1:ntnrZV+6a3dB504qwC5PN/Yg9PBiDNt1EVqbW2kORVk=
k8s.io/apimachinery v0.18.2/go.mod h1:9SnR/e11v5IbyPCGbvJViimtJ0SwHG4nfZFjU77ftcA=
k8s.io/apimachinery v0.20.1/go.mod h1:WlLqWAHZGg07AeltaI0MV5uk1Omp8xaN0JGLY6gkRpU=
k8s.io/apimachinery v0.20.2/go.mod h1:WlLqWAHZGg07AeltaI0MV5uk1Omp8xaN0JGLY6gkRpU=
-k8s.io/apimachinery v0.20.4 h1:vhxQ0PPUUU2Ns1b9r4/UFp13UPs8cw2iOoTjnY9faa0=
-k8s.io/apimachinery v0.20.4/go.mod h1:WlLqWAHZGg07AeltaI0MV5uk1Omp8xaN0JGLY6gkRpU=
+k8s.io/apimachinery v0.23.0/go.mod h1:fFCTTBKvKcwTPFzjlcxp91uPFZr+JA0FubU4fLzzFYc=
+k8s.io/apimachinery v0.23.1 h1:sfBjlDFwj2onG0Ijx5C+SrAoeUscPrmghm7wHP+uXlo=
+k8s.io/apimachinery v0.23.1/go.mod h1:SADt2Kl8/sttJ62RRsi9MIV4o8f5S3coArm0Iu3fBno=
k8s.io/apiserver v0.18.2/go.mod h1:Xbh066NqrZO8cbsoenCwyDJ1OSi8Ag8I2lezeHxzwzw=
k8s.io/apiserver v0.20.1/go.mod h1:ro5QHeQkgMS7ZGpvf4tSMx6bBOgPfE+f52KwvXfScaU=
k8s.io/client-go v0.18.2/go.mod h1:Xcm5wVGXX9HAA2JJ2sSBUn3tCJ+4SVlCbl2MNNv+CIU=
k8s.io/client-go v0.20.1/go.mod h1:/zcHdt1TeWSd5HoUe6elJmHSQ6uLLgp4bIJHVEuy+/Y=
k8s.io/client-go v0.20.2/go.mod h1:kH5brqWqp7HDxUFKoEgiI4v8G1xzbe9giaCenUWJzgE=
-k8s.io/client-go v0.20.4 h1:85crgh1IotNkLpKYKZHVNI1JT86nr/iDCvq2iWKsql4=
-k8s.io/client-go v0.20.4/go.mod h1:LiMv25ND1gLUdBeYxBIwKpkSC5IsozMMmOOeSJboP+k=
+k8s.io/client-go v0.23.1 h1:Ma4Fhf/p07Nmj9yAB1H7UwbFHEBrSPg8lviR24U2GiQ=
+k8s.io/client-go v0.23.1/go.mod h1:6QSI8fEuqD4zgFK0xbdwfB/PthBsIxCJMa3s17WlcO0=
k8s.io/code-generator v0.18.2/go.mod h1:+UHX5rSbxmR8kzS+FAv7um6dtYrZokQvjHpDSYRVkTc=
k8s.io/code-generator v0.20.1/go.mod h1:UsqdF+VX4PU2g46NC2JRs4gc+IfrctnwHb76RNbWHJg=
k8s.io/code-generator v0.20.2/go.mod h1:UsqdF+VX4PU2g46NC2JRs4gc+IfrctnwHb76RNbWHJg=
@@ -1265,8 +1460,9 @@ k8s.io/gengo v0.0.0-20200114144118-36b2048a9120/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8
k8s.io/gengo v0.0.0-20200413195148-3a45101e95ac/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0=
k8s.io/gengo v0.0.0-20201113003025-83324d819ded/go.mod h1:FiNAH4ZV3gBg2Kwh89tzAEV2be7d5xI0vBa/VySYy3E=
k8s.io/gengo v0.0.0-20201203183100-97869a43a9d9/go.mod h1:FiNAH4ZV3gBg2Kwh89tzAEV2be7d5xI0vBa/VySYy3E=
-k8s.io/gengo v0.0.0-20201214224949-b6c5ce23f027 h1:Uusb3oh8XcdzDF/ndlI4ToKTYVlkCSJP39SRY2mfRAw=
k8s.io/gengo v0.0.0-20201214224949-b6c5ce23f027/go.mod h1:FiNAH4ZV3gBg2Kwh89tzAEV2be7d5xI0vBa/VySYy3E=
+k8s.io/gengo v0.0.0-20210813121822-485abfe95c7c h1:GohjlNKauSai7gN4wsJkeZ3WAJx4Sh+oT/b5IYn5suA=
+k8s.io/gengo v0.0.0-20210813121822-485abfe95c7c/go.mod h1:FiNAH4ZV3gBg2Kwh89tzAEV2be7d5xI0vBa/VySYy3E=
k8s.io/klog v0.0.0-20181102134211-b9b56d5dfc92/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk=
k8s.io/klog v0.2.0/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk=
k8s.io/klog v0.3.0/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk=
@@ -1275,16 +1471,20 @@ k8s.io/klog v1.0.0/go.mod h1:4Bi6QPql/J/LkTDqv7R/cd3hPo4k2DG6Ptcz060Ez5I=
k8s.io/klog/v2 v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE=
k8s.io/klog/v2 v2.2.0/go.mod h1:Od+F08eJP+W3HUb4pSrPpgp9DGU4GzlpG/TmITuYh/Y=
k8s.io/klog/v2 v2.4.0/go.mod h1:Od+F08eJP+W3HUb4pSrPpgp9DGU4GzlpG/TmITuYh/Y=
-k8s.io/klog/v2 v2.8.0 h1:Q3gmuM9hKEjefWFFYF0Mat+YyFJvsUyYuwyNNJ5C9Ts=
k8s.io/klog/v2 v2.8.0/go.mod h1:hy9LJ/NvuK+iVyP4Ehqva4HxZG/oXyIS3n3Jmire4Ec=
+k8s.io/klog/v2 v2.30.0 h1:bUO6drIvCIsvZ/XFgfxoGFQU/a4Qkh0iAlvUR7vlHJw=
+k8s.io/klog/v2 v2.30.0/go.mod h1:y1WjHnz7Dj687irZUWR/WLkLc5N1YHtjLdmgWjndZn0=
k8s.io/kube-openapi v0.0.0-20200121204235-bf4fb3bd569c/go.mod h1:GRQhZsXIAJ1xR0C9bd8UpWHZ5plfAS9fzPjJuQ6JL3E=
k8s.io/kube-openapi v0.0.0-20201113171705-d219536bb9fd/go.mod h1:WOJ3KddDSol4tAGcJo0Tvi+dK12EcqSLqcWsryKMpfM=
-k8s.io/kube-openapi v0.0.0-20210305001622-591a79e4bda7 h1:vEx13qjvaZ4yfObSSXW7BrMc/KQBBT/Jyee8XtLf4x0=
k8s.io/kube-openapi v0.0.0-20210305001622-591a79e4bda7/go.mod h1:wXW5VT87nVfh/iLV8FpR2uDvrFyomxbtb1KivDbvPTE=
+k8s.io/kube-openapi v0.0.0-20211115234752-e816edb12b65 h1:E3J9oCLlaobFUqsjG9DfKbP2BmgwBL2p7pn0A3dG9W4=
+k8s.io/kube-openapi v0.0.0-20211115234752-e816edb12b65/go.mod h1:sX9MT8g7NVZM5lVL/j8QyCCJe8YSMW30QvGZWaCIDIk=
k8s.io/kubernetes v1.13.0/go.mod h1:ocZa8+6APFNC2tX1DZASIbocyYT5jHzqFVsY5aoB7Jk=
k8s.io/utils v0.0.0-20200324210504-a9aa75ae1b89/go.mod h1:sZAwmy6armz5eXlNoLmJcl4F1QuKu7sr+mFQ0byX7Ew=
-k8s.io/utils v0.0.0-20201110183641-67b214c5f920 h1:CbnUZsM497iRC5QMVkHwyl8s2tB3g7yaSHkYPkpgelw=
k8s.io/utils v0.0.0-20201110183641-67b214c5f920/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA=
+k8s.io/utils v0.0.0-20210802155522-efc7438f0176/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA=
+k8s.io/utils v0.0.0-20210930125809-cb0fa318a74b h1:wxEMGetGMur3J1xuGLQY7GEQYg9bZxKn3tKo5k/eYcs=
+k8s.io/utils v0.0.0-20210930125809-cb0fa318a74b/go.mod h1:jPW/WVKK9YHAvNhRxK0md/EJ228hCsBRufyofKtW8HA=
launchpad.net/gocheck v0.0.0-20140225173054-000000000087/go.mod h1:hj7XX3B/0A+80Vse0e+BUHsHMTEhd0O4cpUHr/e/BUM=
mvdan.cc/xurls/v2 v2.1.0/go.mod h1:5GrSd9rOnKOpZaji1OZLYL/yeAAtGDlo/cFe+8K5n8E=
rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
@@ -1295,11 +1495,14 @@ sigs.k8s.io/apiserver-network-proxy/konnectivity-client v0.0.14/go.mod h1:LEScyz
sigs.k8s.io/controller-runtime v0.8.0/go.mod h1:v9Lbj5oX443uR7GXYY46E0EE2o7k2YxQ58GxVNeXSW4=
sigs.k8s.io/controller-tools v0.4.1/go.mod h1:G9rHdZMVlBDocIxGkK3jHLWqcTMNvveypYJwrvYKjWU=
sigs.k8s.io/gateway-api v0.2.0/go.mod h1:IUbl4vAjUFoa2nt2gER8NsUrAu84x2edpWXbXBvcNis=
+sigs.k8s.io/json v0.0.0-20211020170558-c049b76a60c6 h1:fD1pz4yfdADVNfFmcP2aBEtudwUQ1AlLnRBALr33v3s=
+sigs.k8s.io/json v0.0.0-20211020170558-c049b76a60c6/go.mod h1:p4QtZmO4uMYipTQNzagwnNoseA6OxSUutVw05NhYDRs=
sigs.k8s.io/structured-merge-diff/v3 v3.0.0-20200116222232-67a7b8c61874/go.mod h1:PlARxl6Hbt/+BC80dRLi1qAmnMqwqDg62YvvVkZjemw=
sigs.k8s.io/structured-merge-diff/v3 v3.0.0/go.mod h1:PlARxl6Hbt/+BC80dRLi1qAmnMqwqDg62YvvVkZjemw=
sigs.k8s.io/structured-merge-diff/v4 v4.0.2/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw=
-sigs.k8s.io/structured-merge-diff/v4 v4.1.0 h1:C4r9BgJ98vrKnnVCjwCSXcWjWe0NKcUQkmzDXZXGwH8=
sigs.k8s.io/structured-merge-diff/v4 v4.1.0/go.mod h1:bJZC9H9iH24zzfZ/41RGcq60oK1F7G282QMXDPYydCw=
+sigs.k8s.io/structured-merge-diff/v4 v4.1.2 h1:Hr/htKFmJEbtMgS/UD0N+gtgctAqz81t3nu+sPzynno=
+sigs.k8s.io/structured-merge-diff/v4 v4.1.2/go.mod h1:j/nl6xW8vLS49O8YvXW1ocPhZawJtm+Yrr7PPRQ0Vg4=
sigs.k8s.io/yaml v1.1.0/go.mod h1:UJmg0vDUVViEyp3mgSv9WPwZCDxu4rQW1olrI1uml+o=
sigs.k8s.io/yaml v1.2.0 h1:kr/MCeFWJWTwyaHoR9c8EjH9OumOmoF9YGiZd7lFm/Q=
sigs.k8s.io/yaml v1.2.0/go.mod h1:yfXDCHCao9+ENCvLSE62v9VSji2MKu5jeNfTrofGhJc=
diff --git a/submarine-cloud-v2/main.go b/submarine-cloud-v2/main.go
index 486dda54..f977ae93 100644
--- a/submarine-cloud-v2/main.go
+++ b/submarine-cloud-v2/main.go
@@ -33,8 +33,11 @@ import (
"k8s.io/client-go/tools/clientcmd"
"k8s.io/klog/v2"
- traefikclientset "github.com/traefik/traefik/v2/pkg/provider/kubernetes/crd/generated/clientset/versioned"
- traefikinformers "github.com/traefik/traefik/v2/pkg/provider/kubernetes/crd/generated/informers/externalversions"
+ // traefikclientset "github.com/traefik/traefik/v2/pkg/provider/kubernetes/crd/generated/clientset/versioned"
+ // traefikinformers "github.com/traefik/traefik/v2/pkg/provider/kubernetes/crd/generated/informers/externalversions"
+
+ istioClientset "istio.io/client-go/pkg/clientset/versioned"
+ istioInformers "istio.io/client-go/pkg/informers/externalversions"
)
var (
@@ -75,14 +78,22 @@ func main() {
klog.Fatalf("Error building submarine clientset: %s", err.Error())
}
- traefikClient, err := traefikclientset.NewForConfig(cfg)
+ /*
+ traefikClient, err := traefikclientset.NewForConfig(cfg)
+ if err != nil {
+ klog.Fatalf("Error building traefik clientset: %s", err.Error())
+ }
+ */
+
+ istioClient, err := istioClientset.NewForConfig(cfg)
if err != nil {
- klog.Fatalf("Error building traefik clientset: %s", err.Error())
+ klog.Fatalf("Error building istio clientset: %s", err.Error())
}
kubeInformerFactory := kubeinformers.NewSharedInformerFactory(kubeClient, time.Second*30)
submarineInformerFactory := informers.NewSharedInformerFactory(submarineClient, time.Second*30)
- traefikInformerFactory := traefikinformers.NewSharedInformerFactory(traefikClient, time.Second*30)
+ // traefikInformerFactory := traefikinformers.NewSharedInformerFactory(traefikClient, time.Second*30)
+ istioInformerFactory := istioInformers.NewSharedInformerFactory(istioClient, time.Second*30)
// TODO: Pass informers to NewController()
// ex: namespace informer
@@ -94,17 +105,20 @@ func main() {
createPodSecurityPolicy,
kubeClient,
submarineClient,
- traefikClient,
+ // traefikClient,
+ istioClient,
kubeInformerFactory,
submarineInformerFactory,
- traefikInformerFactory,
+ // traefikInformerFactory,
+ istioInformerFactory,
)
// notice that there is no need to run Start methods in a separate goroutine. (i.e. go kubeInformerFactory.Start(stopCh)
// Start method is non-blocking and runs all registered informers in a dedicated goroutine.
kubeInformerFactory.Start(stopCh)
submarineInformerFactory.Start(stopCh)
- traefikInformerFactory.Start(stopCh)
+ // traefikInformerFactory.Start(stopCh)
+ istioInformerFactory.Start(stopCh)
// Run controller
if err = submarineController.Run(1, stopCh); err != nil {
@@ -118,10 +132,12 @@ func NewSubmarineController(
createPodSecurityPolicy bool,
kubeClient *kubernetes.Clientset,
submarineClient *clientset.Clientset,
- traefikClient *traefikclientset.Clientset,
+ // traefikClient *traefikclientset.Clientset,
+ istioClient *istioClientset.Clientset,
kubeInformerFactory kubeinformers.SharedInformerFactory,
submarineInformerFactory informers.SharedInformerFactory,
- traefikInformerFactory traefikinformers.SharedInformerFactory,
+ // traefikInformerFactory traefikinformers.SharedInformerFactory,
+ istioInformerFactory istioInformers.SharedInformerFactory,
) *controller.Controller {
bc := controller.NewControllerBuilderConfig()
bc.
@@ -130,7 +146,8 @@ func NewSubmarineController(
WithCreatePodSecurityPolicy(createPodSecurityPolicy).
WithKubeClientset(kubeClient).
WithSubmarineClientset(submarineClient).
- WithTraefikClientset(traefikClient).
+ // WithTraefikClientset(traefikClient).
+ WithVirtualServiceClientset(istioClient).
WithSubmarineInformer(submarineInformerFactory.Submarine().V1alpha1().Submarines()).
WithDeploymentInformer(kubeInformerFactory.Apps().V1().Deployments()).
WithStatefulSetInformer(kubeInformerFactory.Apps().V1().StatefulSets()).
@@ -139,7 +156,8 @@ func NewSubmarineController(
WithServiceAccountInformer(kubeInformerFactory.Core().V1().ServiceAccounts()).
WithPersistentVolumeClaimInformer(kubeInformerFactory.Core().V1().PersistentVolumeClaims()).
WithIngressInformer(kubeInformerFactory.Extensions().V1beta1().Ingresses()).
- WithIngressRouteInformer(traefikInformerFactory.Traefik().V1alpha1().IngressRoutes()).
+ // WithIngressRouteInformer(traefikInformerFactory.Traefik().V1alpha1().IngressRoutes()).
+ WithVirtualServiceInformer(istioInformerFactory.Networking().V1alpha3().VirtualServices()).
WithRoleInformer(kubeInformerFactory.Rbac().V1().Roles()).
WithRoleBindingInformer(kubeInformerFactory.Rbac().V1().RoleBindings())
diff --git a/submarine-cloud-v2/pkg/apis/submarine/v1alpha1/zz_generated.deepcopy.go b/submarine-cloud-v2/pkg/apis/submarine/v1alpha1/zz_generated.deepcopy.go
index 9d5736db..95c428eb 100644
--- a/submarine-cloud-v2/pkg/apis/submarine/v1alpha1/zz_generated.deepcopy.go
+++ b/submarine-cloud-v2/pkg/apis/submarine/v1alpha1/zz_generated.deepcopy.go
@@ -1,3 +1,4 @@
+//go:build !ignore_autogenerated
// +build !ignore_autogenerated
/*
diff --git a/submarine-cloud-v2/pkg/controller/controller.go b/submarine-cloud-v2/pkg/controller/controller.go
index cf7bd038..10117c99 100644
--- a/submarine-cloud-v2/pkg/controller/controller.go
+++ b/submarine-cloud-v2/pkg/controller/controller.go
@@ -27,6 +27,8 @@ import (
clientset "github.com/apache/submarine/submarine-cloud-v2/pkg/client/clientset/versioned"
listers "github.com/apache/submarine/submarine-cloud-v2/pkg/client/listers/submarine/v1alpha1"
+ // traefik "github.com/traefik/traefik/v2/pkg/provider/kubernetes/crd/generated/clientset/versioned"
+
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
rbacv1 "k8s.io/api/rbac/v1"
@@ -45,8 +47,11 @@ import (
"k8s.io/client-go/util/workqueue"
"k8s.io/klog/v2"
- traefik "github.com/traefik/traefik/v2/pkg/provider/kubernetes/crd/generated/clientset/versioned"
- traefiklisters "github.com/traefik/traefik/v2/pkg/provider/kubernetes/crd/generated/listers/traefik/v1alpha1"
+ // traefik "github.com/traefik/traefik/v2/pkg/provider/kubernetes/crd/generated/clientset/versioned"
+ // traefiklisters "github.com/traefik/traefik/v2/pkg/provider/kubernetes/crd/generated/listers/traefik/v1alpha1"
+
+ istioClientset "istio.io/client-go/pkg/clientset/versioned"
+ istioListers "istio.io/client-go/pkg/listers/networking/v1alpha3"
)
const controllerAgentName = "submarine-controller"
@@ -63,6 +68,7 @@ const (
minioName = "submarine-minio"
storageName = "submarine-storage"
ingressName = serverName + "-ingress"
+ virtualServiceName = "submarine-virtual-service"
databasePvcName = databaseName + "-pvc"
tensorboardPvcName = tensorboardName + "-pvc"
tensorboardServiceName = tensorboardName + "-service"
@@ -124,7 +130,8 @@ type Controller struct {
kubeclientset kubernetes.Interface
// sampleclientset is a clientset for our own API group
submarineclientset clientset.Interface
- traefikclientset traefik.Interface
+ // traefikclientset traefik.Interface
+ istioClientset istioClientset.Interface
submarinesLister listers.SubmarineLister
submarinesSynced cache.InformerSynced
@@ -136,9 +143,10 @@ type Controller struct {
serviceLister corelisters.ServiceLister
persistentvolumeclaimLister corelisters.PersistentVolumeClaimLister
ingressLister extlisters.IngressLister
- ingressrouteLister traefiklisters.IngressRouteLister
- roleLister rbaclisters.RoleLister
- rolebindingLister rbaclisters.RoleBindingLister
+ // ingressrouteLister traefiklisters.IngressRouteLister
+ virtualServiceLister istioListers.VirtualServiceLister
+ roleLister rbaclisters.RoleLister
+ rolebindingLister rbaclisters.RoleBindingLister
// workqueue is a rate limited work queue. This is used to queue work to be
// processed instead of performing it as soon as a change happens. This
// means we can ensure we only process a fixed amount of resources at a
diff --git a/submarine-cloud-v2/pkg/controller/controller_builder.go b/submarine-cloud-v2/pkg/controller/controller_builder.go
index 17b8fca2..192fea4e 100644
--- a/submarine-cloud-v2/pkg/controller/controller_builder.go
+++ b/submarine-cloud-v2/pkg/controller/controller_builder.go
@@ -74,7 +74,8 @@ func (cb *ControllerBuilder) initialize() *ControllerBuilder {
func (cb *ControllerBuilder) addClientsets() *ControllerBuilder {
cb.controller.kubeclientset = cb.config.kubeclientset
cb.controller.submarineclientset = cb.config.submarineclientset
- cb.controller.traefikclientset = cb.config.traefikclientset
+ // cb.controller.traefikclientset = cb.config.traefikclientset
+ cb.controller.istioClientset = cb.config.istioClientset
return cb
}
@@ -90,7 +91,8 @@ func (cb *ControllerBuilder) addListers() *ControllerBuilder {
cb.controller.serviceaccountLister = cb.config.serviceaccountInformer.Lister()
cb.controller.persistentvolumeclaimLister = cb.config.persistentvolumeclaimInformer.Lister()
cb.controller.ingressLister = cb.config.ingressInformer.Lister()
- cb.controller.ingressrouteLister = cb.config.ingressrouteInformer.Lister()
+ // cb.controller.ingressrouteLister = cb.config.ingressrouteInformer.Lister()
+ cb.controller.virtualServiceLister = cb.config.virtualServiceInformer.Lister()
cb.controller.roleLister = cb.config.roleInformer.Lister()
cb.controller.rolebindingLister = cb.config.rolebindingInformer.Lister()
@@ -110,7 +112,7 @@ func (cb *ControllerBuilder) addEventHandlers() *ControllerBuilder {
cb.addServiceAccountEventHandlers()
cb.addPersistentVolumeClaimEventHandlers()
cb.addIngressEventHandlers()
- cb.addIngressRouteEventHandlers()
+ // cb.addIngressRouteEventHandlers()
cb.addRoleEventHandlers()
cb.addRoleBindingEventHandlers()
diff --git a/submarine-cloud-v2/pkg/controller/controller_builder_config.go b/submarine-cloud-v2/pkg/controller/controller_builder_config.go
index 337616bc..b8a2dcc9 100644
--- a/submarine-cloud-v2/pkg/controller/controller_builder_config.go
+++ b/submarine-cloud-v2/pkg/controller/controller_builder_config.go
@@ -20,8 +20,11 @@ package controller
import (
clientset "github.com/apache/submarine/submarine-cloud-v2/pkg/client/clientset/versioned"
informers "github.com/apache/submarine/submarine-cloud-v2/pkg/client/informers/externalversions/submarine/v1alpha1"
- traefik "github.com/traefik/traefik/v2/pkg/provider/kubernetes/crd/generated/clientset/versioned"
- traefikinformers "github.com/traefik/traefik/v2/pkg/provider/kubernetes/crd/generated/informers/externalversions/traefik/v1alpha1"
+
+ // traefik "github.com/traefik/traefik/v2/pkg/provider/kubernetes/crd/generated/clientset/versioned"
+ // traefikinformers "github.com/traefik/traefik/v2/pkg/provider/kubernetes/crd/generated/informers/externalversions/traefik/v1alpha1"
+ istio "istio.io/client-go/pkg/clientset/versioned"
+ istioInformers "istio.io/client-go/pkg/informers/externalversions/networking/v1alpha3"
appsinformers "k8s.io/client-go/informers/apps/v1"
coreinformers "k8s.io/client-go/informers/core/v1"
extinformers "k8s.io/client-go/informers/extensions/v1beta1"
@@ -30,12 +33,13 @@ import (
)
type BuilderConfig struct {
- incluster bool
- clusterType string
- createPodSecurityPolicy bool
- kubeclientset kubernetes.Interface
- submarineclientset clientset.Interface
- traefikclientset traefik.Interface
+ incluster bool
+ clusterType string
+ createPodSecurityPolicy bool
+ kubeclientset kubernetes.Interface
+ submarineclientset clientset.Interface
+ // traefikclientset traefik.Interface
+ istioClientset istio.Interface
namespaceInformer coreinformers.NamespaceInformer
deploymentInformer appsinformers.DeploymentInformer
statefulsetInformer appsinformers.StatefulSetInformer
@@ -43,10 +47,11 @@ type BuilderConfig struct {
serviceaccountInformer coreinformers.ServiceAccountInformer
persistentvolumeclaimInformer coreinformers.PersistentVolumeClaimInformer
ingressInformer extinformers.IngressInformer
- ingressrouteInformer traefikinformers.IngressRouteInformer
- roleInformer rbacinformers.RoleInformer
- rolebindingInformer rbacinformers.RoleBindingInformer
- submarineInformer informers.SubmarineInformer
+ // ingressrouteInformer traefikinformers.IngressRouteInformer
+ virtualServiceInformer istioInformers.VirtualServiceInformer
+ roleInformer rbacinformers.RoleInformer
+ rolebindingInformer rbacinformers.RoleBindingInformer
+ submarineInformer informers.SubmarineInformer
}
func NewControllerBuilderConfig() *BuilderConfig {
@@ -88,12 +93,21 @@ func (bc *BuilderConfig) WithSubmarineClientset(
return bc
}
+/*
func (bc *BuilderConfig) WithTraefikClientset(
traefikclientset traefik.Interface,
) *BuilderConfig {
bc.traefikclientset = traefikclientset
return bc
}
+*/
+
+func (bc *BuilderConfig) WithVirtualServiceClientset(
+ istioClientset istio.Interface,
+) *BuilderConfig {
+ bc.istioClientset = istioClientset
+ return bc
+}
func (bc *BuilderConfig) WithSubmarineInformer(
submarineInformer informers.SubmarineInformer,
@@ -151,12 +165,21 @@ func (bc *BuilderConfig) WithIngressInformer(
return bc
}
+/*
func (bc *BuilderConfig) WithIngressRouteInformer(
ingressrouteInformer traefikinformers.IngressRouteInformer,
) *BuilderConfig {
bc.ingressrouteInformer = ingressrouteInformer
return bc
}
+*/
+
+func (bc *BuilderConfig) WithVirtualServiceInformer(
+ virtualServiceInformer istioInformers.VirtualServiceInformer,
+) *BuilderConfig {
+ bc.virtualServiceInformer = virtualServiceInformer
+ return bc
+}
func (bc *BuilderConfig) WithRoleInformer(
roleInformer rbacinformers.RoleInformer,
diff --git a/submarine-cloud-v2/pkg/controller/controller_event_handlers.go b/submarine-cloud-v2/pkg/controller/controller_event_handlers.go
index ff15e270..1e0036c5 100644
--- a/submarine-cloud-v2/pkg/controller/controller_event_handlers.go
+++ b/submarine-cloud-v2/pkg/controller/controller_event_handlers.go
@@ -18,7 +18,7 @@
package controller
import (
- traefikv1alpha1 "github.com/traefik/traefik/v2/pkg/provider/kubernetes/crd/traefik/v1alpha1"
+ // traefikv1alpha1 "github.com/traefik/traefik/v2/pkg/provider/kubernetes/crd/traefik/v1alpha1"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
extensionsv1beta1 "k8s.io/api/extensions/v1beta1"
@@ -139,6 +139,7 @@ func (cb *ControllerBuilder) addIngressEventHandlers() *ControllerBuilder {
return cb
}
+/*
func (cb *ControllerBuilder) addIngressRouteEventHandlers() *ControllerBuilder {
cb.config.ingressrouteInformer.Informer().AddEventHandler(cache.ResourceEventHandlerFuncs{
AddFunc: cb.controller.handleObject,
@@ -155,6 +156,7 @@ func (cb *ControllerBuilder) addIngressRouteEventHandlers() *ControllerBuilder {
return cb
}
+*/
func (cb *ControllerBuilder) addRoleEventHandlers() *ControllerBuilder {
cb.config.roleInformer.Informer().AddEventHandler(cache.ResourceEventHandlerFuncs{
diff --git a/submarine-cloud-v2/pkg/controller/parser.go b/submarine-cloud-v2/pkg/controller/parser.go
index 57279594..e844f3e8 100644
--- a/submarine-cloud-v2/pkg/controller/parser.go
+++ b/submarine-cloud-v2/pkg/controller/parser.go
@@ -26,6 +26,7 @@ import (
"github.com/pkg/errors"
traefikv1alpha1 "github.com/traefik/traefik/v2/pkg/provider/kubernetes/crd/traefik/v1alpha1"
+ istiov1alpha3 "istio.io/client-go/pkg/apis/networking/v1alpha3"
appsv1 "k8s.io/api/apps/v1"
v1 "k8s.io/api/core/v1"
extensionsv1beta1 "k8s.io/api/extensions/v1beta1"
@@ -178,3 +179,14 @@ func ParseIngressRouteYaml(relativePath string) (*traefikv1alpha1.IngressRoute,
json.Unmarshal(marshaled, &ingressRoute)
return &ingressRoute, nil
}
+
+// ParseVirtualService parse VirtualService from yaml file.
+func ParseVirtualService(relativePath string) (*istiov1alpha3.VirtualService, error) {
+ var virtualService istiov1alpha3.VirtualService
+ marshaled, err := parseYaml(relativePath, "VirtualService")
+ if err != nil {
+ return nil, err
+ }
+ json.Unmarshal(marshaled, &virtualService)
+ return &virtualService, nil
+}
diff --git a/submarine-cloud-v2/pkg/controller/submarine_ingress.go b/submarine-cloud-v2/pkg/controller/submarine_ingress.go
index 375f9f01..6daf1f93 100644
--- a/submarine-cloud-v2/pkg/controller/submarine_ingress.go
+++ b/submarine-cloud-v2/pkg/controller/submarine_ingress.go
@@ -22,25 +22,25 @@ import (
"fmt"
v1alpha1 "github.com/apache/submarine/submarine-cloud-v2/pkg/apis/submarine/v1alpha1"
+ istiov1alpha3 "istio.io/client-go/pkg/apis/networking/v1alpha3"
corev1 "k8s.io/api/core/v1"
- extensionsv1beta1 "k8s.io/api/extensions/v1beta1"
"k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/klog/v2"
)
-func newSubmarineServerIngress(submarine *v1alpha1.Submarine) *extensionsv1beta1.Ingress {
- ingress, err := ParseIngressYaml(ingressYamlPath)
+func newSubmarineVirtualService(submarine *v1alpha1.Submarine) *istiov1alpha3.VirtualService {
+ virtualService, err := ParseVirtualService(ingressYamlPath)
if err != nil {
- klog.Info("[Error] ParseIngressYaml", err)
+ klog.Info("[Error] ParseGatewayYaml", err)
}
- ingress.ObjectMeta.OwnerReferences = []metav1.OwnerReference{
+ virtualService.ObjectMeta.OwnerReferences = []metav1.OwnerReference{
*metav1.NewControllerRef(submarine, v1alpha1.SchemeGroupVersion.WithKind("Submarine")),
}
- return ingress
+ return virtualService
}
// createIngress is a function to create Ingress.
@@ -48,14 +48,13 @@ func newSubmarineServerIngress(submarine *v1alpha1.Submarine) *extensionsv1beta1
func (c *Controller) createIngress(submarine *v1alpha1.Submarine) error {
klog.Info("[createIngress]")
- // Step1: Create ServiceAccount
- ingress, err := c.ingressLister.Ingresses(submarine.Namespace).Get(ingressName)
+ virtualService, err := c.virtualServiceLister.VirtualServices(submarine.Namespace).Get(virtualServiceName)
// If the resource doesn't exist, we'll create it
if errors.IsNotFound(err) {
- ingress, err = c.kubeclientset.ExtensionsV1beta1().Ingresses(submarine.Namespace).Create(context.TODO(),
- newSubmarineServerIngress(submarine),
+ virtualService, err = c.istioClientset.NetworkingV1alpha3().VirtualServices(submarine.Namespace).Create(context.TODO(),
+ newSubmarineVirtualService(submarine),
metav1.CreateOptions{})
- klog.Info(" Create Ingress: ", ingress.Name)
+ klog.Info(" Create Ingress: ", virtualService.Name)
}
// If an error occurs during Get/Create, we'll requeue the item so we can
@@ -65,8 +64,8 @@ func (c *Controller) createIngress(submarine *v1alpha1.Submarine) error {
return err
}
- if !metav1.IsControlledBy(ingress, submarine) {
- msg := fmt.Sprintf(MessageResourceExists, ingress.Name)
+ if !metav1.IsControlledBy(virtualService, submarine) {
+ msg := fmt.Sprintf(MessageResourceExists, virtualService.Name)
c.recorder.Event(submarine, corev1.EventTypeWarning, ErrResourceExists, msg)
return fmt.Errorf(msg)
}
diff --git a/submarine-cloud-v2/pkg/controller/submarine_minio.go b/submarine-cloud-v2/pkg/controller/submarine_minio.go
index c38650cf..545b2b63 100644
--- a/submarine-cloud-v2/pkg/controller/submarine_minio.go
+++ b/submarine-cloud-v2/pkg/controller/submarine_minio.go
@@ -28,8 +28,6 @@ import (
"k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/klog/v2"
-
- traefikv1alpha1 "github.com/traefik/traefik/v2/pkg/provider/kubernetes/crd/traefik/v1alpha1"
)
func newSubmarineMinioPersistentVolumeClaim(submarine *v1alpha1.Submarine) *corev1.PersistentVolumeClaim {
@@ -67,17 +65,6 @@ func newSubmarineMinioService(submarine *v1alpha1.Submarine) *corev1.Service {
return service
}
-func newSubmarineMinioIngressRoute(submarine *v1alpha1.Submarine) *traefikv1alpha1.IngressRoute {
- ingressRoute, err := ParseIngressRouteYaml(minioYamlPath)
- if err != nil {
- klog.Info("[Error] ParseIngressRouteYaml", err)
- }
- ingressRoute.ObjectMeta.OwnerReferences = []metav1.OwnerReference{
- *metav1.NewControllerRef(submarine, v1alpha1.SchemeGroupVersion.WithKind("Submarine")),
- }
- return ingressRoute
-}
-
// createSubmarineMinio is a function to create submarine-minio.
// Reference: https://github.com/apache/submarine/blob/master/helm-charts/submarine/templates/submarine-minio.yaml
func (c *Controller) createSubmarineMinio(submarine *v1alpha1.Submarine) error {
@@ -153,28 +140,5 @@ func (c *Controller) createSubmarineMinio(submarine *v1alpha1.Submarine) error {
return fmt.Errorf(msg)
}
- // Step 4: Create IngressRoute
- ingressroute, err := c.ingressrouteLister.IngressRoutes(submarine.Namespace).Get(minioIngressRouteName)
- // If the resource doesn't exist, we'll create it
- if errors.IsNotFound(err) {
- ingressroute, err = c.traefikclientset.TraefikV1alpha1().IngressRoutes(submarine.Namespace).Create(context.TODO(), newSubmarineMinioIngressRoute(submarine), metav1.CreateOptions{})
- if err != nil {
- klog.Info(err)
- }
- klog.Info(" Create IngressRoute: ", ingressroute.Name)
- }
- // If an error occurs during Get/Create, we'll requeue the item so we can
- // attempt processing again later. This could have been caused by a
- // temporary network failure, or any other transient reason.
- if err != nil {
- return err
- }
-
- if !metav1.IsControlledBy(ingressroute, submarine) {
- msg := fmt.Sprintf(MessageResourceExists, ingressroute.Name)
- c.recorder.Event(submarine, corev1.EventTypeWarning, ErrResourceExists, msg)
- return fmt.Errorf(msg)
- }
-
return nil
}
diff --git a/submarine-cloud-v2/pkg/controller/submarine_mlflow.go b/submarine-cloud-v2/pkg/controller/submarine_mlflow.go
index c5c963dc..91d9f5c8 100644
--- a/submarine-cloud-v2/pkg/controller/submarine_mlflow.go
+++ b/submarine-cloud-v2/pkg/controller/submarine_mlflow.go
@@ -22,7 +22,6 @@ import (
"fmt"
v1alpha1 "github.com/apache/submarine/submarine-cloud-v2/pkg/apis/submarine/v1alpha1"
- traefikv1alpha1 "github.com/traefik/traefik/v2/pkg/provider/kubernetes/crd/traefik/v1alpha1"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/errors"
@@ -65,17 +64,6 @@ func newSubmarineMlflowService(submarine *v1alpha1.Submarine) *corev1.Service {
return service
}
-func newSubmarineMlflowIngressRoute(submarine *v1alpha1.Submarine) *traefikv1alpha1.IngressRoute {
- ingressRoute, err := ParseIngressRouteYaml(mlflowYamlPath)
- if err != nil {
- klog.Info("[Error] ParseIngressRouteYaml", err)
- }
- ingressRoute.ObjectMeta.OwnerReferences = []metav1.OwnerReference{
- *metav1.NewControllerRef(submarine, v1alpha1.SchemeGroupVersion.WithKind("Submarine")),
- }
- return ingressRoute
-}
-
// createSubmarineMlflow is a function to create submarine-mlflow.
// Reference: https://github.com/apache/submarine/blob/master/helm-charts/submarine/templates/submarine-mlflow.yaml
func (c *Controller) createSubmarineMlflow(submarine *v1alpha1.Submarine) error {
@@ -151,28 +139,5 @@ func (c *Controller) createSubmarineMlflow(submarine *v1alpha1.Submarine) error
return fmt.Errorf(msg)
}
- // Step 4: Create IngressRoute
- ingressroute, err := c.ingressrouteLister.IngressRoutes(submarine.Namespace).Get(mlflowIngressRouteName)
- // If the resource doesn't exist, we'll create it
- if errors.IsNotFound(err) {
- ingressroute, err = c.traefikclientset.TraefikV1alpha1().IngressRoutes(submarine.Namespace).Create(context.TODO(), newSubmarineMlflowIngressRoute(submarine), metav1.CreateOptions{})
- if err != nil {
- klog.Info(err)
- }
- klog.Info(" Create IngressRoute: ", ingressroute.Name)
- }
- // If an error occurs during Get/Create, we'll requeue the item so we can
- // attempt processing again later. This could have been caused by a
- // temporary network failure, or any other transient reason.
- if err != nil {
- return err
- }
-
- if !metav1.IsControlledBy(ingressroute, submarine) {
- msg := fmt.Sprintf(MessageResourceExists, ingressroute.Name)
- c.recorder.Event(submarine, corev1.EventTypeWarning, ErrResourceExists, msg)
- return fmt.Errorf(msg)
- }
-
return nil
}
diff --git a/submarine-cloud-v2/pkg/controller/submarine_tensorboard.go b/submarine-cloud-v2/pkg/controller/submarine_tensorboard.go
index b8b29f3b..d2b42a9c 100644
--- a/submarine-cloud-v2/pkg/controller/submarine_tensorboard.go
+++ b/submarine-cloud-v2/pkg/controller/submarine_tensorboard.go
@@ -28,8 +28,6 @@ import (
"k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/klog/v2"
-
- traefikv1alpha1 "github.com/traefik/traefik/v2/pkg/provider/kubernetes/crd/traefik/v1alpha1"
)
func newSubmarineTensorboardPersistentVolumeClaim(submarine *v1alpha1.Submarine) *corev1.PersistentVolumeClaim {
@@ -67,17 +65,6 @@ func newSubmarineTensorboardService(submarine *v1alpha1.Submarine) *corev1.Servi
return service
}
-func newSubmarineTensorboardIngressRoute(submarine *v1alpha1.Submarine) *traefikv1alpha1.IngressRoute {
- ingressRoute, err := ParseIngressRouteYaml(tensorboardYamlPath)
- if err != nil {
- klog.Info("[Error] ParseIngressRouteYaml", err)
- }
- ingressRoute.ObjectMeta.OwnerReferences = []metav1.OwnerReference{
- *metav1.NewControllerRef(submarine, v1alpha1.SchemeGroupVersion.WithKind("Submarine")),
- }
- return ingressRoute
-}
-
// createSubmarineTensorboard is a function to create submarine-tensorboard.
// Reference: https://github.com/apache/submarine/blob/master/helm-charts/submarine/templates/submarine-tensorboard.yaml
func (c *Controller) createSubmarineTensorboard(submarine *v1alpha1.Submarine) error {
@@ -153,28 +140,5 @@ func (c *Controller) createSubmarineTensorboard(submarine *v1alpha1.Submarine) e
return fmt.Errorf(msg)
}
- // Step 4: Create IngressRoute
- ingressroute, err := c.ingressrouteLister.IngressRoutes(submarine.Namespace).Get(tensorboardIngressRouteName)
- // If the resource doesn't exist, we'll create it
- if errors.IsNotFound(err) {
- ingressroute, err = c.traefikclientset.TraefikV1alpha1().IngressRoutes(submarine.Namespace).Create(context.TODO(), newSubmarineTensorboardIngressRoute(submarine), metav1.CreateOptions{})
- if err != nil {
- klog.Info(err)
- }
- klog.Info(" Create IngressRoute: ", ingressroute.Name)
- }
- // If an error occurs during Get/Create, we'll requeue the item so we can
- // attempt processing again later. This could have been caused by a
- // temporary network failure, or any other transient reason.
- if err != nil {
- return err
- }
-
- if !metav1.IsControlledBy(ingressroute, submarine) {
- msg := fmt.Sprintf(MessageResourceExists, ingressroute.Name)
- c.recorder.Event(submarine, corev1.EventTypeWarning, ErrResourceExists, msg)
- return fmt.Errorf(msg)
- }
-
return nil
}
diff --git a/submarine-serve/src/main/java/org/apache/submarine/serve/istio/IstioHTTPDestination.java b/submarine-serve/src/main/java/org/apache/submarine/serve/istio/IstioHTTPDestination.java
index fc375de1..93b888bf 100644
--- a/submarine-serve/src/main/java/org/apache/submarine/serve/istio/IstioHTTPDestination.java
+++ b/submarine-serve/src/main/java/org/apache/submarine/serve/istio/IstioHTTPDestination.java
@@ -25,12 +25,14 @@ public class IstioHTTPDestination {
@SerializedName("destination")
private IstioDestination destination;
- public IstioHTTPDestination(String host){
+ public IstioHTTPDestination(String host) {
this.destination = new IstioDestination(host);
}
+ public IstioHTTPDestination(String host, Integer port) {
+ this.destination = new IstioDestination(host, port);
+ }
-
- public static class IstioDestination{
+ public static class IstioDestination {
@SerializedName("host")
private String host;
@@ -38,8 +40,12 @@ public class IstioHTTPDestination {
private IstioPort port;
public IstioDestination(String host) {
+ this(host, IstioConstants.DEFAULT_SERVE_POD_PORT);
+ }
+
+ public IstioDestination(String host, Integer port) {
this.host = host;
- this.port = new IstioPort(IstioConstants.DEFAULT_SERVE_POD_PORT);
+ this.port = new IstioPort(port);
}
}
@@ -47,7 +53,7 @@ public class IstioHTTPDestination {
@SerializedName("number")
private Integer number;
- public IstioPort(Integer port){
+ public IstioPort(Integer port) {
this.number = port;
}
}
diff --git a/submarine-serve/src/main/java/org/apache/submarine/serve/istio/IstioHTTPRoute.java b/submarine-serve/src/main/java/org/apache/submarine/serve/istio/IstioHTTPRoute.java
index 7f7ff9ab..f370d213 100644
--- a/submarine-serve/src/main/java/org/apache/submarine/serve/istio/IstioHTTPRoute.java
+++ b/submarine-serve/src/main/java/org/apache/submarine/serve/istio/IstioHTTPRoute.java
@@ -38,6 +38,11 @@ public class IstioHTTPRoute {
this.rewrite = new IstioRewrite(IstioConstants.REWRITE_URL);
}
+ public IstioHTTPRoute(String matchURIPrefix, String routeDestinationHost, Integer routeDestinationPort) {
+ this.match.add(new IstioHTTPMatchRequest(matchURIPrefix));
+ this.route.add(new IstioHTTPDestination(routeDestinationHost, routeDestinationPort));
+ }
+
@Override
public String toString() {
return "'rewrite': {'uri': " + rewrite.getRewrite() + "}";
diff --git a/submarine-serve/src/main/java/org/apache/submarine/serve/istio/IstioVirtualService.java b/submarine-serve/src/main/java/org/apache/submarine/serve/istio/IstioVirtualService.java
index 54d5526d..420d5707 100644
--- a/submarine-serve/src/main/java/org/apache/submarine/serve/istio/IstioVirtualService.java
+++ b/submarine-serve/src/main/java/org/apache/submarine/serve/istio/IstioVirtualService.java
@@ -23,9 +23,6 @@ import io.kubernetes.client.common.KubernetesObject;
import io.kubernetes.client.openapi.models.V1ObjectMeta;
import org.apache.submarine.serve.utils.IstioConstants;
-import java.util.ArrayList;
-import java.util.List;
-
public class IstioVirtualService implements KubernetesObject {
@SerializedName("apiVersion")
private String apiVersion = IstioConstants.API_VERSION;
@@ -46,6 +43,11 @@ public class IstioVirtualService implements KubernetesObject {
private transient String plural = IstioConstants.PLURAL;
+ public IstioVirtualService(V1ObjectMeta metadata, IstioVirtualServiceSpec spec) {
+ this.metadata = metadata;
+ this.spec = spec;
+ }
+
public IstioVirtualService(String modelName, Integer modelVersion) {
V1ObjectMeta metadata = new V1ObjectMeta();
metadata.setName(modelName);
@@ -110,45 +112,4 @@ public class IstioVirtualService implements KubernetesObject {
this.spec = istioVirtualServiceSpec;
}
- public static class IstioVirtualServiceSpec {
- @SerializedName("hosts")
- private List<String> hosts = new ArrayList<>();
- @SerializedName("gateways")
- private List<String> gateways = new ArrayList<>();
- @SerializedName("http")
- private List<IstioHTTPRoute> httpRoute = new ArrayList<>();
-
- public IstioVirtualServiceSpec(String modelName, Integer modelVersion) {
- hosts.add(IstioConstants.DEFAULT_INGRESS_HOST);
- gateways.add(IstioConstants.DEFAULT_GATEWAY);
- IstioHTTPDestination destination = new IstioHTTPDestination(
- modelName + "-" + IstioConstants.DEFAULT_NAMESPACE);
- IstioHTTPMatchRequest matchRequest = new IstioHTTPMatchRequest("/" + modelName
- + "/" + String.valueOf(modelVersion) + "/");
- IstioHTTPRoute httpRoute = new IstioHTTPRoute();
- httpRoute.addHTTPDestination(destination);
- httpRoute.addHTTPMatchRequest(matchRequest);
- setHTTPRoute(httpRoute);
- }
-
- public List<String> getHosts() {
- return this.hosts;
- }
-
- public void addHost(String host) {
- hosts.add(host);
- }
-
- public List<String> getGateways() {
- return this.gateways;
- }
-
- public void addGateway(String gateway) {
- gateways.add(gateway);
- }
-
- public void setHTTPRoute(IstioHTTPRoute istioHTTPRoute){
- this.httpRoute.add(istioHTTPRoute);
- }
- }
}
diff --git a/submarine-serve/src/main/java/org/apache/submarine/serve/istio/IstioVirtualServiceSpec.java b/submarine-serve/src/main/java/org/apache/submarine/serve/istio/IstioVirtualServiceSpec.java
new file mode 100644
index 00000000..e8182cbb
--- /dev/null
+++ b/submarine-serve/src/main/java/org/apache/submarine/serve/istio/IstioVirtualServiceSpec.java
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.submarine.serve.istio;
+
+import com.google.gson.annotations.SerializedName;
+import org.apache.submarine.serve.utils.IstioConstants;
+
+import java.util.ArrayList;
+import java.util.List;
+
+public class IstioVirtualServiceSpec {
+ @SerializedName("hosts")
+ private List<String> hosts = new ArrayList<>();
+ @SerializedName("gateways")
+ private List<String> gateways = new ArrayList<>();
+ @SerializedName("http")
+ private List<IstioHTTPRoute> httpRoute = new ArrayList<>();
+
+ public IstioVirtualServiceSpec() {
+ }
+
+ public IstioVirtualServiceSpec(String modelName, Integer modelVersion) {
+ hosts.add(IstioConstants.DEFAULT_INGRESS_HOST);
+ gateways.add(IstioConstants.DEFAULT_GATEWAY);
+ IstioHTTPDestination destination = new IstioHTTPDestination(
+ modelName + "-" + IstioConstants.DEFAULT_NAMESPACE);
+ IstioHTTPMatchRequest matchRequest = new IstioHTTPMatchRequest("/" + modelName
+ + "/" + String.valueOf(modelVersion) + "/");
+ IstioHTTPRoute httpRoute = new IstioHTTPRoute();
+ httpRoute.addHTTPDestination(destination);
+ httpRoute.addHTTPMatchRequest(matchRequest);
+ setHTTPRoute(httpRoute);
+ }
+
+ public List<String> getHosts() {
+ return this.hosts;
+ }
+
+ public void addHost(String host) {
+ hosts.add(host);
+ }
+
+ public List<String> getGateways() {
+ return this.gateways;
+ }
+
+ public void addGateway(String gateway) {
+ gateways.add(gateway);
+ }
+
+ public void setHTTPRoute(IstioHTTPRoute istioHTTPRoute) {
+ this.httpRoute.add(istioHTTPRoute);
+ }
+}
diff --git a/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/experiment/Info.java b/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/experiment/Info.java
index 42a35898..acae15ea 100644
--- a/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/experiment/Info.java
+++ b/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/experiment/Info.java
@@ -19,9 +19,9 @@
package org.apache.submarine.server.api.experiment;
-public class Info {
+public abstract class Info {
public boolean available;
- public String url;
+ public final String url;
public Info(boolean available, String url) {
this.available = available;
@@ -40,7 +40,8 @@ public class Info {
return url;
}
- public void setUrl(String url) {
- this.url = url;
+ @Override
+ public String toString() {
+ return this.getClass().getSimpleName() + "{" + "available=" + available + ", url='" + url + "'" + "}";
}
}
diff --git a/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/experiment/MlflowInfo.java b/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/experiment/MlflowInfo.java
index b85b83f2..67cff28f 100644
--- a/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/experiment/MlflowInfo.java
+++ b/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/experiment/MlflowInfo.java
@@ -19,17 +19,8 @@
package org.apache.submarine.server.api.experiment;
-public class MlflowInfo extends Info{
-
- public MlflowInfo(Info info) {
- super(info.isAvailable(), info.getUrl());
- }
-
- @Override
- public String toString() {
- return "MLflowInfo{" +
- "available=" + available +
- ", url='" + url + '\'' +
- '}';
+public class MlflowInfo extends Info {
+ public MlflowInfo(boolean available) {
+ super(available, "/mlflow/");
}
}
diff --git a/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/experiment/TensorboardInfo.java b/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/experiment/TensorboardInfo.java
index 96002eb0..9769ee6e 100644
--- a/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/experiment/TensorboardInfo.java
+++ b/submarine-server/server-api/src/main/java/org/apache/submarine/server/api/experiment/TensorboardInfo.java
@@ -19,17 +19,8 @@
package org.apache.submarine.server.api.experiment;
-public class TensorboardInfo extends Info{
-
- public TensorboardInfo(Info info) {
- super(info.isAvailable(), info.getUrl());
- }
-
- @Override
- public String toString() {
- return "TensorboardInfo{" +
- "available=" + available +
- ", url='" + url + '\'' +
- '}';
+public class TensorboardInfo extends Info {
+ public TensorboardInfo(boolean available) {
+ super(available, "/tensorboard/");
}
}
diff --git a/submarine-server/server-submitter/submitter-k8s/src/main/java/org/apache/submarine/server/submitter/k8s/K8sSubmitter.java b/submarine-server/server-submitter/submitter-k8s/src/main/java/org/apache/submarine/server/submitter/k8s/K8sSubmitter.java
index d06eef39..afa9296f 100644
--- a/submarine-server/server-submitter/submitter-k8s/src/main/java/org/apache/submarine/server/submitter/k8s/K8sSubmitter.java
+++ b/submarine-server/server-submitter/submitter-k8s/src/main/java/org/apache/submarine/server/submitter/k8s/K8sSubmitter.java
@@ -25,7 +25,6 @@ import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
-import java.util.Optional;
import java.util.Set;
import java.util.function.Function;
@@ -63,8 +62,10 @@ import org.apache.commons.lang3.StringUtils;
import org.apache.submarine.commons.utils.SubmarineConfVars;
import org.apache.submarine.commons.utils.SubmarineConfiguration;
import org.apache.submarine.commons.utils.exception.SubmarineRuntimeException;
+import org.apache.submarine.serve.istio.IstioHTTPRoute;
import org.apache.submarine.serve.istio.IstioVirtualService;
import org.apache.submarine.serve.istio.IstioVirtualServiceList;
+import org.apache.submarine.serve.istio.IstioVirtualServiceSpec;
import org.apache.submarine.serve.pytorch.SeldonPytorchServing;
import org.apache.submarine.serve.seldon.SeldonDeployment;
import org.apache.submarine.serve.seldon.SeldonDeploymentList;
@@ -77,7 +78,6 @@ import org.apache.submarine.server.api.common.CustomResourceType;
import org.apache.submarine.server.api.exception.InvalidSpecException;
import org.apache.submarine.server.api.experiment.Experiment;
import org.apache.submarine.server.api.experiment.ExperimentLog;
-import org.apache.submarine.server.api.experiment.Info;
import org.apache.submarine.server.api.experiment.MlflowInfo;
import org.apache.submarine.server.api.experiment.TensorboardInfo;
import org.apache.submarine.server.api.model.ServeSpec;
@@ -105,7 +105,6 @@ import org.apache.submarine.server.submitter.k8s.util.MLJobConverter;
import org.apache.submarine.server.submitter.k8s.util.NotebookUtils;
import org.apache.submarine.server.submitter.k8s.util.OwnerReferenceUtils;
-
import org.joda.time.DateTime;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -442,9 +441,8 @@ public class K8sSubmitter implements Submitter {
@Override
public TensorboardInfo getTensorboardInfo() throws SubmarineRuntimeException {
final String name = "submarine-tensorboard";
- final String ingressRouteName = "submarine-tensorboard-ingressroute";
try {
- return new TensorboardInfo(getInfo(name, ingressRouteName));
+ return new TensorboardInfo(isDeploymentAvailable(name));
} catch (ApiException e) {
throw new SubmarineRuntimeException(e.getCode(), e.getMessage());
}
@@ -453,34 +451,18 @@ public class K8sSubmitter implements Submitter {
@Override
public MlflowInfo getMlflowInfo() throws SubmarineRuntimeException {
final String name = "submarine-mlflow";
- final String ingressRouteName = "submarine-mlflow-ingressroute";
try {
- return new MlflowInfo(getInfo(name, ingressRouteName));
+ return new MlflowInfo(isDeploymentAvailable(name));
} catch (ApiException e) {
throw new SubmarineRuntimeException(e.getCode(), e.getMessage());
}
}
- public Info getInfo(String name, String ingressRouteName) throws ApiException {
+ private boolean isDeploymentAvailable(String name) throws ApiException{
V1Deployment deploy = appsV1Api.readNamespacedDeploymentStatus(name, getServerNamespace(), "true");
- boolean available = Optional.ofNullable(deploy.getStatus().getAvailableReplicas())
- .map(ar -> ar > 0).orElse(false); // at least one replica is running
-
- IngressRoute ingressRoute = new IngressRoute();
- V1ObjectMeta meta = new V1ObjectMeta();
- meta.setName(ingressRouteName);
- meta.setNamespace(getServerNamespace());
- ingressRoute.setMetadata(meta);
-
- IngressRoute result = ingressRouteClient.get(getServerNamespace(), ingressRouteName)
- .throwsApiException().getObject();
-
- String route = result.getSpec().getRoutes().stream().findFirst().get().getMatch();
-
- String url = route.replace("PathPrefix(`", "").replace("`)", "/");
-
- return new Info(available, url);
+ return deploy.getStatus().getAvailableReplicas() > 0; // at least one replica is running
}
+
@Override
public Notebook createNotebook(NotebookSpec spec, String notebookId) throws SubmarineRuntimeException {
Notebook notebook;
@@ -553,6 +535,7 @@ public class K8sSubmitter implements Submitter {
}
// create notebook Traefik custom resource
+ /*
try {
createIngressRoute(notebookCR.getMetadata().getNamespace(), notebookCR.getMetadata().getName());
} catch (ApiException e) {
@@ -564,6 +547,21 @@ public class K8sSubmitter implements Submitter {
throw new SubmarineRuntimeException(e.getCode(), "K8s submitter: ingressroute for Notebook " +
"object failed by " + e.getMessage());
}
+ */
+
+ // create notebook VirtualService custom resource
+ try {
+ createVirtualService(notebookCR.getMetadata().getNamespace(), notebookCR.getMetadata().getName());
+ } catch (ApiException e) {
+ LOG.error("K8s submitter: Create VirtualService for Notebook object failed by " +
+ e.getMessage(), e);
+ rollbackCreationNotebook(notebookCR, namespace);
+ if (needOverwrite) rollbackCreationConfigMap(namespace, configmap);
+ rollbackCreationPVC(namespace, workspacePvc, userPvc);
+ throw new SubmarineRuntimeException(e.getCode(), "K8s submitter: VirtualService for Notebook " +
+ "object failed by " + e.getMessage());
+ }
+
return notebook;
}
@@ -621,7 +619,10 @@ public class K8sSubmitter implements Submitter {
}
// delete ingress route
- deleteIngressRoute(namespace, name);
+ // deleteIngressRoute(namespace, name);
+
+ // delete VirtualService
+ deleteVirtualService(namespace, name);
// delete pvc
// workspace pvc
@@ -798,6 +799,46 @@ public class K8sSubmitter implements Submitter {
return spec;
}
+ private void createVirtualService(String namespace, String name) throws ApiException {
+ try {
+ V1ObjectMeta meta = new V1ObjectMeta();
+ meta.setName(name);
+ meta.setNamespace(namespace);
+ meta.setOwnerReferences(OwnerReferenceUtils.getOwnerReference());
+ IstioVirtualServiceSpec spec = parseVirtualServiceSpec(meta.getNamespace(), meta.getName());
+ IstioVirtualService virtualService = new IstioVirtualService(meta, spec);
+ istioVirtualServiceClient.create(namespace, virtualService, new CreateOptions()).throwsApiException();
+ } catch (ApiException e) {
+ LOG.error("K8s submitter: Create notebook VirtualService custom resource object failed by " +
+ e.getMessage(), e);
+ throw new SubmarineRuntimeException(e.getCode(), e.getMessage());
+ } catch (JsonSyntaxException e) {
+ LOG.error("K8s submitter: parse response object failed by " + e.getMessage(), e);
+ throw new SubmarineRuntimeException(500, "K8s Submitter parse upstream response failed.");
+ }
+ }
+
+ private void deleteVirtualService(String namespace, String name) {
+ try {
+ istioVirtualServiceClient.delete(namespace, name, getDeleteOptions(IstioConstants.API_VERSION))
+ .throwsApiException();
+ } catch (ApiException e) {
+ LOG.error("K8s submitter: Delete notebook VirtualService custom resource object failed by " +
+ e.getMessage(), e);
+ API_EXCEPTION_404_CONSUMER.apply(e);
+ }
+ }
+
+ private IstioVirtualServiceSpec parseVirtualServiceSpec(String namespace, String name) {
+ IstioVirtualServiceSpec spec = new IstioVirtualServiceSpec();
+ spec.addHost(IstioConstants.DEFAULT_INGRESS_HOST);
+ // TODO(operator): Do not hard code the gateway
+ spec.addGateway("submarine/submarine-gateway");
+ String matchURIPrefix = "/notebook/" + namespace + "/" + name;
+ spec.setHTTPRoute(new IstioHTTPRoute(matchURIPrefix, name, 80));
+ return spec;
+ }
+
private SeldonDeployment parseServeSpec(ServeSpec spec) throws SubmarineRuntimeException {
String modelName = spec.getModelName();
String modelType = spec.getModelType();
diff --git a/submarine-server/server-submitter/submitter-k8s/src/main/java/org/apache/submarine/server/submitter/k8s/model/AgentPod.java b/submarine-server/server-submitter/submitter-k8s/src/main/java/org/apache/submarine/server/submitter/k8s/model/AgentPod.java
index 765a5c72..0d47d699 100644
--- a/submarine-server/server-submitter/submitter-k8s/src/main/java/org/apache/submarine/server/submitter/k8s/model/AgentPod.java
+++ b/submarine-server/server-submitter/submitter-k8s/src/main/java/org/apache/submarine/server/submitter/k8s/model/AgentPod.java
@@ -20,7 +20,9 @@
package org.apache.submarine.server.submitter.k8s.model;
import java.util.ArrayList;
+import java.util.HashMap;
import java.util.List;
+import java.util.Map;
import org.apache.submarine.commons.utils.SubmarineConfiguration;
import org.apache.submarine.server.api.common.CustomResourceType;
@@ -40,9 +42,11 @@ public class AgentPod extends V1Pod{
String resourceId) {
super();
V1ObjectMeta meta = new V1ObjectMeta();
-
+ Map<String, String> labels = new HashMap<>();
+ labels.put("app", type.toString().toLowerCase());
meta.setName(getNormalizePodName(type, name, resourceId));
meta.setNamespace(namespace);
+ meta.setLabels(labels);
this.setMetadata(meta);
V1PodSpec spec = new V1PodSpec();
diff --git a/website/docs/devDocs/Development.md b/website/docs/devDocs/Development.md
index 54b63cb4..7d810685 100644
--- a/website/docs/devDocs/Development.md
+++ b/website/docs/devDocs/Development.md
@@ -98,7 +98,7 @@ Checkstyle plugin may help to detect violations directly from the IDE.
3. Delete the server deployment and the operator will create a new one using the new image
```bash
- kubectl delete deployment submarine-server
+ kubectl delete deployment submarine-server -n submarine-user-test
```
## Develop workbench
diff --git a/website/docs/gettingStarted/quickstart.md b/website/docs/gettingStarted/quickstart.md
index b1ffc4d7..44d1bd27 100644
--- a/website/docs/gettingStarted/quickstart.md
+++ b/website/docs/gettingStarted/quickstart.md
@@ -33,66 +33,77 @@ This document gives you a quick view on the basic usage of Submarine platform. Y
- [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
- [helm](https://helm.sh/docs/intro/install/) (Helm v3 is minimum requirement.)
- [minikube](https://minikube.sigs.k8s.io/docs/start/).
+- [istioctl](https://istio.io/latest/docs/setup/getting-started/#download)
+
+2. Start minikube cluster and install Istio
-2. Start minikube cluster
```
-minikube start --vm-driver=docker --cpus 8 --memory 4096 --kubernetes-version v1.21.2
+minikube start --vm-driver=docker --cpus 8 --memory 8192 --kubernetes-version v1.21.2
+istioctl install -y
+# Or if you want to support Pod Security Policy (https://minikube.sigs.k8s.io/docs/tutorials/using_psp), you can use the following command to start cluster
+minikube start --extra-config=apiserver.enable-admission-plugins=PodSecurityPolicy --addons=pod-security-policy --vm-driver=docker --cpus 8 --memory 4096 --kubernetes-version v1.21.2
```
### Launch submarine in the cluster
1. Clone the project
+
```
git clone https://github.com/apache/submarine.git
+cd submarine
```
-2. Install the submarine operator and dependencies by helm chart
+2. Create necessary namespaces
+
```
-cd submarine
-helm install submarine ./helm-charts/submarine
+kubectl create namespace submarine
+kubectl create namespace submarine-user-test
+kubectl label namespace submarine istio-injection=enabled
+kubectl label namespace submarine-user-test istio-injection=enabled
```
-3. Create a Submarine custom resource and the operator will create the submarine server, database, etc. for us.
+3. Install the submarine operator and dependencies by helm chart
+
```
-kubectl apply -f submarine-cloud-v2/artifacts/examples/example-submarine.yaml
+helm install submarine ./helm-charts/submarine -n submarine
```
-### Ensure submarine is ready
+4. Create a Submarine custom resource and the operator will create the submarine server, database, etc. for us.
-1. Use kubectl to query the status of pods
```
-kubectl get pods
+kubectl apply -f submarine-cloud-v2/artifacts/examples/example-submarine.yaml -n submarine-user-test
```
-2. Make sure each pod is `Running`
+### Ensure submarine is ready
+
```
+$ kubectl get pods -n submarine
NAME READY STATUS RESTARTS AGE
-notebook-controller-deployment-5d4f5f874c-mnbc8 1/1 Running 0 61m
-pytorch-operator-844c866d54-xm8nl 1/1 Running 2 61m
-submarine-database-85bd68dbc5-qggtm 1/1 Running 0 11m
-submarine-minio-76465444f6-hdgdp 1/1 Running 0 11m
-submarine-mlflow-75f86d8f4d-rj2z7 1/1 Running 0 11m
-submarine-operator-5dd79cdf86-gpm2p 1/1 Running 0 61m
-submarine-server-68985b767-vjdvx 1/1 Running 0 11m
-submarine-tensorboard-5df8499fd4-vnklf 1/1 Running 0 11m
-submarine-traefik-7cbcfd4bd9-wbf8b 1/1 Running 0 61m
-tf-job-operator-6bb69fd44-zmlmr 1/1 Running 1 61m
+notebook-controller-deployment-66d85984bf-x562z 1/1 Running 0 7h7m
+pytorch-operator-7d778f4859-g7xph 2/2 Running 0 7h7m
+tf-job-operator-7d895bf77c-75n72 2/2 Running 0 7h7m
+
+$ kubectl get pods -n submarine-user-test
+NAME READY STATUS RESTARTS AGE
+submarine-database-bdcb77549-rq2ds 2/2 Running 0 7h6m
+submarine-minio-686b8777ff-zg4d2 2/2 Running 0 7h6m
+submarine-mlflow-68c5559dcb-lkq4g 2/2 Running 0 7h6m
+submarine-server-7c6d7bcfd8-5p42w 2/2 Running 0 9m33s
+submarine-tensorboard-57c5b64778-t4lww 2/2 Running 0 7h6m
```
### Connect to workbench
1. Exposing service
- ```
- # Method 1 -- use minikube ip
- minikube ip # you'll get the IP address of minikube, ex: 192.168.49.2
- # Method 2 -- use port-forwarding
- kubectl port-forward --address 0.0.0.0 service/submarine-traefik 32080:80
- ```
+```
+kubectl port-forward --address 0.0.0.0 -n istio-system service/istio-ingressgateway 32080:80
+```
2. View workbench
- If you use method 1, go to `http://{minikube ip}:32080`. For example, `http://192.168.49.2:32080`. If you use method 2, go to `http://0.0.0.0:32080`.
- 
+
+Go to `http://0.0.0.0:32080`
+
## Example: Submit a mnist distributed example
@@ -177,6 +188,7 @@ if __name__ == '__main__':
```
### 2. Prepare an environment compatible with the training
+
Build a docker image equipped with the requirement of the environment.
```bash
@@ -190,14 +202,14 @@ eval $(minikube docker-env)
2. Choose `Define your experiment`
3. Fill the form accordingly. Here we set 3 workers.
- 1. Step 1
- 
- 2. Step 2
- 
- 3. Step 3
- 
- 4. The experiment is successfully submitted
- 
+ 1. Step 1
+ 
+ 2. Step 2
+ 
+ 3. Step 3
+ 
+ 4. The experiment is successfully submitted
+ 
### 4. Monitor the process
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@submarine.apache.org
For additional commands, e-mail: dev-help@submarine.apache.org