You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@openwhisk.apache.org by GitBox <gi...@apache.org> on 2022/05/06 05:43:11 UTC
[GitHub] [openwhisk-deploy-kube] hunhoffe opened a new pull request, #729: Proof-of-concept for running new scheduler
hunhoffe opened a new pull request, #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729
I have been working on adapting the openwhisk-deploy-kube repo to support deploying the new scheduler. I am creating this PR even though it is just a minimal working example to get some feedback (hopefully) on my approach, and also to make the solution public since I believe there was some community interest in this capability. It is not ready to be merged.
What this PR contains:
- Deploy etcd and the scheduler components
- Allow option to enable or disable the scheduler (similar to ansible)
- Only works with single controller and single scheduler
- Tested with and without scheduler enabled on 3-node Kubernetes cluster with CouchDB activation store
Remaining TODOs:
- [ ] Use Kubernetes Akka cluster bootstrap to support multiple schedulers and/or controllers (this will involve changes in this repo and also the code in the main OpenWhisk repo. My plan is to model the solution similar to how the Akka cluster was bootstrapped in the ShardingContainerPoolBalancer)
- [ ] More rigorous testing (multiple invokers, ElasticSearch as activation store, etc.)
- [ ] Create openwhisk/scheduler image on dockerhub (currently using my own docker image)
- [ ] etcd will currently always start fresh; I think it would be appropriate to enable persistence?
- [ ] Double check components are starting up the right order (right now I don't have a waitForEtcd container, and I'm not 100% sure I'm starting waitForScheduler in the most proper place)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] hunhoffe commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
hunhoffe commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r970251615
##########
helm/openwhisk/values.yaml:
##########
@@ -249,20 +255,69 @@ nginx:
# Controller configurations
controller:
imageName: "openwhisk/controller"
- imageTag: "c5970a6"
+ imageTag: "a1639f0"
+ imagePullPolicy: "IfNotPresent"
+ replicaCount: 1
+ restartPolicy: "Always"
+ port: 8080
+ options: ""
+ jvmHeapMB: "1024"
+ jvmOptions: ""
+ loglevel: "INFO"
+
+# Scheduler configurations
+scheduler:
+ enable: true
Review Comment:
Do we want this to default to true or not? I set it to true here so that the travis tests would run by default with the scheduler, but perhaps the correct thing to do is run travis tests both with and without the scheduler?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] style95 commented on pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
style95 commented on PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#issuecomment-1254594376
> I could create an issue for it so it is not forgotten if that's acceptable.
I am ok with this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] hunhoffe commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
hunhoffe commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r918524619
##########
helm/openwhisk/values.yaml:
##########
@@ -248,8 +254,8 @@ nginx:
# Controller configurations
controller:
- imageName: "openwhisk/controller"
- imageTag: "c5970a6"
+ imageName: "hunhoffe/controller"
Review Comment:
TODO: need to update this once apache/openwhisk has accepted changes needed for controller to replicate properly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] hunhoffe commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
hunhoffe commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r977058484
##########
helm/openwhisk/values.yaml:
##########
@@ -139,9 +139,9 @@ whisk:
includeSystemTests: false
versions:
openwhisk:
- buildDate: "2022-06-23-06:18:00Z"
Review Comment:
I made the details of this timestamp up. Am I supposed to have copied it from somewhere?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] hunhoffe commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
hunhoffe commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r972478031
##########
helm/openwhisk/templates/scheduler-pod.yaml:
##########
@@ -0,0 +1,273 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+{{- if .Values.scheduler.enable }}
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: {{ .Release.Name }}-scheduler
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 4 }}
+spec:
+ serviceName: {{ .Release.Name }}-scheduler
+ podManagementPolicy: "Parallel"
+ replicas: {{ .Values.scheduler.replicaCount }}
+ selector:
+ matchLabels:
+ name: {{ .Release.Name }}-scheduler
+ template:
+ metadata:
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 8 }}
+
+ {{- if .Values.metrics.prometheusEnabled }}
+ annotations:
+ prometheus.io/scrape: 'true'
+ prometheus.io/port: '{{ .Values.scheduler.port }}'
+ {{- end }}
+
+ spec:
+ serviceAccountName: {{ .Release.Name }}-core
+ restartPolicy: {{ .Values.scheduler.restartPolicy }}
+
+ {{- if .Values.affinity.enabled }}
+ affinity:
+{{ include "openwhisk.affinity.core" . | indent 8 }}
+{{ include "openwhisk.affinity.selfAntiAffinity" ( printf "%s-scheduler" .Release.Name ) | indent 8 }}
+ {{- end }}
+
+ {{- if .Values.toleration.enabled }}
+ tolerations:
+{{ include "openwhisk.toleration.core" . | indent 8 }}
+ {{- end }}
+
+ initContainers:
+ # The scheduler must wait for kafka and/or couchdb to be ready before it starts
+{{ include "openwhisk.readiness.waitForController" . | indent 6 }}
+{{ include "openwhisk.docker.imagePullSecrets" . | indent 6 }}
+
+ containers:
+ - name: scheduler
+ imagePullPolicy: {{ .Values.scheduler.imagePullPolicy | quote }}
+ image: "{{- .Values.docker.registry.name -}}{{- .Values.scheduler.imageName -}}:{{- .Values.scheduler.imageTag -}}"
+ command: ["/bin/bash", "-c", "/init.sh `hostname | awk -F '-' '{print $NF}'`"]
+ ports:
+ - name: scheduler
+ containerPort: {{ .Values.scheduler.port }}
+ - name: rpc
+ containerPort: {{ .Values.scheduler.rpcPort }}
+ - name: akka-remoting
+ containerPort: 25520
+ - name: akka-mgmt-http
+ containerPort: 19999
+ livenessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.livenessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.livenessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.livenessProbe.timeoutSeconds }}
+ readinessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.readinessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.readinessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.readinessProbe.timeoutSeconds }}
+ env:
Review Comment:
Regarding ```whisk_cluster_name```: I believe you name a deployment when you create it with helm (which becomes ```Release.name``` in the helm template files), which would allow you to deploy multiple deployments on the same kubernetes without conflict (I am speaking hypothetically, as I have not tried this myself). If I am understanding correctly, perhaps ```whisk_cluster_name``` is not needed here. @dgrove-oss does this seem correct?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] hunhoffe commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
hunhoffe commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r918524619
##########
helm/openwhisk/values.yaml:
##########
@@ -248,8 +254,8 @@ nginx:
# Controller configurations
controller:
- imageName: "openwhisk/controller"
- imageTag: "c5970a6"
+ imageName: "hunhoffe/controller"
Review Comment:
TODO: need to update this once apache/openwhisk has accepted changes needed for controller to replicate properly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] hunhoffe commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
hunhoffe commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r918524790
##########
helm/openwhisk/values.yaml:
##########
@@ -259,6 +265,54 @@ controller:
jvmOptions: ""
loglevel: "INFO"
+# Scheduler configurations
+scheduler:
+ enable: true
+ imageName: "hunhoffe/scheduler"
Review Comment:
TODO: need to update this once apache/openwhisk has accepted changes needed for scheduler to replicate properly, and also once scheduler images are pushed to dockerhub.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] dgrove-oss commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
dgrove-oss commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r971446437
##########
helm/openwhisk/values.yaml:
##########
@@ -249,20 +255,69 @@ nginx:
# Controller configurations
controller:
imageName: "openwhisk/controller"
- imageTag: "c5970a6"
+ imageTag: "a1639f0"
+ imagePullPolicy: "IfNotPresent"
+ replicaCount: 1
+ restartPolicy: "Always"
+ port: 8080
+ options: ""
+ jvmHeapMB: "1024"
+ jvmOptions: ""
+ loglevel: "INFO"
+
+# Scheduler configurations
+scheduler:
+ enable: true
Review Comment:
It would be ideal to have some entries in the travis matrix that run with the new scheduler and some that run with the old scheduler.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] style95 commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
style95 commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r977278443
##########
helm/openwhisk/values.yaml:
##########
@@ -139,9 +139,9 @@ whisk:
includeSystemTests: false
versions:
openwhisk:
- buildDate: "2022-06-23-06:18:00Z"
Review Comment:
I believe @dgrove-oss has an idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] hunhoffe commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
hunhoffe commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r920066240
##########
tools/travis/deploy-chart.sh:
##########
@@ -216,12 +216,17 @@ if [ "${OW_LEAN_MODE:-false}" == "false" ]; then
# Wait for the controller to confirm that it has at least one healthy invoker
verifyHealthyInvoker
+ # Wait for scheduler to be up
+ statefulsetHealthCheck "ow4travis-scheduler"
Review Comment:
Is there are an additional, more rigorous check to do than just this? e.g., something analogous to the invoker health check but for the scheduler.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] hunhoffe commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
hunhoffe commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r918524790
##########
helm/openwhisk/values.yaml:
##########
@@ -259,6 +265,54 @@ controller:
jvmOptions: ""
loglevel: "INFO"
+# Scheduler configurations
+scheduler:
+ enable: true
+ imageName: "hunhoffe/scheduler"
Review Comment:
TODO: need to update this once apache/openwhisk has accepted changes needed for scheduler to replicate properly, and also once scheduler images are pushed to dockerhub.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] hunhoffe commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
hunhoffe commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r973326256
##########
helm/openwhisk/templates/scheduler-pod.yaml:
##########
@@ -0,0 +1,273 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+{{- if .Values.scheduler.enable }}
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: {{ .Release.Name }}-scheduler
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 4 }}
+spec:
+ serviceName: {{ .Release.Name }}-scheduler
+ podManagementPolicy: "Parallel"
+ replicas: {{ .Values.scheduler.replicaCount }}
+ selector:
+ matchLabels:
+ name: {{ .Release.Name }}-scheduler
+ template:
+ metadata:
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 8 }}
+
+ {{- if .Values.metrics.prometheusEnabled }}
+ annotations:
+ prometheus.io/scrape: 'true'
+ prometheus.io/port: '{{ .Values.scheduler.port }}'
+ {{- end }}
+
+ spec:
+ serviceAccountName: {{ .Release.Name }}-core
+ restartPolicy: {{ .Values.scheduler.restartPolicy }}
+
+ {{- if .Values.affinity.enabled }}
+ affinity:
+{{ include "openwhisk.affinity.core" . | indent 8 }}
+{{ include "openwhisk.affinity.selfAntiAffinity" ( printf "%s-scheduler" .Release.Name ) | indent 8 }}
+ {{- end }}
+
+ {{- if .Values.toleration.enabled }}
+ tolerations:
+{{ include "openwhisk.toleration.core" . | indent 8 }}
+ {{- end }}
+
+ initContainers:
+ # The scheduler must wait for kafka and/or couchdb to be ready before it starts
+{{ include "openwhisk.readiness.waitForController" . | indent 6 }}
+{{ include "openwhisk.docker.imagePullSecrets" . | indent 6 }}
+
+ containers:
+ - name: scheduler
+ imagePullPolicy: {{ .Values.scheduler.imagePullPolicy | quote }}
+ image: "{{- .Values.docker.registry.name -}}{{- .Values.scheduler.imageName -}}:{{- .Values.scheduler.imageTag -}}"
+ command: ["/bin/bash", "-c", "/init.sh `hostname | awk -F '-' '{print $NF}'`"]
+ ports:
+ - name: scheduler
+ containerPort: {{ .Values.scheduler.port }}
+ - name: rpc
+ containerPort: {{ .Values.scheduler.rpcPort }}
+ - name: akka-remoting
+ containerPort: 25520
+ - name: akka-mgmt-http
+ containerPort: 19999
+ livenessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.livenessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.livenessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.livenessProbe.timeoutSeconds }}
+ readinessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.readinessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.readinessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.readinessProbe.timeoutSeconds }}
+ env:
Review Comment:
I have some confusion with how ```whisk cluster name``` is implemented in the main OpenWhisk repo, and I'd appreciate your help in understanding why it is the way it is.
In the link you pasted it is ```CONFIG_whisk_cluster_name```, but in config hierarchy it is ```whisk.cluster_name```. My understanding is that it should be ```CONFIG_whisk_clusterName``` to follow convention.
To my understanding it is etcd specific, so shouldn't it be under ```etcd.clusterName``` and not in ```whisk.clusterName```? I believe elasticsearch config takes this approach here: https://github.com/apache/openwhisk/blob/a1639f0e4d7270c9a230190ac26acb61413b6bbb/ansible/roles/elasticsearch/templates/elasticsearch.yml.j2#L1
I am asking these questions because I don't want to modify the main whisk config for scheduler-specific things if I don't have to. If there is a reason for it to be the way it is currently in OpenWhisk, I can still add it to this PR. But I wanted some clarification first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] style95 commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
style95 commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r977253705
##########
helm/openwhisk/templates/scheduler-pod.yaml:
##########
@@ -0,0 +1,273 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+{{- if .Values.scheduler.enable }}
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: {{ .Release.Name }}-scheduler
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 4 }}
+spec:
+ serviceName: {{ .Release.Name }}-scheduler
+ podManagementPolicy: "Parallel"
+ replicas: {{ .Values.scheduler.replicaCount }}
+ selector:
+ matchLabels:
+ name: {{ .Release.Name }}-scheduler
+ template:
+ metadata:
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 8 }}
+
+ {{- if .Values.metrics.prometheusEnabled }}
+ annotations:
+ prometheus.io/scrape: 'true'
+ prometheus.io/port: '{{ .Values.scheduler.port }}'
+ {{- end }}
+
+ spec:
+ serviceAccountName: {{ .Release.Name }}-core
+ restartPolicy: {{ .Values.scheduler.restartPolicy }}
+
+ {{- if .Values.affinity.enabled }}
+ affinity:
+{{ include "openwhisk.affinity.core" . | indent 8 }}
+{{ include "openwhisk.affinity.selfAntiAffinity" ( printf "%s-scheduler" .Release.Name ) | indent 8 }}
+ {{- end }}
+
+ {{- if .Values.toleration.enabled }}
+ tolerations:
+{{ include "openwhisk.toleration.core" . | indent 8 }}
+ {{- end }}
+
+ initContainers:
+ # The scheduler must wait for kafka and/or couchdb to be ready before it starts
+{{ include "openwhisk.readiness.waitForController" . | indent 6 }}
+{{ include "openwhisk.docker.imagePullSecrets" . | indent 6 }}
+
+ containers:
+ - name: scheduler
+ imagePullPolicy: {{ .Values.scheduler.imagePullPolicy | quote }}
+ image: "{{- .Values.docker.registry.name -}}{{- .Values.scheduler.imageName -}}:{{- .Values.scheduler.imageTag -}}"
+ command: ["/bin/bash", "-c", "/init.sh `hostname | awk -F '-' '{print $NF}'`"]
+ ports:
+ - name: scheduler
+ containerPort: {{ .Values.scheduler.port }}
+ - name: rpc
+ containerPort: {{ .Values.scheduler.rpcPort }}
+ - name: akka-remoting
+ containerPort: 25520
+ - name: akka-mgmt-http
+ containerPort: 19999
+ livenessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.livenessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.livenessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.livenessProbe.timeoutSeconds }}
+ readinessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.readinessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.readinessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.readinessProbe.timeoutSeconds }}
+ env:
+ - name: "PORT"
+ value: {{ .Values.scheduler.port | quote }}
+
+ - name: "POD_IP"
+ valueFrom:
+ fieldRef:
+ fieldPath: status.podIP
+
+ - name: "WHISK_SCHEDULER_ENDPOINTS_RPCPORT"
+ value: {{ .Values.scheduler.rpcPort | quote }}
+
+ - name: "WHISK_SCHEDULER_ENDPOINTS_AKKAPORT"
+ value: "25520"
Review Comment:
Seems this is still not configurable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] style95 commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
style95 commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r973816585
##########
helm/openwhisk/templates/scheduler-pod.yaml:
##########
@@ -0,0 +1,273 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+{{- if .Values.scheduler.enable }}
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: {{ .Release.Name }}-scheduler
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 4 }}
+spec:
+ serviceName: {{ .Release.Name }}-scheduler
+ podManagementPolicy: "Parallel"
+ replicas: {{ .Values.scheduler.replicaCount }}
+ selector:
+ matchLabels:
+ name: {{ .Release.Name }}-scheduler
+ template:
+ metadata:
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 8 }}
+
+ {{- if .Values.metrics.prometheusEnabled }}
+ annotations:
+ prometheus.io/scrape: 'true'
+ prometheus.io/port: '{{ .Values.scheduler.port }}'
+ {{- end }}
+
+ spec:
+ serviceAccountName: {{ .Release.Name }}-core
+ restartPolicy: {{ .Values.scheduler.restartPolicy }}
+
+ {{- if .Values.affinity.enabled }}
+ affinity:
+{{ include "openwhisk.affinity.core" . | indent 8 }}
+{{ include "openwhisk.affinity.selfAntiAffinity" ( printf "%s-scheduler" .Release.Name ) | indent 8 }}
+ {{- end }}
+
+ {{- if .Values.toleration.enabled }}
+ tolerations:
+{{ include "openwhisk.toleration.core" . | indent 8 }}
+ {{- end }}
+
+ initContainers:
+ # The scheduler must wait for kafka and/or couchdb to be ready before it starts
+{{ include "openwhisk.readiness.waitForController" . | indent 6 }}
+{{ include "openwhisk.docker.imagePullSecrets" . | indent 6 }}
+
+ containers:
+ - name: scheduler
+ imagePullPolicy: {{ .Values.scheduler.imagePullPolicy | quote }}
+ image: "{{- .Values.docker.registry.name -}}{{- .Values.scheduler.imageName -}}:{{- .Values.scheduler.imageTag -}}"
+ command: ["/bin/bash", "-c", "/init.sh `hostname | awk -F '-' '{print $NF}'`"]
+ ports:
+ - name: scheduler
+ containerPort: {{ .Values.scheduler.port }}
+ - name: rpc
+ containerPort: {{ .Values.scheduler.rpcPort }}
+ - name: akka-remoting
+ containerPort: 25520
+ - name: akka-mgmt-http
+ containerPort: 19999
+ livenessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.livenessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.livenessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.livenessProbe.timeoutSeconds }}
+ readinessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.readinessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.readinessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.readinessProbe.timeoutSeconds }}
+ env:
Review Comment:
It is not only being used by schedulers but also by controllers too.
https://github.com/apache/openwhisk/blob/1fbfd6bcb3d65845a328382641c1db7182763ced/core/controller/src/main/scala/org/apache/openwhisk/core/loadBalancer/FPCPoolBalancer.scala#L72
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] style95 commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
style95 commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r971917119
##########
helm/openwhisk/templates/scheduler-pod.yaml:
##########
@@ -0,0 +1,273 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+{{- if .Values.scheduler.enable }}
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: {{ .Release.Name }}-scheduler
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 4 }}
+spec:
+ serviceName: {{ .Release.Name }}-scheduler
+ podManagementPolicy: "Parallel"
+ replicas: {{ .Values.scheduler.replicaCount }}
+ selector:
+ matchLabels:
+ name: {{ .Release.Name }}-scheduler
+ template:
+ metadata:
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 8 }}
+
+ {{- if .Values.metrics.prometheusEnabled }}
+ annotations:
+ prometheus.io/scrape: 'true'
+ prometheus.io/port: '{{ .Values.scheduler.port }}'
+ {{- end }}
+
+ spec:
+ serviceAccountName: {{ .Release.Name }}-core
+ restartPolicy: {{ .Values.scheduler.restartPolicy }}
+
+ {{- if .Values.affinity.enabled }}
+ affinity:
+{{ include "openwhisk.affinity.core" . | indent 8 }}
+{{ include "openwhisk.affinity.selfAntiAffinity" ( printf "%s-scheduler" .Release.Name ) | indent 8 }}
+ {{- end }}
+
+ {{- if .Values.toleration.enabled }}
+ tolerations:
+{{ include "openwhisk.toleration.core" . | indent 8 }}
+ {{- end }}
+
+ initContainers:
+ # The scheduler must wait for kafka and/or couchdb to be ready before it starts
+{{ include "openwhisk.readiness.waitForController" . | indent 6 }}
+{{ include "openwhisk.docker.imagePullSecrets" . | indent 6 }}
+
+ containers:
+ - name: scheduler
+ imagePullPolicy: {{ .Values.scheduler.imagePullPolicy | quote }}
+ image: "{{- .Values.docker.registry.name -}}{{- .Values.scheduler.imageName -}}:{{- .Values.scheduler.imageTag -}}"
+ command: ["/bin/bash", "-c", "/init.sh `hostname | awk -F '-' '{print $NF}'`"]
+ ports:
+ - name: scheduler
+ containerPort: {{ .Values.scheduler.port }}
+ - name: rpc
+ containerPort: {{ .Values.scheduler.rpcPort }}
+ - name: akka-remoting
+ containerPort: 25520
+ - name: akka-mgmt-http
+ containerPort: 19999
+ livenessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.livenessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.livenessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.livenessProbe.timeoutSeconds }}
+ readinessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.readinessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.readinessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.readinessProbe.timeoutSeconds }}
+ env:
Review Comment:
I think the `whick_cluster_name` needs to be added to differentiate different versions of clusters.
https://github.com/apache/openwhisk/blob/master/ansible/roles/schedulers/tasks/deploy.yml#L223
##########
docs/configurationChoices.md:
##########
@@ -180,6 +190,8 @@ k8s:
enabled: false
```
+Currently, etcd persistence is not supported.
Review Comment:
👍
It is recommended to run ETCD on physical machines with SSD.
##########
helm/openwhisk/templates/scheduler-pod.yaml:
##########
@@ -0,0 +1,273 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+{{- if .Values.scheduler.enable }}
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: {{ .Release.Name }}-scheduler
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 4 }}
+spec:
+ serviceName: {{ .Release.Name }}-scheduler
+ podManagementPolicy: "Parallel"
+ replicas: {{ .Values.scheduler.replicaCount }}
+ selector:
+ matchLabels:
+ name: {{ .Release.Name }}-scheduler
+ template:
+ metadata:
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 8 }}
+
+ {{- if .Values.metrics.prometheusEnabled }}
+ annotations:
+ prometheus.io/scrape: 'true'
+ prometheus.io/port: '{{ .Values.scheduler.port }}'
+ {{- end }}
+
+ spec:
+ serviceAccountName: {{ .Release.Name }}-core
+ restartPolicy: {{ .Values.scheduler.restartPolicy }}
+
+ {{- if .Values.affinity.enabled }}
+ affinity:
+{{ include "openwhisk.affinity.core" . | indent 8 }}
+{{ include "openwhisk.affinity.selfAntiAffinity" ( printf "%s-scheduler" .Release.Name ) | indent 8 }}
+ {{- end }}
+
+ {{- if .Values.toleration.enabled }}
+ tolerations:
+{{ include "openwhisk.toleration.core" . | indent 8 }}
+ {{- end }}
+
+ initContainers:
+ # The scheduler must wait for kafka and/or couchdb to be ready before it starts
+{{ include "openwhisk.readiness.waitForController" . | indent 6 }}
Review Comment:
Should this be `waitForKafka` and `waitForCouchDB`?
##########
helm/openwhisk/templates/scheduler-pod.yaml:
##########
@@ -0,0 +1,273 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+{{- if .Values.scheduler.enable }}
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: {{ .Release.Name }}-scheduler
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 4 }}
+spec:
+ serviceName: {{ .Release.Name }}-scheduler
+ podManagementPolicy: "Parallel"
+ replicas: {{ .Values.scheduler.replicaCount }}
+ selector:
+ matchLabels:
+ name: {{ .Release.Name }}-scheduler
+ template:
+ metadata:
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 8 }}
+
+ {{- if .Values.metrics.prometheusEnabled }}
+ annotations:
+ prometheus.io/scrape: 'true'
+ prometheus.io/port: '{{ .Values.scheduler.port }}'
+ {{- end }}
+
+ spec:
+ serviceAccountName: {{ .Release.Name }}-core
+ restartPolicy: {{ .Values.scheduler.restartPolicy }}
+
+ {{- if .Values.affinity.enabled }}
+ affinity:
+{{ include "openwhisk.affinity.core" . | indent 8 }}
+{{ include "openwhisk.affinity.selfAntiAffinity" ( printf "%s-scheduler" .Release.Name ) | indent 8 }}
+ {{- end }}
+
+ {{- if .Values.toleration.enabled }}
+ tolerations:
+{{ include "openwhisk.toleration.core" . | indent 8 }}
+ {{- end }}
+
+ initContainers:
+ # The scheduler must wait for kafka and/or couchdb to be ready before it starts
+{{ include "openwhisk.readiness.waitForController" . | indent 6 }}
+{{ include "openwhisk.docker.imagePullSecrets" . | indent 6 }}
+
+ containers:
+ - name: scheduler
+ imagePullPolicy: {{ .Values.scheduler.imagePullPolicy | quote }}
+ image: "{{- .Values.docker.registry.name -}}{{- .Values.scheduler.imageName -}}:{{- .Values.scheduler.imageTag -}}"
+ command: ["/bin/bash", "-c", "/init.sh `hostname | awk -F '-' '{print $NF}'`"]
+ ports:
+ - name: scheduler
+ containerPort: {{ .Values.scheduler.port }}
+ - name: rpc
+ containerPort: {{ .Values.scheduler.rpcPort }}
+ - name: akka-remoting
+ containerPort: 25520
+ - name: akka-mgmt-http
+ containerPort: 19999
+ livenessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.livenessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.livenessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.livenessProbe.timeoutSeconds }}
+ readinessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.readinessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.readinessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.readinessProbe.timeoutSeconds }}
+ env:
+ - name: "PORT"
+ value: {{ .Values.scheduler.port | quote }}
+
+ - name: "POD_IP"
+ valueFrom:
+ fieldRef:
+ fieldPath: status.podIP
+
+ - name: "WHISK_SCHEDULER_ENDPOINTS_RPCPORT"
+ value: {{ .Values.scheduler.rpcPort | quote }}
+
+ - name: "WHISK_SCHEDULER_ENDPOINTS_AKKAPORT"
+ value: "25520"
Review Comment:
Would be great to make this configurable.
##########
helm/openwhisk/templates/scheduler-pod.yaml:
##########
@@ -0,0 +1,273 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+{{- if .Values.scheduler.enable }}
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: {{ .Release.Name }}-scheduler
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 4 }}
+spec:
+ serviceName: {{ .Release.Name }}-scheduler
+ podManagementPolicy: "Parallel"
+ replicas: {{ .Values.scheduler.replicaCount }}
+ selector:
+ matchLabels:
+ name: {{ .Release.Name }}-scheduler
+ template:
+ metadata:
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 8 }}
+
+ {{- if .Values.metrics.prometheusEnabled }}
+ annotations:
+ prometheus.io/scrape: 'true'
+ prometheus.io/port: '{{ .Values.scheduler.port }}'
+ {{- end }}
+
+ spec:
+ serviceAccountName: {{ .Release.Name }}-core
+ restartPolicy: {{ .Values.scheduler.restartPolicy }}
+
+ {{- if .Values.affinity.enabled }}
+ affinity:
+{{ include "openwhisk.affinity.core" . | indent 8 }}
+{{ include "openwhisk.affinity.selfAntiAffinity" ( printf "%s-scheduler" .Release.Name ) | indent 8 }}
+ {{- end }}
+
+ {{- if .Values.toleration.enabled }}
+ tolerations:
+{{ include "openwhisk.toleration.core" . | indent 8 }}
+ {{- end }}
+
+ initContainers:
+ # The scheduler must wait for kafka and/or couchdb to be ready before it starts
+{{ include "openwhisk.readiness.waitForController" . | indent 6 }}
+{{ include "openwhisk.docker.imagePullSecrets" . | indent 6 }}
+
+ containers:
+ - name: scheduler
+ imagePullPolicy: {{ .Values.scheduler.imagePullPolicy | quote }}
+ image: "{{- .Values.docker.registry.name -}}{{- .Values.scheduler.imageName -}}:{{- .Values.scheduler.imageTag -}}"
+ command: ["/bin/bash", "-c", "/init.sh `hostname | awk -F '-' '{print $NF}'`"]
+ ports:
+ - name: scheduler
+ containerPort: {{ .Values.scheduler.port }}
+ - name: rpc
+ containerPort: {{ .Values.scheduler.rpcPort }}
+ - name: akka-remoting
+ containerPort: 25520
+ - name: akka-mgmt-http
+ containerPort: 19999
+ livenessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.livenessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.livenessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.livenessProbe.timeoutSeconds }}
+ readinessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.readinessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.readinessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.readinessProbe.timeoutSeconds }}
+ env:
Review Comment:
The `scheduling` configurations are required.
```
"CONFIG_whisk_scheduler_scheduling_staleThreshold": "{{ scheduler.scheduling.staleThreshold }}"
"CONFIG_whisk_scheduler_scheduling_checkInterval": "{{ scheduler.scheduling.checkInterval }}"
"CONFIG_whisk_scheduler_scheduling_dropInterval": "{{ scheduler.scheduling.dropInterval }}"
```
https://github.com/apache/openwhisk/blob/master/ansible/roles/schedulers/tasks/deploy.yml#L117
##########
helm/openwhisk/templates/scheduler-pod.yaml:
##########
@@ -0,0 +1,273 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+{{- if .Values.scheduler.enable }}
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: {{ .Release.Name }}-scheduler
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 4 }}
+spec:
+ serviceName: {{ .Release.Name }}-scheduler
+ podManagementPolicy: "Parallel"
+ replicas: {{ .Values.scheduler.replicaCount }}
+ selector:
+ matchLabels:
+ name: {{ .Release.Name }}-scheduler
+ template:
+ metadata:
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 8 }}
+
+ {{- if .Values.metrics.prometheusEnabled }}
+ annotations:
+ prometheus.io/scrape: 'true'
+ prometheus.io/port: '{{ .Values.scheduler.port }}'
+ {{- end }}
+
+ spec:
+ serviceAccountName: {{ .Release.Name }}-core
+ restartPolicy: {{ .Values.scheduler.restartPolicy }}
+
+ {{- if .Values.affinity.enabled }}
+ affinity:
+{{ include "openwhisk.affinity.core" . | indent 8 }}
+{{ include "openwhisk.affinity.selfAntiAffinity" ( printf "%s-scheduler" .Release.Name ) | indent 8 }}
+ {{- end }}
+
+ {{- if .Values.toleration.enabled }}
+ tolerations:
+{{ include "openwhisk.toleration.core" . | indent 8 }}
+ {{- end }}
+
+ initContainers:
+ # The scheduler must wait for kafka and/or couchdb to be ready before it starts
+{{ include "openwhisk.readiness.waitForController" . | indent 6 }}
+{{ include "openwhisk.docker.imagePullSecrets" . | indent 6 }}
+
+ containers:
+ - name: scheduler
+ imagePullPolicy: {{ .Values.scheduler.imagePullPolicy | quote }}
+ image: "{{- .Values.docker.registry.name -}}{{- .Values.scheduler.imageName -}}:{{- .Values.scheduler.imageTag -}}"
+ command: ["/bin/bash", "-c", "/init.sh `hostname | awk -F '-' '{print $NF}'`"]
+ ports:
+ - name: scheduler
+ containerPort: {{ .Values.scheduler.port }}
+ - name: rpc
+ containerPort: {{ .Values.scheduler.rpcPort }}
+ - name: akka-remoting
+ containerPort: 25520
+ - name: akka-mgmt-http
+ containerPort: 19999
+ livenessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.livenessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.livenessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.livenessProbe.timeoutSeconds }}
+ readinessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.readinessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.readinessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.readinessProbe.timeoutSeconds }}
+ env:
Review Comment:
Similarly, would be great to add `kafka_topic_prefix` too.
https://github.com/apache/openwhisk/blob/master/ansible/roles/schedulers/tasks/deploy.yml#L150
BTW, it seems the prefix configuration for kafka topic is not applied in this repo at all.
##########
helm/openwhisk/values.yaml:
##########
@@ -249,20 +255,69 @@ nginx:
# Controller configurations
controller:
imageName: "openwhisk/controller"
- imageTag: "c5970a6"
+ imageTag: "a1639f0"
+ imagePullPolicy: "IfNotPresent"
+ replicaCount: 1
+ restartPolicy: "Always"
+ port: 8080
+ options: ""
+ jvmHeapMB: "1024"
+ jvmOptions: ""
+ loglevel: "INFO"
+
+# Scheduler configurations
+scheduler:
+ enable: true
Review Comment:
+1 to David's opinion.
We may take the same approach with the core repo.
<img width="1091" alt="image" src="https://user-images.githubusercontent.com/3447251/190398762-83320684-d6a0-4c2d-a5c1-bb732b5e1643.png">
It has a dedicated CI pipeline for the new scheduler.
##########
tools/travis/deploy-chart.sh:
##########
@@ -216,12 +216,17 @@ if [ "${OW_LEAN_MODE:-false}" == "false" ]; then
# Wait for the controller to confirm that it has at least one healthy invoker
verifyHealthyInvoker
+ # Wait for scheduler to be up
+ statefulsetHealthCheck "ow4travis-scheduler"
Review Comment:
If we would not actually invoke a health action, I think it would be enough to go.
##########
helm/openwhisk/templates/scheduler-pod.yaml:
##########
@@ -0,0 +1,273 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+{{- if .Values.scheduler.enable }}
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: {{ .Release.Name }}-scheduler
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 4 }}
+spec:
+ serviceName: {{ .Release.Name }}-scheduler
+ podManagementPolicy: "Parallel"
+ replicas: {{ .Values.scheduler.replicaCount }}
+ selector:
+ matchLabels:
+ name: {{ .Release.Name }}-scheduler
+ template:
+ metadata:
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 8 }}
+
+ {{- if .Values.metrics.prometheusEnabled }}
+ annotations:
+ prometheus.io/scrape: 'true'
+ prometheus.io/port: '{{ .Values.scheduler.port }}'
+ {{- end }}
+
+ spec:
+ serviceAccountName: {{ .Release.Name }}-core
+ restartPolicy: {{ .Values.scheduler.restartPolicy }}
+
+ {{- if .Values.affinity.enabled }}
+ affinity:
+{{ include "openwhisk.affinity.core" . | indent 8 }}
+{{ include "openwhisk.affinity.selfAntiAffinity" ( printf "%s-scheduler" .Release.Name ) | indent 8 }}
+ {{- end }}
+
+ {{- if .Values.toleration.enabled }}
+ tolerations:
+{{ include "openwhisk.toleration.core" . | indent 8 }}
+ {{- end }}
+
+ initContainers:
+ # The scheduler must wait for kafka and/or couchdb to be ready before it starts
+{{ include "openwhisk.readiness.waitForController" . | indent 6 }}
+{{ include "openwhisk.docker.imagePullSecrets" . | indent 6 }}
+
+ containers:
+ - name: scheduler
+ imagePullPolicy: {{ .Values.scheduler.imagePullPolicy | quote }}
+ image: "{{- .Values.docker.registry.name -}}{{- .Values.scheduler.imageName -}}:{{- .Values.scheduler.imageTag -}}"
+ command: ["/bin/bash", "-c", "/init.sh `hostname | awk -F '-' '{print $NF}'`"]
+ ports:
+ - name: scheduler
+ containerPort: {{ .Values.scheduler.port }}
+ - name: rpc
+ containerPort: {{ .Values.scheduler.rpcPort }}
+ - name: akka-remoting
+ containerPort: 25520
+ - name: akka-mgmt-http
+ containerPort: 19999
+ livenessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.livenessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.livenessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.livenessProbe.timeoutSeconds }}
+ readinessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.readinessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.readinessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.readinessProbe.timeoutSeconds }}
+ env:
Review Comment:
`CONFIG_whisk_scheduler_queue_maxBlackboxRetentionMs` is required.
https://github.com/apache/openwhisk/blob/master/ansible/roles/schedulers/tasks/deploy.yml#L128
##########
helm/openwhisk/templates/_helpers.tpl:
##########
@@ -224,12 +238,20 @@ app: {{ template "openwhisk.fullname" . }}
value: {{ .Values.whisk.kafka.topics.health.retentionMs | quote }}
- name: "CONFIG_whisk_kafka_topics_health_segmentBytes"
value: {{ .Values.whisk.kafka.topics.health.segmentBytes | quote }}
+
- name: "CONFIG_whisk_kafka_topics_invoker_retentionBytes"
value: {{ .Values.whisk.kafka.topics.invoker.retentionBytes | quote }}
- name: "CONFIG_whisk_kafka_topics_invoker_retentionMs"
value: {{ .Values.whisk.kafka.topics.invoker.retentionMs | quote }}
- name: "CONFIG_whisk_kafka_topics_invoker_segmentBytes"
value: {{ .Values.whisk.kafka.topics.invoker.segmentBytes | quote }}
+
+- name: "CONFIG_whisk_kafka_topics_scheduler_retentionBytes"
+ value: {{ .Values.whisk.kafka.topics.scheduler.retentionBytes | quote }}
+- name: "CONFIG_whisk_kafka_topics_scheduler_retentionMs"
+ value: {{ .Values.whisk.kafka.topics.scheduler.retentionMs | quote }}
+- name: "CONFIG_whisk_kafka_topics_scheduler_segmentBytes"
+ value: {{ .Values.whisk.kafka.topics.scheduler.segmentBytes | quote }}
Review Comment:
Not only topic configurations for the `scheduler` topic but also for the `creationAck` topic are required.
https://github.com/apache/openwhisk/blob/master/ansible/roles/schedulers/tasks/deploy.yml#L144
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] hunhoffe commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
hunhoffe commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r920066240
##########
tools/travis/deploy-chart.sh:
##########
@@ -216,12 +216,17 @@ if [ "${OW_LEAN_MODE:-false}" == "false" ]; then
# Wait for the controller to confirm that it has at least one healthy invoker
verifyHealthyInvoker
+ # Wait for scheduler to be up
+ statefulsetHealthCheck "ow4travis-scheduler"
Review Comment:
Is there are an additional, more rigorous check to do than just this? e.g., something analogous to the invoker health check but for the scheduler. @style95, if you wouldn't mind chiming in?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] style95 commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
style95 commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r972595284
##########
helm/openwhisk/templates/scheduler-pod.yaml:
##########
@@ -0,0 +1,273 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+{{- if .Values.scheduler.enable }}
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: {{ .Release.Name }}-scheduler
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 4 }}
+spec:
+ serviceName: {{ .Release.Name }}-scheduler
+ podManagementPolicy: "Parallel"
+ replicas: {{ .Values.scheduler.replicaCount }}
+ selector:
+ matchLabels:
+ name: {{ .Release.Name }}-scheduler
+ template:
+ metadata:
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 8 }}
+
+ {{- if .Values.metrics.prometheusEnabled }}
+ annotations:
+ prometheus.io/scrape: 'true'
+ prometheus.io/port: '{{ .Values.scheduler.port }}'
+ {{- end }}
+
+ spec:
+ serviceAccountName: {{ .Release.Name }}-core
+ restartPolicy: {{ .Values.scheduler.restartPolicy }}
+
+ {{- if .Values.affinity.enabled }}
+ affinity:
+{{ include "openwhisk.affinity.core" . | indent 8 }}
+{{ include "openwhisk.affinity.selfAntiAffinity" ( printf "%s-scheduler" .Release.Name ) | indent 8 }}
+ {{- end }}
+
+ {{- if .Values.toleration.enabled }}
+ tolerations:
+{{ include "openwhisk.toleration.core" . | indent 8 }}
+ {{- end }}
+
+ initContainers:
+ # The scheduler must wait for kafka and/or couchdb to be ready before it starts
+{{ include "openwhisk.readiness.waitForController" . | indent 6 }}
+{{ include "openwhisk.docker.imagePullSecrets" . | indent 6 }}
+
+ containers:
+ - name: scheduler
+ imagePullPolicy: {{ .Values.scheduler.imagePullPolicy | quote }}
+ image: "{{- .Values.docker.registry.name -}}{{- .Values.scheduler.imageName -}}:{{- .Values.scheduler.imageTag -}}"
+ command: ["/bin/bash", "-c", "/init.sh `hostname | awk -F '-' '{print $NF}'`"]
+ ports:
+ - name: scheduler
+ containerPort: {{ .Values.scheduler.port }}
+ - name: rpc
+ containerPort: {{ .Values.scheduler.rpcPort }}
+ - name: akka-remoting
+ containerPort: 25520
+ - name: akka-mgmt-http
+ containerPort: 19999
+ livenessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.livenessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.livenessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.livenessProbe.timeoutSeconds }}
+ readinessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.readinessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.readinessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.readinessProbe.timeoutSeconds }}
+ env:
Review Comment:
The cluster name is used to differentiate multiple versions when two different releases share the same ETCD.
https://github.com/apache/openwhisk/blob/master/common/scala/src/main/scala/org/apache/openwhisk/core/etcd/EtcdUtils.scala#L72
In this change, it seems each release uses its own ETCD but when an external ETCD is shared among multiple clusters for some reason(some may want to deploy clusters in a blue-green way), the config would be required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] style95 commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
style95 commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r977253141
##########
helm/openwhisk/templates/_helpers.tpl:
##########
@@ -224,12 +238,20 @@ app: {{ template "openwhisk.fullname" . }}
value: {{ .Values.whisk.kafka.topics.health.retentionMs | quote }}
- name: "CONFIG_whisk_kafka_topics_health_segmentBytes"
value: {{ .Values.whisk.kafka.topics.health.segmentBytes | quote }}
+
- name: "CONFIG_whisk_kafka_topics_invoker_retentionBytes"
value: {{ .Values.whisk.kafka.topics.invoker.retentionBytes | quote }}
- name: "CONFIG_whisk_kafka_topics_invoker_retentionMs"
value: {{ .Values.whisk.kafka.topics.invoker.retentionMs | quote }}
- name: "CONFIG_whisk_kafka_topics_invoker_segmentBytes"
value: {{ .Values.whisk.kafka.topics.invoker.segmentBytes | quote }}
+
+- name: "CONFIG_whisk_kafka_topics_scheduler_retentionBytes"
+ value: {{ .Values.whisk.kafka.topics.scheduler.retentionBytes | quote }}
+- name: "CONFIG_whisk_kafka_topics_scheduler_retentionMs"
+ value: {{ .Values.whisk.kafka.topics.scheduler.retentionMs | quote }}
+- name: "CONFIG_whisk_kafka_topics_scheduler_segmentBytes"
+ value: {{ .Values.whisk.kafka.topics.scheduler.segmentBytes | quote }}
Review Comment:
Can we add the same configurations for the `creationAck` topic too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] hunhoffe commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
hunhoffe commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r972478031
##########
helm/openwhisk/templates/scheduler-pod.yaml:
##########
@@ -0,0 +1,273 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+{{- if .Values.scheduler.enable }}
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: {{ .Release.Name }}-scheduler
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 4 }}
+spec:
+ serviceName: {{ .Release.Name }}-scheduler
+ podManagementPolicy: "Parallel"
+ replicas: {{ .Values.scheduler.replicaCount }}
+ selector:
+ matchLabels:
+ name: {{ .Release.Name }}-scheduler
+ template:
+ metadata:
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 8 }}
+
+ {{- if .Values.metrics.prometheusEnabled }}
+ annotations:
+ prometheus.io/scrape: 'true'
+ prometheus.io/port: '{{ .Values.scheduler.port }}'
+ {{- end }}
+
+ spec:
+ serviceAccountName: {{ .Release.Name }}-core
+ restartPolicy: {{ .Values.scheduler.restartPolicy }}
+
+ {{- if .Values.affinity.enabled }}
+ affinity:
+{{ include "openwhisk.affinity.core" . | indent 8 }}
+{{ include "openwhisk.affinity.selfAntiAffinity" ( printf "%s-scheduler" .Release.Name ) | indent 8 }}
+ {{- end }}
+
+ {{- if .Values.toleration.enabled }}
+ tolerations:
+{{ include "openwhisk.toleration.core" . | indent 8 }}
+ {{- end }}
+
+ initContainers:
+ # The scheduler must wait for kafka and/or couchdb to be ready before it starts
+{{ include "openwhisk.readiness.waitForController" . | indent 6 }}
+{{ include "openwhisk.docker.imagePullSecrets" . | indent 6 }}
+
+ containers:
+ - name: scheduler
+ imagePullPolicy: {{ .Values.scheduler.imagePullPolicy | quote }}
+ image: "{{- .Values.docker.registry.name -}}{{- .Values.scheduler.imageName -}}:{{- .Values.scheduler.imageTag -}}"
+ command: ["/bin/bash", "-c", "/init.sh `hostname | awk -F '-' '{print $NF}'`"]
+ ports:
+ - name: scheduler
+ containerPort: {{ .Values.scheduler.port }}
+ - name: rpc
+ containerPort: {{ .Values.scheduler.rpcPort }}
+ - name: akka-remoting
+ containerPort: 25520
+ - name: akka-mgmt-http
+ containerPort: 19999
+ livenessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.livenessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.livenessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.livenessProbe.timeoutSeconds }}
+ readinessProbe:
+ httpGet:
+ path: "/ping"
+ port: {{ .Values.scheduler.port }}
+ scheme: "HTTP"
+ initialDelaySeconds: {{ .Values.probes.scheduler.readinessProbe.initialDelaySeconds }}
+ periodSeconds: {{ .Values.probes.scheduler.readinessProbe.periodSeconds }}
+ timeoutSeconds: {{ .Values.probes.scheduler.readinessProbe.timeoutSeconds }}
+ env:
Review Comment:
Regarding ```whisk_cluster_name```: I believe you name a deployment when you create it with helm (which becomes ```Release.name``` in the helm template files), which would allow you to deploy multiple clusters without conflict (I am speaking hypothetically, as I have not tried this myself). If I am understanding correctly, perhaps ```whisk_cluster_name``` is not needed here. @dgrove-oss does this seem correct?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] hunhoffe commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
hunhoffe commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r973281374
##########
helm/openwhisk/templates/scheduler-pod.yaml:
##########
@@ -0,0 +1,273 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+{{- if .Values.scheduler.enable }}
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+ name: {{ .Release.Name }}-scheduler
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 4 }}
+spec:
+ serviceName: {{ .Release.Name }}-scheduler
+ podManagementPolicy: "Parallel"
+ replicas: {{ .Values.scheduler.replicaCount }}
+ selector:
+ matchLabels:
+ name: {{ .Release.Name }}-scheduler
+ template:
+ metadata:
+ labels:
+ name: {{ .Release.Name }}-scheduler
+{{ include "openwhisk.label_boilerplate" . | indent 8 }}
+
+ {{- if .Values.metrics.prometheusEnabled }}
+ annotations:
+ prometheus.io/scrape: 'true'
+ prometheus.io/port: '{{ .Values.scheduler.port }}'
+ {{- end }}
+
+ spec:
+ serviceAccountName: {{ .Release.Name }}-core
+ restartPolicy: {{ .Values.scheduler.restartPolicy }}
+
+ {{- if .Values.affinity.enabled }}
+ affinity:
+{{ include "openwhisk.affinity.core" . | indent 8 }}
+{{ include "openwhisk.affinity.selfAntiAffinity" ( printf "%s-scheduler" .Release.Name ) | indent 8 }}
+ {{- end }}
+
+ {{- if .Values.toleration.enabled }}
+ tolerations:
+{{ include "openwhisk.toleration.core" . | indent 8 }}
+ {{- end }}
+
+ initContainers:
+ # The scheduler must wait for kafka and/or couchdb to be ready before it starts
+{{ include "openwhisk.readiness.waitForController" . | indent 6 }}
Review Comment:
I changed to ```waitForKafka```, ```waitForCouchDB```, and ```waitForEtcd```, and also added ```waitForEtcd``` to the controller.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] hunhoffe commented on pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
hunhoffe commented on PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#issuecomment-1182782774
Pushed but having issues with inProgressJobRetention parsing in new scheduler image, see: https://github.com/apache/openwhisk/pull/5281
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] hunhoffe commented on pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
hunhoffe commented on PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#issuecomment-1254328584
I believe I've addressed all comments except the external etcd feature. I wrote a comment that it is not yet supported in the values.yaml file, and I'm hoping that it is a feature that could be added later; I could create an issue for it so it is not forgotten.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] dgrove-oss commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
dgrove-oss commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r977534981
##########
helm/openwhisk/values.yaml:
##########
@@ -139,9 +139,9 @@ whisk:
includeSystemTests: false
versions:
openwhisk:
- buildDate: "2022-06-23-06:18:00Z"
Review Comment:
My habit has been to use the UTC time of the commit we are using in `gitTag` as the `buildDate`. I don't think this actually matters, but it's what I'd been doing ;)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] style95 commented on pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
style95 commented on PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#issuecomment-1248054310
@hunhoffe
This is great!
Thank you for your contribution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] hunhoffe commented on a diff in pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
hunhoffe commented on code in PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#discussion_r973069049
##########
helm/openwhisk/values.yaml:
##########
@@ -249,20 +255,69 @@ nginx:
# Controller configurations
controller:
imageName: "openwhisk/controller"
- imageTag: "c5970a6"
+ imageTag: "a1639f0"
+ imagePullPolicy: "IfNotPresent"
+ replicaCount: 1
+ restartPolicy: "Always"
+ port: 8080
+ options: ""
+ jvmHeapMB: "1024"
+ jvmOptions: ""
+ loglevel: "INFO"
+
+# Scheduler configurations
+scheduler:
+ enable: true
Review Comment:
I've added a scheduler check to the travis test matrix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] hunhoffe commented on pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
hunhoffe commented on PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729#issuecomment-1255227074
I created an issue for the external etcd support and fixed the timestamp (https://github.com/apache/openwhisk-deploy-kube/issues/751). Pending any further feedback, I believe this is ready for merge. Thank you for your time in reviewing @style95 and @dgrove-oss!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [openwhisk-deploy-kube] dgrove-oss merged pull request #729: Deploy with new scheduler
Posted by GitBox <gi...@apache.org>.
dgrove-oss merged PR #729:
URL: https://github.com/apache/openwhisk-deploy-kube/pull/729
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@openwhisk.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org