You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/04/18 14:46:58 UTC
[GitHub] [beam] elink21 opened a new pull request, #17383: Add cron groovy job for credentials rotation
elink21 opened a new pull request, #17383:
URL: https://github.com/apache/beam/pull/17383
A new cron job was created to rotate cluster credentials automatically, the cron job will be executed each 2 months.
Also a maintenance window was set to avoid disruption in the clusters.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kerrydc commented on a diff in pull request #17383: [BEAM-14169] Add Credentials rotation cron job for clusters
Posted by GitBox <gi...@apache.org>.
kerrydc commented on code in PR #17383:
URL: https://github.com/apache/beam/pull/17383#discussion_r866115080
##########
.test-infra/jenkins/job_ClusterCredentialsRotation.groovy:
##########
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+
+job('Rotate Cluster Credentials') {
+ description('Rotates Certificates and performs an IP rotation for metrics and io-datastores')
+
+ // Set common parameters.
+ commonJobProperties.setTopLevelMainJobProperties(delegate)
+
+ // Sets that this is a cron job.
+ commonJobProperties.setCronJob(delegate, 'H 2 1 */2 *')// At 00:02am every second month.
+
+ steps {
+
+ //Credentials rotation for metrics and io-datastores
+
+ //Set a maintenance window
+ shell('''gcloud container clusters update metrics \
+ --zone=us-central1-a --maintenance-window=06:00 --quiet''')
+
+ shell('''gcloud container clusters update io-datastores \
+ --zone=us-central1-a --maintenance-window=06:00 --quiet''')
+
+ //Starting credential rotation
+ // it's necessary to rebuild the nodes after rotation to avoid apiservices issues
+ shell('''gcloud container clusters update metrics \
+ --start-credential-rotation --zone=us-central1-a --quiet''')
+
+ shell('''gcloud container clusters update io-datastores \
+ --start-credential-rotation --zone=us-central1-a --quiet''')
+
+ //Rebuilding the nodes
+ shell('''gcloud container clusters upgrade metrics \
Review Comment:
What happens if these commands close with an error state?
##########
.test-infra/jenkins/job_ClusterCredentialsRotation.groovy:
##########
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+
+job('Rotate Cluster Credentials') {
+ description('Rotates Certificates and performs an IP rotation for metrics and io-datastores')
+
+ // Set common parameters.
+ commonJobProperties.setTopLevelMainJobProperties(delegate)
+
+ // Sets that this is a cron job.
+ commonJobProperties.setCronJob(delegate, 'H 2 1 */2 *')// At 00:02am every second month.
+
+ steps {
+
+ //Credentials rotation for metrics and io-datastores
+
+ //Set a maintenance window
+ shell('''gcloud container clusters update metrics \
+ --zone=us-central1-a --maintenance-window=06:00 --quiet''')
+
+ shell('''gcloud container clusters update io-datastores \
+ --zone=us-central1-a --maintenance-window=06:00 --quiet''')
Review Comment:
same here.
##########
.test-infra/jenkins/job_ClusterCredentialsRotation.groovy:
##########
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+
+job('Rotate Cluster Credentials') {
+ description('Rotates Certificates and performs an IP rotation for metrics and io-datastores')
+
+ // Set common parameters.
+ commonJobProperties.setTopLevelMainJobProperties(delegate)
+
+ // Sets that this is a cron job.
+ commonJobProperties.setCronJob(delegate, 'H 2 1 */2 *')// At 00:02am every second month.
+
+ steps {
+
+ //Credentials rotation for metrics and io-datastores
+
+ //Set a maintenance window
+ shell('''gcloud container clusters update metrics \
+ --zone=us-central1-a --maintenance-window=06:00 --quiet''')
Review Comment:
Why do you set a daily maintenance window for 6AM UTC? This command sets a daily 4 hour maintenance window.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] elink21 commented on a diff in pull request #17383: [BEAM-14169] Add Credentials rotation cron job for clusters
Posted by GitBox <gi...@apache.org>.
elink21 commented on code in PR #17383:
URL: https://github.com/apache/beam/pull/17383#discussion_r868163713
##########
.test-infra/jenkins/job_ClusterCredentialsRotation.groovy:
##########
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+
+job('Rotate Cluster Credentials') {
+ description('Rotates Certificates and performs an IP rotation for metrics and io-datastores')
+
+ // Set common parameters.
+ commonJobProperties.setTopLevelMainJobProperties(delegate)
+
+ // Sets that this is a cron job.
+ commonJobProperties.setCronJob(delegate, 'H 2 1 */2 *')// At 00:02am every second month.
+
+ steps {
+
+ //Credentials rotation for metrics and io-datastores
+
+ //Set a maintenance window
+ shell('''gcloud container clusters update metrics \
+ --zone=us-central1-a --maintenance-window=06:00 --quiet''')
+
+ shell('''gcloud container clusters update io-datastores \
+ --zone=us-central1-a --maintenance-window=06:00 --quiet''')
+
+ //Starting credential rotation
+ // it's necessary to rebuild the nodes after rotation to avoid apiservices issues
+ shell('''gcloud container clusters update metrics \
+ --start-credential-rotation --zone=us-central1-a --quiet''')
+
+ shell('''gcloud container clusters update io-datastores \
+ --start-credential-rotation --zone=us-central1-a --quiet''')
+
+ //Rebuilding the nodes
+ shell('''gcloud container clusters upgrade metrics \
Review Comment:
Sure, I checked Jenkins configuration and it has the -**xe** flag when executing DSL Jobs, so if a command returns an error the script stops and it will be marked as a fail, I separated the rotation with one job per cluster with separated email notifications to _dev_, pointing to JOB_URL and JENKINS_URL for further details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] asf-ci commented on pull request #17383: Add cron groovy job for credentials rotation
Posted by GitBox <gi...@apache.org>.
asf-ci commented on PR #17383:
URL: https://github.com/apache/beam/pull/17383#issuecomment-1101465828
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] elink21 commented on a diff in pull request #17383: [BEAM-14169] Add Credentials rotation cron job for clusters
Posted by GitBox <gi...@apache.org>.
elink21 commented on code in PR #17383:
URL: https://github.com/apache/beam/pull/17383#discussion_r854422081
##########
.test-infra/jenkins/job_ClusterCredentialsRotation.groovy:
##########
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+
+job('Rotate Cluster Credentials') {
+ description('Rotates Certificates and performs an IP rotation for metrics and io-datastores')
+
+ // Set common parameters.
+ commonJobProperties.setTopLevelMainJobProperties(delegate)
+
+ // Sets that this is a cron job.
+ commonJobProperties.setCronJob(delegate, 'H 2 1 */2 *')// At 00:02am every second month.
+
+ steps {
+
+ //Credentials rotation for metrics and io-datastores
+
+ //Set a maintenance window
+ shell('''printf 'yes'| gcloud container clusters update metrics \
Review Comment:
Yes, it works, I avoided to use --quiet cause I thought it would remove the logs, but I tested it and it only auto confirms the command, I removed printf and added --quiet instead, thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kennknowles commented on pull request #17383: [BEAM-14169] Add Credentials rotation cron job for clusters
Posted by GitBox <gi...@apache.org>.
kennknowles commented on PR #17383:
URL: https://github.com/apache/beam/pull/17383#issuecomment-1105618690
Are you able to validate this now? The code looks fine but I am not certain how I can add another pair of eyes to checking on the functionality.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #17383: [BEAM-14169] Add Credentials rotation cron job for clusters
Posted by GitBox <gi...@apache.org>.
aaltay commented on PR #17383:
URL: https://github.com/apache/beam/pull/17383#issuecomment-1118677129
What is the next step on this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] elink21 commented on a diff in pull request #17383: [BEAM-14169] Add Credentials rotation cron job for clusters
Posted by GitBox <gi...@apache.org>.
elink21 commented on code in PR #17383:
URL: https://github.com/apache/beam/pull/17383#discussion_r868163713
##########
.test-infra/jenkins/job_ClusterCredentialsRotation.groovy:
##########
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+
+job('Rotate Cluster Credentials') {
+ description('Rotates Certificates and performs an IP rotation for metrics and io-datastores')
+
+ // Set common parameters.
+ commonJobProperties.setTopLevelMainJobProperties(delegate)
+
+ // Sets that this is a cron job.
+ commonJobProperties.setCronJob(delegate, 'H 2 1 */2 *')// At 00:02am every second month.
+
+ steps {
+
+ //Credentials rotation for metrics and io-datastores
+
+ //Set a maintenance window
+ shell('''gcloud container clusters update metrics \
+ --zone=us-central1-a --maintenance-window=06:00 --quiet''')
+
+ shell('''gcloud container clusters update io-datastores \
+ --zone=us-central1-a --maintenance-window=06:00 --quiet''')
+
+ //Starting credential rotation
+ // it's necessary to rebuild the nodes after rotation to avoid apiservices issues
+ shell('''gcloud container clusters update metrics \
+ --start-credential-rotation --zone=us-central1-a --quiet''')
+
+ shell('''gcloud container clusters update io-datastores \
+ --start-credential-rotation --zone=us-central1-a --quiet''')
+
+ //Rebuilding the nodes
+ shell('''gcloud container clusters upgrade metrics \
Review Comment:
Sure, I checked Jenkins configuration and it has the -**xe** flag when execution DSL Jobs, so if a command returns an error the script stops and it will be marked as a fail, I separated the rotation with one job per cluster with separated email notifications to _dev_, pointing to JOB_URL and JENKINS_URL for further details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] elink21 commented on pull request #17383: [BEAM-14169] Add Credentials rotation cron job for clusters
Posted by GitBox <gi...@apache.org>.
elink21 commented on PR #17383:
URL: https://github.com/apache/beam/pull/17383#issuecomment-1118733274
> What is the next step on this PR?
After Kerry or someone else review the approach, the next step is to seed the job in a Jenkins Worker to test the functionality so it can be added as a permanent cron job and finally merge it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kerrydc commented on a diff in pull request #17383: [BEAM-14169] Add Credentials rotation cron job for clusters
Posted by GitBox <gi...@apache.org>.
kerrydc commented on code in PR #17383:
URL: https://github.com/apache/beam/pull/17383#discussion_r866844464
##########
.test-infra/jenkins/job_ClusterCredentialsRotation.groovy:
##########
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+
+job('Rotate Cluster Credentials') {
+ description('Rotates Certificates and performs an IP rotation for metrics and io-datastores')
+
+ // Set common parameters.
+ commonJobProperties.setTopLevelMainJobProperties(delegate)
+
+ // Sets that this is a cron job.
+ commonJobProperties.setCronJob(delegate, 'H 2 1 */2 *')// At 00:02am every second month.
+
+ steps {
+
+ //Credentials rotation for metrics and io-datastores
+
+ //Set a maintenance window
+ shell('''gcloud container clusters update metrics \
+ --zone=us-central1-a --maintenance-window=06:00 --quiet''')
+
+ shell('''gcloud container clusters update io-datastores \
+ --zone=us-central1-a --maintenance-window=06:00 --quiet''')
+
+ //Starting credential rotation
+ // it's necessary to rebuild the nodes after rotation to avoid apiservices issues
+ shell('''gcloud container clusters update metrics \
+ --start-credential-rotation --zone=us-central1-a --quiet''')
+
+ shell('''gcloud container clusters update io-datastores \
+ --start-credential-rotation --zone=us-central1-a --quiet''')
+
+ //Rebuilding the nodes
+ shell('''gcloud container clusters upgrade metrics \
Review Comment:
Please add the condition and email alert. I also recommend making each cluster independent, with separate alerts in case of failure. So, try to start rotation for io-datastores, complete upgrade and update, then do the same for metrics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] elink21 commented on pull request #17383: [BEAM-14169] Add Credentials rotation cron job for clusters
Posted by GitBox <gi...@apache.org>.
elink21 commented on PR #17383:
URL: https://github.com/apache/beam/pull/17383#issuecomment-1104088081
R: @kennknowles
Hi Kenn, could you review this PR please? Thanks in advance
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] elink21 commented on pull request #17383: [BEAM-14169] Add Credentials rotation cron job for clusters
Posted by GitBox <gi...@apache.org>.
elink21 commented on PR #17383:
URL: https://github.com/apache/beam/pull/17383#issuecomment-1101494107
R: @kileys
Hi Kiley, can you review this PR please?
Also could you please run a seed job to test the new credentials rotation? Thanks in advance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kileys commented on pull request #17383: [BEAM-14169] Add Credentials rotation cron job for clusters
Posted by GitBox <gi...@apache.org>.
kileys commented on PR #17383:
URL: https://github.com/apache/beam/pull/17383#issuecomment-1101547886
R: @kennknowles
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kennknowles commented on a diff in pull request #17383: [BEAM-14169] Add Credentials rotation cron job for clusters
Posted by GitBox <gi...@apache.org>.
kennknowles commented on code in PR #17383:
URL: https://github.com/apache/beam/pull/17383#discussion_r854384489
##########
.test-infra/jenkins/job_ClusterCredentialsRotation.groovy:
##########
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+
+job('Rotate Cluster Credentials') {
+ description('Rotates Certificates and performs an IP rotation for metrics and io-datastores')
+
+ // Set common parameters.
+ commonJobProperties.setTopLevelMainJobProperties(delegate)
+
+ // Sets that this is a cron job.
+ commonJobProperties.setCronJob(delegate, 'H 2 1 */2 *')// At 00:02am every second month.
+
+ steps {
+
+ //Credentials rotation for metrics and io-datastores
+
+ //Set a maintenance window
+ shell('''printf 'yes'| gcloud container clusters update metrics \
Review Comment:
From `gcloud container clusters update --help` I see there is a gcloud global flag so I checked `gcloud --help` and this might simplify the commands here:
```
--quiet, -q
Disable all interactive prompts when running gcloud commands. If input
is required, defaults will be used, or an error will be raised.
Overrides the default core/disable_prompts property value for this
command invocation. This is equivalent to setting the environment
variable CLOUDSDK_CORE_DISABLE_PROMPTS to 1.
```
Would this work?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] elink21 commented on pull request #17383: [BEAM-14169] Add Credentials rotation cron job for clusters
Posted by GitBox <gi...@apache.org>.
elink21 commented on PR #17383:
URL: https://github.com/apache/beam/pull/17383#issuecomment-1105761566
Sorry Kenneth, I got confused with the names, now I realized I asked for the seed job with the same person in the chat and here, my apologies for the confusion.
We have tested this against replicas of the actual clusters and also we executed each step manually on the real ones, but I agree that extra validation is required , do you know who can I ask for this?.
Thanks in advance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] asf-ci commented on pull request #17383: Add cron groovy job for credentials rotation
Posted by GitBox <gi...@apache.org>.
asf-ci commented on PR #17383:
URL: https://github.com/apache/beam/pull/17383#issuecomment-1101465827
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] asf-ci commented on pull request #17383: Add cron groovy job for credentials rotation
Posted by GitBox <gi...@apache.org>.
asf-ci commented on PR #17383:
URL: https://github.com/apache/beam/pull/17383#issuecomment-1101465833
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kennknowles merged pull request #17383: [BEAM-14169] Add Credentials rotation cron job for clusters
Posted by GitBox <gi...@apache.org>.
kennknowles merged PR #17383:
URL: https://github.com/apache/beam/pull/17383
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] elink21 commented on a diff in pull request #17383: [BEAM-14169] Add Credentials rotation cron job for clusters
Posted by GitBox <gi...@apache.org>.
elink21 commented on code in PR #17383:
URL: https://github.com/apache/beam/pull/17383#discussion_r866174263
##########
.test-infra/jenkins/job_ClusterCredentialsRotation.groovy:
##########
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+
+job('Rotate Cluster Credentials') {
+ description('Rotates Certificates and performs an IP rotation for metrics and io-datastores')
+
+ // Set common parameters.
+ commonJobProperties.setTopLevelMainJobProperties(delegate)
+
+ // Sets that this is a cron job.
+ commonJobProperties.setCronJob(delegate, 'H 2 1 */2 *')// At 00:02am every second month.
+
+ steps {
+
+ //Credentials rotation for metrics and io-datastores
+
+ //Set a maintenance window
+ shell('''gcloud container clusters update metrics \
+ --zone=us-central1-a --maintenance-window=06:00 --quiet''')
+
+ shell('''gcloud container clusters update io-datastores \
+ --zone=us-central1-a --maintenance-window=06:00 --quiet''')
+
+ //Starting credential rotation
+ // it's necessary to rebuild the nodes after rotation to avoid apiservices issues
+ shell('''gcloud container clusters update metrics \
+ --start-credential-rotation --zone=us-central1-a --quiet''')
+
+ shell('''gcloud container clusters update io-datastores \
+ --start-credential-rotation --zone=us-central1-a --quiet''')
+
+ //Rebuilding the nodes
+ shell('''gcloud container clusters upgrade metrics \
Review Comment:
If anything goes wrong after line 50, GCP will automatically complete the process after 7 days ([Automatic completion](https://cloud.google.com/kubernetes-engine/docs/how-to/credential-rotation#performing_a_credential_rotation)).
If something fails during the first 2 commands (_--start-credential-rotation_), the rotation will not start and the cluster will keep the previous credentials. We will add a condition to only execute the following steps if the start-credential-rotation part was successful. I could also add an email alert in case of failure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] elink21 commented on pull request #17383: [BEAM-14169] Add Credentials rotation cron job for clusters
Posted by GitBox <gi...@apache.org>.
elink21 commented on PR #17383:
URL: https://github.com/apache/beam/pull/17383#issuecomment-1106731456
I found that Kerry Donny-Clark was in charge of the manual rotation for [BEAM-13763](https://issues.apache.org/jira/browse/BEAM-13763), maybe we could add him as an additional reviewer?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kennknowles commented on pull request #17383: [BEAM-14169] Add Credentials rotation cron job for clusters
Posted by GitBox <gi...@apache.org>.
kennknowles commented on PR #17383:
URL: https://github.com/apache/beam/pull/17383#issuecomment-1108009200
R: @kerrydc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] elink21 commented on a diff in pull request #17383: [BEAM-14169] Add Credentials rotation cron job for clusters
Posted by GitBox <gi...@apache.org>.
elink21 commented on code in PR #17383:
URL: https://github.com/apache/beam/pull/17383#discussion_r866151457
##########
.test-infra/jenkins/job_ClusterCredentialsRotation.groovy:
##########
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+
+job('Rotate Cluster Credentials') {
+ description('Rotates Certificates and performs an IP rotation for metrics and io-datastores')
+
+ // Set common parameters.
+ commonJobProperties.setTopLevelMainJobProperties(delegate)
+
+ // Sets that this is a cron job.
+ commonJobProperties.setCronJob(delegate, 'H 2 1 */2 *')// At 00:02am every second month.
+
+ steps {
+
+ //Credentials rotation for metrics and io-datastores
+
+ //Set a maintenance window
+ shell('''gcloud container clusters update metrics \
+ --zone=us-central1-a --maintenance-window=06:00 --quiet''')
Review Comment:
It was intended to avoid disruption in the nodes, but as now the rebuild step is included in the job I'm going to remove that maintenance window.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] elink21 commented on pull request #17383: [BEAM-14169] Add Credentials rotation cron job for clusters
Posted by GitBox <gi...@apache.org>.
elink21 commented on PR #17383:
URL: https://github.com/apache/beam/pull/17383#issuecomment-1105651185
> Are you able to validate this now? The code looks fine but I am not certain how I can add another pair of eyes to checking on the functionality.
I'm requesting a seed job from Kenneth, to run this on a jenkins worker and be sure everything is fine, will post updates from that seed, thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] elink21 commented on pull request #17383: [BEAM-14169] Add Credentials rotation cron job for clusters
Posted by GitBox <gi...@apache.org>.
elink21 commented on PR #17383:
URL: https://github.com/apache/beam/pull/17383#issuecomment-1104272278
> This needs to be cloned by me and have the seed job run, yes?
Yes. please.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org