You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "bugraoz93 (via GitHub)" <gi...@apache.org> on 2024/09/05 21:20:00 UTC
[PR] Retry Test Airflow Release Step Multiple Times [airflow]
bugraoz93 opened a new pull request, #42043:
URL: https://github.com/apache/airflow/pull/42043
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
closes: #42025
* Script added for retrying non-interactive commands.
* The script is used for retrying breeze commands under `Test Airflow Release Step` 3 times
<!-- Please keep an empty line above the dashes. -->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Retry Test Airflow Release Step Multiple Times [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #42043:
URL: https://github.com/apache/airflow/pull/42043
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Retry Test Airflow Release Step Multiple Times [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #42043:
URL: https://github.com/apache/airflow/pull/42043#discussion_r1749206328
##########
scripts/ci/testing/run_breeze_command_with_retries.sh:
##########
@@ -0,0 +1,66 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# This script should be only used commands without waiting interactive inputs.
+# For example: 'apt-get install [package_name]' will ask you to input 'y' while installing.
+# This is an interactive input which you cannot provide.
+export COLOR_RED=$'\e[31m'
+export COLOR_YELLOW=$'\e[33m'
+export COLOR_RESET=$'\e[0m'
+
+if [ ! "$#" -eq 2 ]; then
+ echo "${COLOR_RED}You must provide exactly two argument!.${COLOR_RESET}"
+ exit 1
+fi
+
+# Param 1: Breeze Command
+# Param 2: Number of Retry(ies)
+COMMAND_TO_ATTEMPT="breeze $1"
+NUMBER_OF_ATTEMPT=$2
+NUMBER_OF_ATTEMPT="${NUMBER_OF_ATTEMPT:-1}"
+CURRENT_ATTEMPT=2
+ATTEMPT_SUCCESS=false
+
+breeze down
+set +e
+CURRENT_RUN_OUTPUT=$($COMMAND_TO_ATTEMPT)
+RESULT=$?
+echo "$CURRENT_RUN_OUTPUT"
+set -e
+
+while [ "${ATTEMPT_SUCCESS}" = false ] && [ "${CURRENT_ATTEMPT}" -le "${NUMBER_OF_ATTEMPT}" ]; do
Review Comment:
I think this script might be vastly simplified (many less environment variables and simpler steps by using for loop and passing all arguments of the script to the breeze command:
Something like that (with added diagnostics and printing of course) should ldo the job - might need some tweaking, as I am writing it from the top of my head :) :
```bash
for i in $(seq 1 "${NUM_ATTEMPTS}")
do
if breeze "$@" then
exit 0
else
RESULT=$?
fi
done
# Error here
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Retry Test Airflow Release Step Multiple Times [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #42043:
URL: https://github.com/apache/airflow/pull/42043#issuecomment-2336531956
It's likely about executable permission for the script
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Retry Test Airflow Release Step Multiple Times [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #42043:
URL: https://github.com/apache/airflow/pull/42043#discussion_r1749206328
##########
scripts/ci/testing/run_breeze_command_with_retries.sh:
##########
@@ -0,0 +1,66 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# This script should be only used commands without waiting interactive inputs.
+# For example: 'apt-get install [package_name]' will ask you to input 'y' while installing.
+# This is an interactive input which you cannot provide.
+export COLOR_RED=$'\e[31m'
+export COLOR_YELLOW=$'\e[33m'
+export COLOR_RESET=$'\e[0m'
+
+if [ ! "$#" -eq 2 ]; then
+ echo "${COLOR_RED}You must provide exactly two argument!.${COLOR_RESET}"
+ exit 1
+fi
+
+# Param 1: Breeze Command
+# Param 2: Number of Retry(ies)
+COMMAND_TO_ATTEMPT="breeze $1"
+NUMBER_OF_ATTEMPT=$2
+NUMBER_OF_ATTEMPT="${NUMBER_OF_ATTEMPT:-1}"
+CURRENT_ATTEMPT=2
+ATTEMPT_SUCCESS=false
+
+breeze down
+set +e
+CURRENT_RUN_OUTPUT=$($COMMAND_TO_ATTEMPT)
+RESULT=$?
+echo "$CURRENT_RUN_OUTPUT"
+set -e
+
+while [ "${ATTEMPT_SUCCESS}" = false ] && [ "${CURRENT_ATTEMPT}" -le "${NUMBER_OF_ATTEMPT}" ]; do
Review Comment:
I think this script might be vastly simplified (many less environment variables and simpler steps) by using for loop and passing all arguments of the script to the breeze command:
Something like that (with added diagnostics and printing of course) should ldo the job - might need some tweaking, as I am writing it from the top of my head :) :
```bash
for i in $(seq 1 "${NUM_ATTEMPTS}")
do
if breeze "$@" then
# success here
exit 0
else
RESULT=$?
# restart here (maybe optimized by exit early `if "$i" == "${NUM_ATTEMPTS}"`
fi
done
# Error here
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Retry Test Airflow Release Step Multiple Times [airflow]
Posted by "bugraoz93 (via GitHub)" <gi...@apache.org>.
bugraoz93 commented on PR #42043:
URL: https://github.com/apache/airflow/pull/42043#issuecomment-2336628591
I thought the same at the beginning either executable permissions or ownership of the file was not exactly correct, I checked how you included the `scripts/ci/testing/run_integration_tests_with_retry.sh` file into the `integration-tests` since the approach should be pretty much the same. #39913 In this one, I couldn't find any trace of `chmod` included for the script somewhere in the project and the `integration-tests` step worked. So I assumed the scripts were included in the CI image without specifying permission of the file. I may be missing something :thinking:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Retry Test Airflow Release Step Multiple Times [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #42043:
URL: https://github.com/apache/airflow/pull/42043#discussion_r1749206328
##########
scripts/ci/testing/run_breeze_command_with_retries.sh:
##########
@@ -0,0 +1,66 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# This script should be only used commands without waiting interactive inputs.
+# For example: 'apt-get install [package_name]' will ask you to input 'y' while installing.
+# This is an interactive input which you cannot provide.
+export COLOR_RED=$'\e[31m'
+export COLOR_YELLOW=$'\e[33m'
+export COLOR_RESET=$'\e[0m'
+
+if [ ! "$#" -eq 2 ]; then
+ echo "${COLOR_RED}You must provide exactly two argument!.${COLOR_RESET}"
+ exit 1
+fi
+
+# Param 1: Breeze Command
+# Param 2: Number of Retry(ies)
+COMMAND_TO_ATTEMPT="breeze $1"
+NUMBER_OF_ATTEMPT=$2
+NUMBER_OF_ATTEMPT="${NUMBER_OF_ATTEMPT:-1}"
+CURRENT_ATTEMPT=2
+ATTEMPT_SUCCESS=false
+
+breeze down
+set +e
+CURRENT_RUN_OUTPUT=$($COMMAND_TO_ATTEMPT)
+RESULT=$?
+echo "$CURRENT_RUN_OUTPUT"
+set -e
+
+while [ "${ATTEMPT_SUCCESS}" = false ] && [ "${CURRENT_ATTEMPT}" -le "${NUMBER_OF_ATTEMPT}" ]; do
Review Comment:
I think this script might be vastly simplified (many less environment variables and simpler steps) by using for loop and passing all arguments of the script to the breeze command:
Something like that (with added diagnostics and printing of course) should ldo the job - might need some tweaking, as I am writing it from the top of my head :) :
```bash
for i in $(seq 1 "${NUM_ATTEMPTS}")
do
if breeze "$@" then
exit 0
else
RESULT=$?
# restart here
fi
done
# Error here
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Retry Test Airflow Release Step Multiple Times [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #42043:
URL: https://github.com/apache/airflow/pull/42043#discussion_r1749206328
##########
scripts/ci/testing/run_breeze_command_with_retries.sh:
##########
@@ -0,0 +1,66 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# This script should be only used commands without waiting interactive inputs.
+# For example: 'apt-get install [package_name]' will ask you to input 'y' while installing.
+# This is an interactive input which you cannot provide.
+export COLOR_RED=$'\e[31m'
+export COLOR_YELLOW=$'\e[33m'
+export COLOR_RESET=$'\e[0m'
+
+if [ ! "$#" -eq 2 ]; then
+ echo "${COLOR_RED}You must provide exactly two argument!.${COLOR_RESET}"
+ exit 1
+fi
+
+# Param 1: Breeze Command
+# Param 2: Number of Retry(ies)
+COMMAND_TO_ATTEMPT="breeze $1"
+NUMBER_OF_ATTEMPT=$2
+NUMBER_OF_ATTEMPT="${NUMBER_OF_ATTEMPT:-1}"
+CURRENT_ATTEMPT=2
+ATTEMPT_SUCCESS=false
+
+breeze down
+set +e
+CURRENT_RUN_OUTPUT=$($COMMAND_TO_ATTEMPT)
+RESULT=$?
+echo "$CURRENT_RUN_OUTPUT"
+set -e
+
+while [ "${ATTEMPT_SUCCESS}" = false ] && [ "${CURRENT_ATTEMPT}" -le "${NUMBER_OF_ATTEMPT}" ]; do
Review Comment:
I think this script might be vastly simplified (many less environment variables and simpler steps) by using for loop and passing all arguments of the script to the breeze command:
Something like that (with added diagnostics and printing of course) should ldo the job - might need some tweaking, as I am writing it from the top of my head :) :
```bash
for i in $(seq 1 "${NUM_ATTEMPTS}")
do
if breeze "$@" then
exit 0
else
RESULT=$?
fi
done
# Error here
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Retry Test Airflow Release Step Multiple Times [airflow]
Posted by "bugraoz93 (via GitHub)" <gi...@apache.org>.
bugraoz93 commented on PR #42043:
URL: https://github.com/apache/airflow/pull/42043#issuecomment-2332994908
The script including retrying the resulting permission denied. I have checked the other scripts such as `install_breeze.sh` but there wasn't any special case. I have started thinking I don't have enough permissions to include custom scripts for protection.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Retry Test Airflow Release Step Multiple Times [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #42043:
URL: https://github.com/apache/airflow/pull/42043#issuecomment-2336655725
> So I assumed the scripts were included in the CI image without specifying permission of the file. I may be missing something
There is no "trace" when you look at the file "content" - you need to see at the git file status a bit mre deeply. See for example accepted answer in https://stackoverflow.com/questions/46175753/how-do-i-see-the-permission-of-a-file-in-git - you need to manually set "chmod a+x" on the file, then add the file again and you will see that permissions of the file in git are diferent than the permission of other files:
You can also see it locally. When I check out your PR, I see:
```
[jarek:~/code/airflow] retry-test-Airflow-release-step+ ± ls -la scripts/ci/testing
total 48
drwxr-xr-x 6 jarek staff 192 Sep 8 04:45 .
drwxr-xr-x 15 jarek staff 480 Aug 27 14:13 ..
-rw-r--r-- 1 jarek staff 2258 Sep 8 04:45 run_breeze_command_with_retries.sh
-rwxr-xr-x 1 jarek staff 1719 Aug 27 14:13 run_integration_tests_with_retry.sh
-rwxr-xr-x 1 jarek staff 10839 Aug 27 14:13 summarize_captured_warnings.py
-rwxr-xr-x 1 jarek staff 3859 Aug 27 14:13 summarize_junit_failures.py
```
as you see - comparing to other scripts in the folder, your script misses the "executable" bit - and the problem is that this executable bit is not stored in `git`.
So in this case:
```bash
chmod a+x ./scripts/testing/run_breeze_command_with_retries.sh
git add ./scripts/testing/run_breeze_command_with_retries.sh
git commit --amend
```
should do the job.
You will see after `git add` that even if the content of the file did not change, the "executable bit" did - and git wil offer you to modify the executable bit for that file when you commit it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Retry Test Airflow Release Step Multiple Times [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #42043:
URL: https://github.com/apache/airflow/pull/42043#issuecomment-2336532037
`chnod a+x script.sh`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Retry Test Airflow Release Step Multiple Times [airflow]
Posted by "bugraoz93 (via GitHub)" <gi...@apache.org>.
bugraoz93 commented on PR #42043:
URL: https://github.com/apache/airflow/pull/42043#issuecomment-2336658325
Thanks for the detailed answer! I thought this was somehow an automated process. How should I miss that :facepalm:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Retry Test Airflow Release Step Multiple Times [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #42043:
URL: https://github.com/apache/airflow/pull/42043#discussion_r1749206328
##########
scripts/ci/testing/run_breeze_command_with_retries.sh:
##########
@@ -0,0 +1,66 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# This script should be only used commands without waiting interactive inputs.
+# For example: 'apt-get install [package_name]' will ask you to input 'y' while installing.
+# This is an interactive input which you cannot provide.
+export COLOR_RED=$'\e[31m'
+export COLOR_YELLOW=$'\e[33m'
+export COLOR_RESET=$'\e[0m'
+
+if [ ! "$#" -eq 2 ]; then
+ echo "${COLOR_RED}You must provide exactly two argument!.${COLOR_RESET}"
+ exit 1
+fi
+
+# Param 1: Breeze Command
+# Param 2: Number of Retry(ies)
+COMMAND_TO_ATTEMPT="breeze $1"
+NUMBER_OF_ATTEMPT=$2
+NUMBER_OF_ATTEMPT="${NUMBER_OF_ATTEMPT:-1}"
+CURRENT_ATTEMPT=2
+ATTEMPT_SUCCESS=false
+
+breeze down
+set +e
+CURRENT_RUN_OUTPUT=$($COMMAND_TO_ATTEMPT)
+RESULT=$?
+echo "$CURRENT_RUN_OUTPUT"
+set -e
+
+while [ "${ATTEMPT_SUCCESS}" = false ] && [ "${CURRENT_ATTEMPT}" -le "${NUMBER_OF_ATTEMPT}" ]; do
Review Comment:
I think this script might be vastly simplified (many less environment variables and simpler steps) by using for loop and passing all arguments of the script to the breeze command:
Something like that (with added diagnostics and printing of course) should ldo the job - might need some tweaking, as I am writing it from the top of my head :) :
```bash
for i in $(seq 1 "${NUM_ATTEMPTS}")
do
if breeze "$@" then
# success here
exit 0
else
RESULT=$?
# restart here
fi
done
# Error here
```
##########
scripts/ci/testing/run_breeze_command_with_retries.sh:
##########
@@ -0,0 +1,66 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# This script should be only used commands without waiting interactive inputs.
+# For example: 'apt-get install [package_name]' will ask you to input 'y' while installing.
+# This is an interactive input which you cannot provide.
+export COLOR_RED=$'\e[31m'
+export COLOR_YELLOW=$'\e[33m'
+export COLOR_RESET=$'\e[0m'
+
+if [ ! "$#" -eq 2 ]; then
+ echo "${COLOR_RED}You must provide exactly two argument!.${COLOR_RESET}"
+ exit 1
+fi
+
+# Param 1: Breeze Command
+# Param 2: Number of Retry(ies)
+COMMAND_TO_ATTEMPT="breeze $1"
+NUMBER_OF_ATTEMPT=$2
+NUMBER_OF_ATTEMPT="${NUMBER_OF_ATTEMPT:-1}"
+CURRENT_ATTEMPT=2
+ATTEMPT_SUCCESS=false
+
+breeze down
+set +e
+CURRENT_RUN_OUTPUT=$($COMMAND_TO_ATTEMPT)
+RESULT=$?
+echo "$CURRENT_RUN_OUTPUT"
+set -e
+
+while [ "${ATTEMPT_SUCCESS}" = false ] && [ "${CURRENT_ATTEMPT}" -le "${NUMBER_OF_ATTEMPT}" ]; do
Review Comment:
I think this script might be vastly simplified (many less environment variables and simpler steps) by using for loop and passing all arguments of the script to the breeze command:
Something like that (with added diagnostics and printing of course) should ldo the job - might need some tweaking, as I am writing it from the top of my head :) :
```bash
for i in $(seq 1 "${NUM_ATTEMPTS}")
do
if breeze "$@" then
# success here
exit 0
else
RESULT=$?
# restart here
fi
done
# Error here
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Retry Test Airflow Release Step Multiple Times [airflow]
Posted by "bugraoz93 (via GitHub)" <gi...@apache.org>.
bugraoz93 commented on code in PR #42043:
URL: https://github.com/apache/airflow/pull/42043#discussion_r1749209028
##########
scripts/ci/testing/run_breeze_command_with_retries.sh:
##########
@@ -0,0 +1,66 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# This script should be only used commands without waiting interactive inputs.
+# For example: 'apt-get install [package_name]' will ask you to input 'y' while installing.
+# This is an interactive input which you cannot provide.
+export COLOR_RED=$'\e[31m'
+export COLOR_YELLOW=$'\e[33m'
+export COLOR_RESET=$'\e[0m'
+
+if [ ! "$#" -eq 2 ]; then
+ echo "${COLOR_RED}You must provide exactly two argument!.${COLOR_RESET}"
+ exit 1
+fi
+
+# Param 1: Breeze Command
+# Param 2: Number of Retry(ies)
+COMMAND_TO_ATTEMPT="breeze $1"
+NUMBER_OF_ATTEMPT=$2
+NUMBER_OF_ATTEMPT="${NUMBER_OF_ATTEMPT:-1}"
+CURRENT_ATTEMPT=2
+ATTEMPT_SUCCESS=false
+
+breeze down
+set +e
+CURRENT_RUN_OUTPUT=$($COMMAND_TO_ATTEMPT)
+RESULT=$?
+echo "$CURRENT_RUN_OUTPUT"
+set -e
+
+while [ "${ATTEMPT_SUCCESS}" = false ] && [ "${CURRENT_ATTEMPT}" -le "${NUMBER_OF_ATTEMPT}" ]; do
Review Comment:
Make sense! Let me apply the logic to the script. Thanks :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Retry Test Airflow Release Step Multiple Times [airflow]
Posted by "bugraoz93 (via GitHub)" <gi...@apache.org>.
bugraoz93 commented on PR #42043:
URL: https://github.com/apache/airflow/pull/42043#issuecomment-2336711522
Awesome! Many thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org