(airflow) branch main updated: Produce proper exit status in case image has been built with timeout (#35282)

[po...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/main by this push:
     new 92c2c3f1ee Produce proper exit status in case image has been built with timeout (#35282)
92c2c3f1ee is described below

commit 92c2c3f1ee2df27e0a2287b60d1d1396753d8a41
Author: Jarek Potiuk <ja...@potiuk.com>
AuthorDate: Tue Oct 31 10:32:01 2023 +0100

    Produce proper exit status in case image has been built with timeout (#35282)
    
    When we build the image with timeout, we fork the process and set alarm
    and create a new process group in order to be sure that all the parallel
    build processes can be killed easily with sending termination signal to
    process group on timeout. In order to wait for the forked process
    to complete we used waitpid, but we did not handle the status code
    properly, so if the build failed, we returned with 0 exit code.
    
    This had the side effect that "Build CI image" did not fail, instead
    the next step (generating source providers failed instead and it was
    not obvious that the CI image build failing was the root cause.
    
    This PR properly retrieves the wait status and converts it to
    exit code - since we are still supporting Python 3.8 this is still
    done using a bit nasty set of if statements - only in Python 3.9 we
    have `os.waitstatus_to_exitcode` method to do it for us, but we cannot
    use the method yet.
---
 .../airflow_breeze/commands/ci_image_commands.py   | 23 ++++++++++++++++++++--
 1 file changed, 21 insertions(+), 2 deletions(-)

diff --git a/dev/breeze/src/airflow_breeze/commands/ci_image_commands.py b/dev/breeze/src/airflow_breeze/commands/ci_image_commands.py
index f5ab0b2d0f..cd7c915d8b 100644
--- a/dev/breeze/src/airflow_breeze/commands/ci_image_commands.py
+++ b/dev/breeze/src/airflow_breeze/commands/ci_image_commands.py
@@ -206,6 +206,20 @@ def kill_process_group(build_process_group_id: int):
         pass
 
 
+def get_exitcode(status: int) -> int:
+    # In Python 3.9+ we will be able to use
+    # os.waitstatus_to_exitcode(status) - see https://github.com/python/cpython/issues/84275
+    # but until then we need to do this ugly conversion
+    if os.WIFSIGNALED(status):
+        return -os.WTERMSIG(status)
+    elif os.WIFEXITED(status):
+        return os.WEXITSTATUS(status)
+    elif os.WIFSTOPPED(status):
+        return -os.WSTOPSIG(status)
+    else:
+        return 1
+
+
 @ci_image.command(name="build")
 @option_python
 @option_run_in_parallel
@@ -277,8 +291,13 @@ def build(
             atexit.register(kill_process_group, pid)
             signal.signal(signal.SIGALRM, handler)
             signal.alarm(build_timeout_minutes * 60)
-            os.waitpid(pid, 0)
-            return
+            child_pid, status = os.waitpid(pid, 0)
+            exit_code = get_exitcode(status)
+            if exit_code:
+                get_console().print(f"[error]Exiting with exit code {exit_code}")
+            else:
+                get_console().print(f"[success]Exiting with exit code {exit_code}")
+            sys.exit(exit_code)
         else:
             # turn us into a process group leader
             os.setpgid(0, 0)