[airflow] branch main updated: Error handling for when Azure container log cannot be read in properly. (#34627)

[el...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/main by this push:
     new 546c850a43 Error handling for when Azure container log cannot be read in properly. (#34627)
546c850a43 is described below

commit 546c850a43d8b00fafc11e02e63fa5caa56b4c07
Author: Krzysztof Furman <k_...@outlook.com>
AuthorDate: Fri Oct 13 13:05:53 2023 +0100

    Error handling for when Azure container log cannot be read in properly. (#34627)
    
    * Error handling for broken or null Azure logs.
    
    * Updated logging syntax to match rest of the code.
    
    * Moved broken log checking closer to the source to avoid catching more general errors.
    
    * Fixed style issues.
    
    * Changed log exception to log error.
    
    * Changed failure case return type to list. Not an empty list as that ruins normal operation - instead [None.
    
    * Cleaner checking of logs list not being empty
    
    Co-authored-by: Wei Lee <we...@gmail.com>
    
    ---------
    
    Co-authored-by: Wei Lee <we...@gmail.com>
---
 airflow/providers/microsoft/azure/hooks/container_instance.py      | 2 ++
 airflow/providers/microsoft/azure/operators/container_instances.py | 3 +++
 2 files changed, 5 insertions(+)

diff --git a/airflow/providers/microsoft/azure/hooks/container_instance.py b/airflow/providers/microsoft/azure/hooks/container_instance.py
index 34edbf74bb..c8fa67ca59 100644
--- a/airflow/providers/microsoft/azure/hooks/container_instance.py
+++ b/airflow/providers/microsoft/azure/hooks/container_instance.py
@@ -169,6 +169,8 @@ class AzureContainerInstanceHook(AzureBaseHook):
         :return: A list of log messages
         """
         logs = self.connection.containers.list_logs(resource_group, name, name, tail=tail)
+        if logs.content is None:
+            return [None]
         return logs.content.splitlines(True)
 
     def delete(self, resource_group: str, name: str) -> None:
diff --git a/airflow/providers/microsoft/azure/operators/container_instances.py b/airflow/providers/microsoft/azure/operators/container_instances.py
index 8abcb48c13..30b4877d3e 100644
--- a/airflow/providers/microsoft/azure/operators/container_instances.py
+++ b/airflow/providers/microsoft/azure/operators/container_instances.py
@@ -318,6 +318,9 @@ class AzureContainerInstancesOperator(BaseOperator):
                 if state in ["Running", "Terminated", "Succeeded"]:
                     try:
                         logs = self._ci_hook.get_logs(resource_group, name)
+                        if logs and logs[0] is None:
+                            self.log.error("Container log is broken, marking as failed.")
+                            return 1
                         last_line_logged = self._log_last(logs, last_line_logged)
                     except CloudError:
                         self.log.exception(