[GitHub] [airflow] jedcunningham commented on a change in pull request #17649: Do not fail KubernetesPodOperator tasks if log following fails

[GitBox <gi...@apache.org>]

jedcunningham commented on a change in pull request #17649:
URL: https://github.com/apache/airflow/pull/17649#discussion_r690623986



##########
File path: airflow/providers/cncf/kubernetes/utils/pod_launcher.py
##########
@@ -217,7 +223,7 @@ def base_container_is_running(self, pod: V1Pod):
             return False
         return status.state.running is not None
 
-    @tenacity.retry(stop=tenacity.stop_after_attempt(3), wait=tenacity.wait_exponential(), reraise=True)
+    @tenacity.retry(stop=tenacity.stop_after_attempt(4), wait=tenacity.wait_exponential(), reraise=True)

Review comment:
       How'd you land on 4 attempts? Do we even need to bump this with the other change?

##########
File path: airflow/providers/cncf/kubernetes/utils/pod_launcher.py
##########
@@ -143,12 +143,21 @@ def monitor_pod(self, pod: V1Pod, get_logs: bool) -> Tuple[State, V1Pod, Optiona
             read_logs_since_sec = None
             last_log_time = None
             while True:
-                logs = self.read_pod_logs(pod, timestamps=True, since_seconds=read_logs_since_sec)
-                for line in logs:
-                    timestamp, message = self.parse_log_line(line.decode('utf-8'))
-                    self.log.info(message)
-                    if timestamp:
-                        last_log_time = timestamp
+                try:
+                    logs = self.read_pod_logs(pod, timestamps=True, since_seconds=read_logs_since_sec)
+                    for line in logs:
+                        timestamp, message = self.parse_log_line(line.decode('utf-8'))
+                        self.log.info(message)
+                        if timestamp:
+                            last_log_time = timestamp
+                except Exception as e:

Review comment:
       This is pretty broad, maybe we only look for `TimeoutError`'s?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org