You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "shashbha14 (via GitHub)" <gi...@apache.org> on 2026/03/09 09:59:12 UTC
[PR] Fix UnicodeDecodeError crash in KubernetesPodOperator deferrable log fetching [airflow]
shashbha14 opened a new pull request, #63176:
URL: https://github.com/apache/airflow/pull/63176
When running KubernetesPodOperator in deferrable mode, pod logs that contain
non-UTF-8 bytes (truncated multi-byte sequences, tqdm progress bars, binary
output, etc.) caused a UnicodeDecodeError to propagate up from
'AsyncKubernetesHook.read_logs()'. Since only 'HTTPError' was being caught,
this bubbled up as an unhandled exception and crashed the task entirely.
The fix catches 'UnicodeDecodeError' in 'read_logs', logs a warning with the
exception details, and returns an empty list for that interval. The next
polling cycle will attempt to fetch logs again, so nothing is lost except
one log window — much better than the whole task failing.
Also added a unit test that mocks 'read_namespaced_pod_log' to raise
'UnicodeDecodeError' and asserts the method returns '[]' without raising.
Fixes #63173
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix UnicodeDecodeError crash in KubernetesPodOperator deferrable log fetching [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun closed pull request #63176: Fix UnicodeDecodeError crash in KubernetesPodOperator deferrable log fetching
URL: https://github.com/apache/airflow/pull/63176
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix UnicodeDecodeError crash in KubernetesPodOperator deferrable log fetching [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #63176:
URL: https://github.com/apache/airflow/pull/63176#discussion_r2908193283
##########
providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/hooks/kubernetes.py:
##########
@@ -1004,6 +1004,13 @@ async def read_logs(
)
logs_list: list[str] = logs.splitlines()
return logs_list
+ except UnicodeDecodeError as e:
Review Comment:
Maybe, how about amking it similar like https://github.com/apache/airflow/pull/62632/ ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix UnicodeDecodeError crash in KubernetesPodOperator deferrable log fetching [airflow]
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on PR #63176:
URL: https://github.com/apache/airflow/pull/63176#issuecomment-4031591636
Closing, PRs from this users are ignoring review comments, not fixing underlying problems they report to fix or complete AI hallucination.
Please refrain from opening such PRs, it's not helping the project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix UnicodeDecodeError crash in KubernetesPodOperator deferrable log fetching [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #63176:
URL: https://github.com/apache/airflow/pull/63176#discussion_r2907862081
##########
providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/hooks/kubernetes.py:
##########
@@ -1004,6 +1004,13 @@ async def read_logs(
)
logs_list: list[str] = logs.splitlines()
return logs_list
+ except UnicodeDecodeError as e:
Review Comment:
Thanks for proposing a fix - we also saw this recently with Airflow 3.
Do you also see an alternative way to clean the bad characters and keep with this at least some parts of the log message? Because otherwise if handled like this it only will be visible in server logs and not user facing, which then is hard to track down if a user calls and tells that "strange, some logs are missing".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org