You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/07/24 05:12:21 UTC
[GitHub] [airflow] KevinOrtman commented on issue #11085: Airflow Elasticsearch configuration log output does not contain required elements
KevinOrtman commented on issue #11085:
URL: https://github.com/apache/airflow/issues/11085#issuecomment-886001470
I think @melle2 did a great job of describing this issue. #6159 effectively broke end-of-task-log detection for ELS logging.
We're using the `json_format = True` flag and our ingestor forwards each correctly formatted json fragment to ELS as-is. The problem with #6159 is that it simply dumps the end_of_log marker to the output stream without json decoration or any fields needed for indexing. The ingestor correctly ignores the end_of_log marker because it is not json, and the marker is never forwarded to ELS, thus a document is never indexed with the end_of_log marker.
Because a document with the end_of_log marker never exists, airflow-webserver will always return `end_of_log: false` and the browser goes into an XHR request/response loop indefinitely.
When a properly formed document with the end_of_log marker is manually indexed, as @melle2 did in their Feb 11 comment, the XHR request returns `end_of_log: true` and all is well.
In short, the end_of_log marker should be emitted as a log record.
I have tested this behavior in 2.1.0 and 2.1.2, and I'm happy to open a PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org