You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/09/06 13:34:38 UTC
[GitHub] [airflow] jens-scheffler-bosch commented on a diff in pull request #26169: Auto tail file logs in Web UI
jens-scheffler-bosch commented on code in PR #26169:
URL: https://github.com/apache/airflow/pull/26169#discussion_r963716278
##########
airflow/providers/microsoft/azure/log/wasb_task_handler.py:
##########
@@ -127,7 +129,7 @@ def _read(self, ti, try_number: int, metadata: Optional[str] = None) -> Tuple[st
log = f'*** Reading remote log from {remote_loc}.\n{remote_log}\n'
return log, {'end_of_log': True}
else:
- return super()._read(ti, try_number)
+ return super()._read(ti, try_number, metadata)
Review Comment:
Thanks @uranusjr for the feedback! I made the changes in the provider log handlers in order to have a consistent status. Passing `metadata` now with this PR should only introduce consistency - as the client passes the `metadata` but current providers do not pass the `metadata` to the respective super-class consistently when running into the fallback.
I double checked and the super class method interface is consistent since at least v2.0.0 so **it is not a breaking change** but a matter of consistency. Not passing `metadata` is and will further be supported.
If you insist on reverting the provider package adjustments then I see only two options:
1) Current log position can not be passed from web UI client to fallback provider class - in this case log would be tailed for running tasks always from the start (as client's last log position is known to super-class (file_task_handler.py)). This would also be the effect when using an older provider package with an Airflow version with this change applied.
Means all users using not the file based log provider and taking a look to UI for a running TaskInstance will over-and-over tail the logs. Not a problem if task is not running anymore or provider package returns w/o super class code.
2) We would need to adjust the interface between Web UI client and `file_task_handler.py` back-end to explicitly enable tailing for some use cases only. Then it would be a breaking change on client side (manageable) and provider packages will not benefit from super-class tailing capabilities.
Means all users with standard file task handler will have a benefit of tailing and every provider package must implement log tailing individually. No tailing also in file fallback.
Please let me know your opinion.
(Overall I have the feeling that the log handling/provider feature detail is very badly documented, took a long time of code reading to understand details and fields... but I did not want to propose a major re-factoring in this area just for this feature)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org