You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/10/06 12:38:37 UTC

[GitHub] [airflow] hterik opened a new issue, #26912: Log-tab under grid view is automatically re-fetching completed logs every 3 sec.

hterik opened a new issue, #26912:
URL: https://github.com/apache/airflow/issues/26912

   ### Apache Airflow version
   
   2.4.1
   
   ### What happened
   
   The new inline log-tab under grid view is fantastic.
   What's not so great though, is that it is automatically reloading the logs on the `/api/v1/dags/.../dagRuns/.../taskInstances/.../logs/1` api endpoint every 3 seconds. Same interval as the reload of the grid status it seems. 
   
   This:
   * Makes it difficult for users to scroll in the log panel and to select text in the log panel, because it is replaced all the time
   * Put unnecessary load on the client and the link between client-webserver.
   * Put unnecssary load on the webserver and on the logging-backend, in our case it involves queries to an external Loki server.
   
   This happens even if the TaskLogReader has set `metadata["end_of_log"] = True`
   
   ### What you think should happen instead
   
   Logs should not automatically be reloaded if `end_of_log=True`
   
   For logs which are not at end, some other slower reload or more incremental query/streaming is preferred.
   
   ### How to reproduce
   
   _No response_
   
   ### Operating System
   
   Debian GNU/Linux 11 (bullseye)
   
   ### Versions of Apache Airflow Providers
   
   apache-airflow-providers-celery==3.0.0
   apache-airflow-providers-cncf-kubernetes==4.4.0
   apache-airflow-providers-common-sql==1.1.0
   apache-airflow-providers-docker==3.2.0
   apache-airflow-providers-ftp==3.1.0
   apache-airflow-providers-http==4.0.0
   apache-airflow-providers-imap==3.0.0
   apache-airflow-providers-postgres==5.2.0
   apache-airflow-providers-sqlite==3.1.0
   
   
   ### Deployment
   
   Other Docker-based deployment
   
   ### Deployment details
   
   _No response_
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] bbovenzi closed issue #26912: Log-tab under grid view is automatically re-fetching completed logs every 3 sec.

Posted by GitBox <gi...@apache.org>.
bbovenzi closed issue #26912: Log-tab under grid view is automatically re-fetching completed logs every 3 sec.
URL: https://github.com/apache/airflow/issues/26912


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] bbovenzi commented on issue #26912: Log-tab under grid view is automatically re-fetching completed logs every 3 sec.

Posted by GitBox <gi...@apache.org>.
bbovenzi commented on issue #26912:
URL: https://github.com/apache/airflow/issues/26912#issuecomment-1275255489

   Oh we should probably include `end_of_log` in the REST API response.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org