You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/09/08 07:16:51 UTC

[GitHub] [airflow] agonen opened a new issue #10791: Async pod launch 

agonen opened a new issue #10791:
URL: https://github.com/apache/airflow/issues/10791


   we have a long-running task in airflow (5hr and more)
   
   I would like the following abilities : 
   1. when waiting for the pod starts or finishes, the log gets prints every 2 sec (this creates a lot of logging lines  ) can we expose this as a parameter ?
   2. can we have a parameter to fire an forget (after the pod started running) and the monitor the status of the task using sensor 
   this can free the worker and reduce resources 
   
   https://github.com/apache/airflow/blob/10ce31127f1ff87176158935925afce46a989917/airflow/kubernetes/pod_launcher.py#L140


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] dimberman commented on issue #10791: Async pod launch

Posted by GitBox <gi...@apache.org>.
dimberman commented on issue #10791:
URL: https://github.com/apache/airflow/issues/10791#issuecomment-705083218


   Hi @agonen, would you like to write this feature in? Would be glad to help you through the development/PR process :)


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] dimberman commented on issue #10791: Async pod launch

Posted by GitBox <gi...@apache.org>.
dimberman commented on issue #10791:
URL: https://github.com/apache/airflow/issues/10791#issuecomment-705084098


   Also worth noting that this PR will ensure that all real pod logs will show up in the airflow UI if you set that flag https://github.com/apache/airflow/pull/11325


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #10791: Async pod launch

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #10791:
URL: https://github.com/apache/airflow/issues/10791#issuecomment-688670439


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] agonen commented on issue #10791: Async pod launch

Posted by GitBox <gi...@apache.org>.
agonen commented on issue #10791:
URL: https://github.com/apache/airflow/issues/10791#issuecomment-696277318


   ?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] agonen commented on issue #10791: Async pod launch

Posted by GitBox <gi...@apache.org>.
agonen commented on issue #10791:
URL: https://github.com/apache/airflow/issues/10791#issuecomment-696277318


   ?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] agonen commented on issue #10791: Async pod launch

Posted by GitBox <gi...@apache.org>.
agonen commented on issue #10791:
URL: https://github.com/apache/airflow/issues/10791#issuecomment-691685228


   ?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org