You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/02/16 13:09:49 UTC

[GitHub] [airflow] alittlesliceoftom opened a new issue #14259: Airflow times out on Long Running Kubernetes Jobs with logging gaps

alittlesliceoftom opened a new issue #14259:
URL: https://github.com/apache/airflow/issues/14259


   This is listed too in other locations:
   
   https://issues.apache.org/jira/browse/AIRFLOW-3534
   https://issues.apache.org/jira/browse/AIRFLOW-4526
   
   Only known fix is to disable log stream: 
   `When disabling the `get_logs` parameter everything works because the log stream is skipped.`
   
   Suggested fix - up the retry count on the below line, or allow this to be configured. 
   https://github.com/apache/airflow/blob/b23fc137812f5eabf7834e07e032915e2a504c17/airflow/kubernetes/pod_launcher.py#L198


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] alittlesliceoftom removed a comment on issue #14259: Airflow times out on Long Running Kubernetes Jobs with logging gaps

Posted by GitBox <gi...@apache.org>.
alittlesliceoftom removed a comment on issue #14259:
URL: https://github.com/apache/airflow/issues/14259#issuecomment-779828195


   )


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] alittlesliceoftom commented on issue #14259: Airflow times out on Long Running Kubernetes Jobs with logging gaps

Posted by GitBox <gi...@apache.org>.
alittlesliceoftom commented on issue #14259:
URL: https://github.com/apache/airflow/issues/14259#issuecomment-779827526


   Happy to raise this as a PR, I.e. changing 
   ` @tenacity.retry(stop=tenacity.stop_after_attempt(3), wait=tenacity.wait_exponential(), reraise=True) `
   to
   ` @tenacity.retry(stop=tenacity.stop_after_attempt(5), wait=tenacity.wait_exponential(), reraise=True) `
   Which I think would solve the issue for most people and make this much more usable, but I'm not used to this code base!


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] alittlesliceoftom commented on issue #14259: Airflow times out on Long Running Kubernetes Jobs with logging gaps

Posted by GitBox <gi...@apache.org>.
alittlesliceoftom commented on issue #14259:
URL: https://github.com/apache/airflow/issues/14259#issuecomment-779828182






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] alittlesliceoftom commented on issue #14259: Airflow times out on Long Running Kubernetes Jobs with logging gaps

Posted by GitBox <gi...@apache.org>.
alittlesliceoftom commented on issue #14259:
URL: https://github.com/apache/airflow/issues/14259#issuecomment-799557185


   Note, I tried this fix and it **didn't work**. The best thread I have found is this one: 
   
   https://github.com/apache/airflow/pull/7428#issuecomment-704994507


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org