You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/01/14 22:42:41 UTC
[GitHub] [airflow] grepthat opened a new pull request #13682: fix k8s scheduler liveness probe for multi scheduler setup
grepthat opened a new pull request #13682:
URL: https://github.com/apache/airflow/pull/13682
Hej together, this should fix the liveness probe for a multi scheduler k8s setup. Currently the liveness probe fails due to `BaseJob::most_recent_job` reporting a random scheduler from the running schedulers which causes `job.hostname == get_hostname()` to fail in the liveness probe. This should fix #13677
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #13682: fix k8s scheduler liveness probe for multi scheduler setup
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #13682:
URL: https://github.com/apache/airflow/pull/13682#discussion_r558584448
##########
File path: airflow/jobs/base_job.py
##########
@@ -117,7 +117,7 @@ def most_recent_job(cls, session=None) -> Optional['BaseJob']:
:param session: Database session
:rtype: BaseJob or None
"""
- return session.query(cls).order_by(cls.latest_heartbeat.desc()).limit(1).first()
+ return session.query(cls).filter(cls.hostname == self.hostname).order_by(cls.latest_heartbeat.desc()).limit(1).first()
Review comment:
This change will prevent our API from returning the correct state.
https://github.com/apache/airflow/blob/b007fc33d481f0f1341d1e1e4cba719a5fe6580d/airflow/www/views.py#L418
https://github.com/apache/airflow/blob/349b0811c3022605426ba57d30936240a7c2848a/airflow/api_connexion/endpoints/health_endpoint.py#L30
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #13682: fix k8s scheduler liveness probe for multi scheduler setup
Posted by GitBox <gi...@apache.org>.
kaxil commented on pull request #13682:
URL: https://github.com/apache/airflow/pull/13682#issuecomment-761260391
closed in favor of https://github.com/apache/airflow/pull/13705
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13682: fix k8s scheduler liveness probe for multi scheduler setup
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13682:
URL: https://github.com/apache/airflow/pull/13682#issuecomment-760530857
[The Workflow run](https://github.com/apache/airflow/actions/runs/486572015) is cancelling this PR. Building images for the PR has failed. Follow the the workflow link to check the reason.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil closed pull request #13682: fix k8s scheduler liveness probe for multi scheduler setup
Posted by GitBox <gi...@apache.org>.
kaxil closed pull request #13682:
URL: https://github.com/apache/airflow/pull/13682
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on pull request #13682: fix k8s scheduler liveness probe for multi scheduler setup
Posted by GitBox <gi...@apache.org>.
kaxil commented on pull request #13682:
URL: https://github.com/apache/airflow/pull/13682#issuecomment-761236297
https://github.com/apache/airflow/pull/13705 will fix it correctly. We at Astronomer have been using that code since 6 months
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org