You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/03/29 16:05:13 UTC
[GitHub] [airflow] marclamberti edited a comment on issue #14261: Airflow Scheduler liveness probe crashing (version 2.0)
marclamberti edited a comment on issue #14261:
URL: https://github.com/apache/airflow/issues/14261#issuecomment-808933078
@kaxil I get the same issue in local with KinD. Every 5 mins as described in scheduler-deployment.yaml
```
# If the scheduler stops heartbeating for 5 minutes (10*30s) kill the
# scheduler and let Kubernetes restart it
livenessProbe:
failureThreshold: 10
periodSeconds: 30
exec:
command:
- python
- -Wignore
- -c
- |
import os
os.environ['AIRFLOW__CORE__LOGGING_LEVEL'] = 'ERROR'
os.environ['AIRFLOW__LOGGING__LOGGING_LEVEL'] = 'ERROR'
from airflow.jobs.scheduler_job import SchedulerJob
from airflow.utils.db import create_session
from airflow.utils.net import get_hostname
import sys
with create_session() as session:
job = session.query(SchedulerJob).filter_by(hostname=get_hostname()).order_by(
SchedulerJob.latest_heartbeat.desc()).limit(1).first()
sys.exit(0 if job.is_alive() else 1)
```
I ran the script in the scheduler pod and it returns True. However, it still continues to fail. I didn't figure out yet why is that.
Even with the suggestions of @stoiandl it doesn't work
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org