You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/09/21 20:31:52 UTC
[GitHub] [airflow] nsAstro opened a new issue, #26574: Some liveness checks don't actually check process
nsAstro opened a new issue, #26574:
URL: https://github.com/apache/airflow/issues/26574
### Description
Current liveness check probes use the 'airflow jobs' command which directly queries the backend DB as opposed to actually querying an endpoint or checking the status of the process itself.
e.g. Triggerer liveness probe
```
exec [sh -c CONNECTION_CHECK_MAX_COUNT=0 AIRFLOW__LOGGING__LOGGING_LEVEL=ERROR exec /entrypoint \
airflow jobs check --job-type TriggererJob --hostname $(hostname)] delay=10s timeout=20s period=60s #success=1 #failure=5
```
This command only checks the backend DB to see if there are any jobs. Additionally, the exit code is always 0 regardless of how many jobs there are. Ideally, the liveness check is done by querying some endpoint on the triggerer to see if it's still running.
### Use case/motivation
Would like a liveness check that is more aware of the process rather than the stored state
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] TruptiM18 commented on issue #26574: Some liveness checks don't actually check process
Posted by GitBox <gi...@apache.org>.
TruptiM18 commented on issue #26574:
URL: https://github.com/apache/airflow/issues/26574#issuecomment-1269395620
@potiuk, Can I take this task?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] amoghrajesh commented on issue #26574: Some liveness checks don't actually check process
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on issue #26574:
URL: https://github.com/apache/airflow/issues/26574#issuecomment-1563848176
@uranusjr @TruptiM18 if you aren't working on this, I would like to try my hand at it.
I see that the liveness checks can be improved for the scheduler and the triggerer here. Would like to hear what kind of liveness probe we would rather prefer. Calling an endpoint instead? Do we have ping for triggerer?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #26574: Some liveness checks don't actually check process
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on issue #26574:
URL: https://github.com/apache/airflow/issues/26574#issuecomment-1563947538
Please feel free.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] TruptiM18 commented on issue #26574: Some liveness checks don't actually check process
Posted by GitBox <gi...@apache.org>.
TruptiM18 commented on issue #26574:
URL: https://github.com/apache/airflow/issues/26574#issuecomment-1262536005
@potiuk, I would like to take this task. Can you please assign it to me?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] TruptiM18 commented on issue #26574: Some liveness checks don't actually check process
Posted by GitBox <gi...@apache.org>.
TruptiM18 commented on issue #26574:
URL: https://github.com/apache/airflow/issues/26574#issuecomment-1272917450
Thanks! I have started working on it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] amoghrajesh commented on issue #26574: Some liveness checks don't actually check process
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on issue #26574:
URL: https://github.com/apache/airflow/issues/26574#issuecomment-1563848185
@uranusjr @TruptiM18 if you aren't working on this, I would like to try my hand at it.
I see that the liveness checks can be improved for the scheduler and the triggerer here. Would like to hear what kind of liveness probe we would rather prefer. Calling an endpoint instead? Do we have ping for triggerer?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #26574: Some liveness checks don't actually check process
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #26574:
URL: https://github.com/apache/airflow/issues/26574#issuecomment-1254197636
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #26574: Some liveness checks don't actually check process
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #26574:
URL: https://github.com/apache/airflow/issues/26574#issuecomment-1255083268
Marked it as good first issue, this is a good idea, and hopefully someone might improve it. BTW. @nsAstro - If you have ideas how to improve - you are most welcome to make PR #- this is an easy way to become one of the ~ 2200 contributors. Otherwise it will just have to wait for someone to pick it up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] amoghrajesh commented on issue #26574: Some liveness checks don't actually check process
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on issue #26574:
URL: https://github.com/apache/airflow/issues/26574#issuecomment-1564019679
Thanks. Any hints or clues on what we are looking to have as new liveness probes? @uranusjr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org