[GitHub] [airflow] potiuk commented on issue #17104: Resource usage in Airflow monitor metrix

[GitBox <gi...@apache.org>]

potiuk commented on issue #17104:
URL: https://github.com/apache/airflow/issues/17104#issuecomment-883580028


   Resource monitoring is not really part of airflow -it's outside of the domain of Airflow. You could bring that under discussion at the devlist if you want to ask for opinion of the community https://airflow.apache.org/community/ but this is whole new big feature that would drag the focus of Airflow from being great at "scheduling" and not "resource management"
   
   As mentioned by @andormarkus K8S provides good information already. Regarding the Celery, it makes no sense to have "per task" metrics, because Celery reuses running workers to run multiple tasks and they share resources, you can have all the insights you want into running Celery via Flower (when you use the Official Helm Chart from the Airflow community, flower is automatically enabled when you run Celery or CeleryKubernetesExecutor) - see https://airflow.apache.org/docs/helm-chart/stable/index.html
   
   For now this one is "close/won't do" - but again, if you have good arguments you can start thread at the devlist.
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org