You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/08/02 09:38:53 UTC

[GitHub] [airflow] potiuk commented on issue #17372: Dag Serialization Speed is so slow

potiuk commented on issue #17372:
URL: https://github.com/apache/airflow/issues/17372#issuecomment-890881685


   This problem is already solved. You should run multiple schedulers.
   
   One of the reasons Airflow HA scheduling is implemented is to allow scaling of serialization part. You can read more about HA scheduler, how it works and it's scalability characteristics in this blog post from Astronomer,
   
   https://www.astronomer.io/blog/airflow-2-scheduler#:~:text=As%20part%20of%20Apache%20Airflow,once%20their%20dependencies%20are%20met.
   
   Please try the same with multiple schedulers and let us know your experience.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org