You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/01/29 04:14:27 UTC

[GitHub] [airflow] jedcunningham opened a new pull request #13969: Docs: Fix FAQ on scheduler latency

jedcunningham opened a new pull request #13969:
URL: https://github.com/apache/airflow/pull/13969


   `parsing_processes` no longer has an impact on scheduler latency, so reword this section and point folks to running more than 1 scheduler.
   
   Also, fix typo in the "promote new flags" output.
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] kaxil commented on a change in pull request #13969: Docs: Fix FAQ on scheduler latency

Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #13969:
URL: https://github.com/apache/airflow/pull/13969#discussion_r566951640



##########
File path: docs/apache-airflow/faq.rst
##########
@@ -205,11 +205,8 @@ This means ``explicit_defaults_for_timestamp`` is disabled in your mysql server
 How to reduce airflow dag scheduling latency in production?
 -----------------------------------------------------------
 
-- ``parsing_processes``: Scheduler will spawn multiple threads in parallel to parse dags.
-  This is controlled by ``parsing_processes`` with default value of 2.
-  User should increase this value to a larger value (e.g numbers of cpus where scheduler runs + 1) in production.
-- If you're using Airflow 1.10.x, consider moving to Airflow 2, which has reduced dag scheduling latency dramatically,
-  and allows for running multiple schedulers.
+Airflow 2 has low DAG scheduling latency out of the box (particularly when compare with Airflow 1.10.x),

Review comment:
       ```suggestion
   Airflow 2 has low DAG scheduling latency out of the box (particularly when compared with Airflow 1.10.x),
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] kaxil commented on pull request #13969: Docs: Fix FAQ on scheduler latency

Posted by GitBox <gi...@apache.org>.
kaxil commented on pull request #13969:
URL: https://github.com/apache/airflow/pull/13969#issuecomment-769970224


   Thanks for the PR @jedcunningham 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] kaxil merged pull request #13969: Docs: Fix FAQ on scheduler latency

Posted by GitBox <gi...@apache.org>.
kaxil merged pull request #13969:
URL: https://github.com/apache/airflow/pull/13969


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] jedcunningham commented on a change in pull request #13969: Docs: Fix FAQ on scheduler latency

Posted by GitBox <gi...@apache.org>.
jedcunningham commented on a change in pull request #13969:
URL: https://github.com/apache/airflow/pull/13969#discussion_r566979652



##########
File path: docs/apache-airflow/faq.rst
##########
@@ -205,11 +205,8 @@ This means ``explicit_defaults_for_timestamp`` is disabled in your mysql server
 How to reduce airflow dag scheduling latency in production?
 -----------------------------------------------------------
 
-- ``parsing_processes``: Scheduler will spawn multiple threads in parallel to parse dags.
-  This is controlled by ``parsing_processes`` with default value of 2.
-  User should increase this value to a larger value (e.g numbers of cpus where scheduler runs + 1) in production.
-- If you're using Airflow 1.10.x, consider moving to Airflow 2, which has reduced dag scheduling latency dramatically,
-  and allows for running multiple schedulers.
+Airflow 2 has low DAG scheduling latency out of the box (particularly when compare with Airflow 1.10.x),

Review comment:
       🤦‍♂️ thanks!




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org