[airflow] branch v2-5-test updated: Update scheduler docs about low priority tasks (#28831)

[pi...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

pierrejeambrun pushed a commit to branch v2-5-test
in repository https://gitbox.apache.org/repos/asf/airflow.git


The following commit(s) were added to refs/heads/v2-5-test by this push:
     new 192fd4b540 Update scheduler docs about low priority tasks (#28831)
192fd4b540 is described below

commit 192fd4b540c2a8ddfb3ffce66b024eaab356f3fa
Author: eladkal <45...@users.noreply.github.com>
AuthorDate: Wed Jan 11 13:25:31 2023 +0200

    Update scheduler docs about low priority tasks (#28831)
    
    Gathered insights from discussion in https://github.com/apache/airflow/issues/26933 into a paragraph in scheduler docs to clarify why sometimes low priority tasks are scheduled before high priority tasks
    
    (cherry picked from commit 493b433ad57088a5f5cabc466c949445e500b4c1)
---
 docs/apache-airflow/concepts/scheduler.rst | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/docs/apache-airflow/concepts/scheduler.rst b/docs/apache-airflow/concepts/scheduler.rst
index 10057c386d..b07c5c16c9 100644
--- a/docs/apache-airflow/concepts/scheduler.rst
+++ b/docs/apache-airflow/concepts/scheduler.rst
@@ -59,6 +59,14 @@ In the UI, it appears as if Airflow is running your tasks a day **late**
 
     You should refer to :doc:`/dag-run` for details on scheduling a DAG.
 
+.. note::
+    The scheduler is designed for high throughput. This is an informed design decision to achieve scheduling
+    tasks as soon as possible. The scheduler checks how many free slots available in a pool and schedule at most that number of tasks instances in one iteration.
+    This means that task priority will only come in to effect when there are more scheduled tasks
+    waiting than the queue slots. Thus there can be cases where low priority tasks will be schedule before high priority tasks if they share the same batch.
+    For more read about that you can reference `this GitHub discussion <https://github.com/apache/airflow/discussions/28809>`__.
+
+
 DAG File Processing
 -------------------