[GitHub] [airflow] eladkal opened a new pull request, #28831: Update scheduler docs about low priority tasks

[GitBox <gi...@apache.org>]

eladkal opened a new pull request, #28831:
URL: https://github.com/apache/airflow/pull/28831

   Gathered insights from discussion in https://github.com/apache/airflow/issues/26933 into a paragraph in scheduler docs to clarify why sometimes low priority tasks are scheduled before high priority tasks
   
   <!--
   Thank you for contributing! Please make sure that your code changes
   are covered with tests. And in case of new features or big changes
   remember to adjust the documentation.
   
   Feel free to ping committers for the review!
   
   In case of an existing issue, reference it using one of the following:
   
   closes: #ISSUE
   related: #ISSUE
   
   How to write a good git commit message:
   http://chris.beams.io/posts/git-commit/
   -->
   
   ---
   **^ Add meaningful description above**
   
   Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] ashb merged pull request #28831: Update scheduler docs about low priority tasks

[GitBox <gi...@apache.org>]

ashb merged PR #28831:
URL: https://github.com/apache/airflow/pull/28831


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] uranusjr commented on a diff in pull request #28831: Update scheduler docs about low priority tasks

[GitBox <gi...@apache.org>]

uranusjr commented on code in PR #28831:
URL: https://github.com/apache/airflow/pull/28831#discussion_r1066682250


##########
docs/apache-airflow/administration-and-deployment/scheduler.rst:
##########
@@ -59,6 +59,14 @@ In the UI, it appears as if Airflow is running your tasks a day **late**
 
     You should refer to :doc:`../core-concepts/dag-run` for details on scheduling a DAG.
 
+.. note::
+    The scheduler is designed for high throughput. This is an informed design decision to achieve scheduling
+    tasks as soon as possible. The scheduler checks how many free slots available in a pool and schedule at most that number of tasks instances in one iteration.
+    This means that task priority will only come in to effect when there are more scheduled tasks
+    waiting than the queue slots. Thus there can be cases where low priority tasks will be schedule before high priority tasks if they share the same batch.
+    For more read about that you can reference this `Github discussion <https://github.com/apache/airflow/discussions/28809>`_.

Review Comment:
   ```suggestion
       For more read about that you can reference `this GitHub discussion <https://github.com/apache/airflow/discussions/28809>`__.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org