You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ep...@apache.org on 2022/07/05 14:47:22 UTC
[airflow] 03/16: Fix doc description of [core] parallelism config setting (#23768)
This is an automated email from the ASF dual-hosted git repository.
ephraimanierobi pushed a commit to branch v2-3-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 6ca7f3a3178e99c7c2742a8d32430b4899a7d3fa
Author: Chris Redekop <32...@users.noreply.github.com>
AuthorDate: Wed May 18 01:08:33 2022 -0600
Fix doc description of [core] parallelism config setting (#23768)
(cherry picked from commit 23037503b51cd8ea2acd2214116fc1693eb98cb4)
---
airflow/config_templates/config.yml | 7 ++++---
airflow/config_templates/default_airflow.cfg | 7 ++++---
2 files changed, 8 insertions(+), 6 deletions(-)
diff --git a/airflow/config_templates/config.yml b/airflow/config_templates/config.yml
index 9da14d48aa..f7409373c2 100644
--- a/airflow/config_templates/config.yml
+++ b/airflow/config_templates/config.yml
@@ -62,9 +62,10 @@
default: "SequentialExecutor"
- name: parallelism
description: |
- This defines the maximum number of task instances that can run concurrently in Airflow
- regardless of scheduler count and worker count. Generally, this value is reflective of
- the number of task instances with the running state in the metadata database.
+ This defines the maximum number of task instances that can run concurrently per scheduler in
+ Airflow, regardless of the worker count. Generally this value, multiplied by the number of
+ schedulers in your cluster, is the maximum number of task instances with the running
+ state in the metadata database.
version_added: ~
type: string
example: ~
diff --git a/airflow/config_templates/default_airflow.cfg b/airflow/config_templates/default_airflow.cfg
index f58b688a6b..6591f82dc0 100644
--- a/airflow/config_templates/default_airflow.cfg
+++ b/airflow/config_templates/default_airflow.cfg
@@ -53,9 +53,10 @@ default_timezone = utc
# full import path to the class when using a custom executor.
executor = SequentialExecutor
-# This defines the maximum number of task instances that can run concurrently in Airflow
-# regardless of scheduler count and worker count. Generally, this value is reflective of
-# the number of task instances with the running state in the metadata database.
+# This defines the maximum number of task instances that can run concurrently per scheduler in
+# Airflow, regardless of the worker count. Generally this value, multiplied by the number of
+# schedulers in your cluster, is the maximum number of task instances with the running
+# state in the metadata database.
parallelism = 32
# The maximum number of task instances allowed to run concurrently in each DAG. To calculate