You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ramitkataria (via GitHub)" <gi...@apache.org> on 2026/05/27 16:33:28 UTC
[PR] Add docs for multi-team triggerer support [airflow]
ramitkataria opened a new pull request, #67608:
URL: https://github.com/apache/airflow/pull/67608
Add a `Team-scoped Triggerer` section to the multi-team documentation covering the `--team-name` CLI argument, behavior for team-scoped vs global triggerers, interaction with `--queues`, and an example deployment.
Add a tip in the deferring docs pointing users toward `--team-name` as the native multi-team solution alongside the existing `--queues` docs.
Remove `Async support` from the `Work in Progress` list since team-scoped triggerer support is now implemented/in PR.
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
<!--
Thank you for contributing!
Please provide above a brief description of the changes made in this pull request.
Write a good git commit message following this guide: http://chris.beams.io/posts/git-commit/
Please make sure that your code changes are covered with tests.
And in case of new features or big changes remember to adjust the documentation.
Feel free to ping (in general) for the review if you do not see reaction for a few days
(72 Hours is the minimum reaction time you can expect from volunteers) - we sometimes miss notifications.
In case of an existing issue, reference it using one of the following:
* closes: #ISSUE
* related: #ISSUE
-->
---
##### Was generative AI tooling used to co-author this PR?
<!--
If generative AI tooling has been used in the process of authoring this PR, please
change below checkbox to `[X]` followed by the name of the tool, uncomment the "Generated-by".
-->
- [ ] Yes (please specify the tool below)
<!--
Generated-by: [Tool Name] following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
-->
---
* Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add docs for multi-team triggerer support [airflow]
Posted by "ramitkataria (via GitHub)" <gi...@apache.org>.
ramitkataria commented on code in PR #67608:
URL: https://github.com/apache/airflow/pull/67608#discussion_r3350218993
##########
airflow-core/docs/core-concepts/multi-team.rst:
##########
@@ -503,6 +504,61 @@ When Multi-Team mode is enabled, the scheduler performs additional logic to dete
teams share the same metadata database and common Airflow infrastructure. For absolutely strict security
requirements, consider separate Airflow deployments.
+.. _multi-team-triggerer:
+
+Team-scoped Triggerer
+---------------------
+
+When Multi-Team mode is enabled, the triggerer can be scoped to a specific team using the ``--team-name`` CLI argument. A team-scoped triggerer only processes deferred tasks (triggers) belonging to that team's Dags. This allows teams to run isolated triggerer instances with independent capacity and failure domains.
Review Comment:
Yes, thanks for the catch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add docs for multi-team triggerer support [airflow]
Posted by "o-nikolas (via GitHub)" <gi...@apache.org>.
o-nikolas merged PR #67608:
URL: https://github.com/apache/airflow/pull/67608
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add docs for multi-team triggerer support [airflow]
Posted by "ramitkataria (via GitHub)" <gi...@apache.org>.
ramitkataria commented on code in PR #67608:
URL: https://github.com/apache/airflow/pull/67608#discussion_r3350212325
##########
airflow-core/docs/core-concepts/multi-team.rst:
##########
@@ -503,6 +504,61 @@ When Multi-Team mode is enabled, the scheduler performs additional logic to dete
teams share the same metadata database and common Airflow infrastructure. For absolutely strict security
requirements, consider separate Airflow deployments.
+.. _multi-team-triggerer:
+
+Team-scoped Triggerer
+---------------------
+
+When Multi-Team mode is enabled, the triggerer can be scoped to a specific team using the ``--team-name`` CLI argument. A team-scoped triggerer only processes deferred tasks (triggers) belonging to that team's Dags. This allows teams to run isolated triggerer instances with independent capacity and failure domains.
+
+Configuration
+^^^^^^^^^^^^^
+
+Start a team-scoped triggerer by passing ``--team-name``:
+
+.. code-block:: bash
+
+ # Triggerer for team_a only
+ airflow triggerer --team-name team_a
+
+ # Triggerer for team_b only
+ airflow triggerer --team-name team_b
+
+ # Global triggerer — processes triggers from Dags with no team association
+ airflow triggerer
+
+Startup validation ensures that ``core.multi_team`` is enabled and the specified team exists in the database.
+
+Behavior
+^^^^^^^^
+
+- **Team-scoped triggerer** (``--team-name team_x``): Only picks up triggers whose originating Dag belongs to a bundle mapped to ``team_x``.
+- **Global triggerer** (no ``--team-name``): Only picks up triggers whose originating Dag belongs to a bundle with no team assignment.
+- **Multi-Team disabled** (``core.multi_team = False``): ``--team-name`` is rejected. No filtering occurs and all triggerers process all triggers (existing behavior).
+
+Interaction with ``--queues``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Team filtering and queue filtering are orthogonal — they combine as AND conditions. For example, a triggerer started with ``--team-name team_a --queues q1,q2`` only processes triggers that both belong to ``team_a`` and were deferred from tasks in queues ``q1`` or ``q2``.
+
+.. note::
+
+ If you use both ``--team-name`` and ``--queues``, ensure that each combination of team and queue has at least one triggerer running, or triggers matching that combination will remain unassigned until one starts.
Review Comment:
Yes, I've now reworded in latest commit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add docs for multi-team triggerer support [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #67608:
URL: https://github.com/apache/airflow/pull/67608#issuecomment-4592399086
@ramitkataria — There are 3 unresolved review thread(s) on this PR from `o-nikolas`. Could you either push a fix or reply in each thread explaining why the feedback doesn't apply? When you believe the feedback is addressed, please mark the threads as resolved and ping the reviewer (`o-nikolas`) for a final look. Thanks!
---
_Note: This comment was drafted by an AI-assisted triage tool and may contain mistakes. Once you have addressed the points above, an Apache Airflow maintainer — a real person — will take the next look at your PR. We use this [two-stage triage process](https://github.com/apache/airflow/blob/main/contributing-docs/25_maintainer_pr_triage.md#why-the-first-pass-is-automated) so that our maintainers' limited time is spent where it matters most: the conversation with you._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add docs for multi-team triggerer support [airflow]
Posted by "o-nikolas (via GitHub)" <gi...@apache.org>.
o-nikolas commented on code in PR #67608:
URL: https://github.com/apache/airflow/pull/67608#discussion_r3314298588
##########
airflow-core/docs/core-concepts/multi-team.rst:
##########
@@ -503,6 +504,61 @@ When Multi-Team mode is enabled, the scheduler performs additional logic to dete
teams share the same metadata database and common Airflow infrastructure. For absolutely strict security
requirements, consider separate Airflow deployments.
+.. _multi-team-triggerer:
+
+Team-scoped Triggerer
+---------------------
+
+When Multi-Team mode is enabled, the triggerer can be scoped to a specific team using the ``--team-name`` CLI argument. A team-scoped triggerer only processes deferred tasks (triggers) belonging to that team's Dags. This allows teams to run isolated triggerer instances with independent capacity and failure domains.
+
+Configuration
+^^^^^^^^^^^^^
+
+Start a team-scoped triggerer by passing ``--team-name``:
+
+.. code-block:: bash
+
+ # Triggerer for team_a only
+ airflow triggerer --team-name team_a
+
+ # Triggerer for team_b only
+ airflow triggerer --team-name team_b
+
+ # Global triggerer — processes triggers from Dags with no team association
+ airflow triggerer
+
+Startup validation ensures that ``core.multi_team`` is enabled and the specified team exists in the database.
+
+Behavior
+^^^^^^^^
+
+- **Team-scoped triggerer** (``--team-name team_x``): Only picks up triggers whose originating Dag belongs to a bundle mapped to ``team_x``.
+- **Global triggerer** (no ``--team-name``): Only picks up triggers whose originating Dag belongs to a bundle with no team assignment.
+- **Multi-Team disabled** (``core.multi_team = False``): ``--team-name`` is rejected. No filtering occurs and all triggerers process all triggers (existing behavior).
+
+Interaction with ``--queues``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Team filtering and queue filtering are orthogonal — they combine as AND conditions. For example, a triggerer started with ``--team-name team_a --queues q1,q2`` only processes triggers that both belong to ``team_a`` and were deferred from tasks in queues ``q1`` or ``q2``.
+
+.. note::
+
+ If you use both ``--team-name`` and ``--queues``, ensure that each combination of team and queue has at least one triggerer running, or triggers matching that combination will remain unassigned until one starts.
Review Comment:
This is true for each individually as well right?
##########
airflow-core/docs/core-concepts/multi-team.rst:
##########
@@ -503,6 +504,61 @@ When Multi-Team mode is enabled, the scheduler performs additional logic to dete
teams share the same metadata database and common Airflow infrastructure. For absolutely strict security
requirements, consider separate Airflow deployments.
+.. _multi-team-triggerer:
+
+Team-scoped Triggerer
+---------------------
+
+When Multi-Team mode is enabled, the triggerer can be scoped to a specific team using the ``--team-name`` CLI argument. A team-scoped triggerer only processes deferred tasks (triggers) belonging to that team's Dags. This allows teams to run isolated triggerer instances with independent capacity and failure domains.
Review Comment:
Should be? Otherwise team triggers aren't going to run, right?
```suggestion
When Multi-Team mode is enabled, a triggerer should be scoped to each specific team using the ``--team-name`` CLI argument. A team-scoped triggerer processes deferred tasks (triggers) belonging to that team's Dags. This allows teams to run isolated triggerer instances with independent capacity and failure domains.
```
##########
airflow-core/docs/core-concepts/multi-team.rst:
##########
@@ -503,6 +504,61 @@ When Multi-Team mode is enabled, the scheduler performs additional logic to dete
teams share the same metadata database and common Airflow infrastructure. For absolutely strict security
requirements, consider separate Airflow deployments.
+.. _multi-team-triggerer:
+
+Team-scoped Triggerer
+---------------------
+
+When Multi-Team mode is enabled, the triggerer can be scoped to a specific team using the ``--team-name`` CLI argument. A team-scoped triggerer only processes deferred tasks (triggers) belonging to that team's Dags. This allows teams to run isolated triggerer instances with independent capacity and failure domains.
+
+Configuration
+^^^^^^^^^^^^^
+
+Start a team-scoped triggerer by passing ``--team-name``:
+
+.. code-block:: bash
+
+ # Triggerer for team_a only
+ airflow triggerer --team-name team_a
+
+ # Triggerer for team_b only
+ airflow triggerer --team-name team_b
+
+ # Global triggerer — processes triggers from Dags with no team association
+ airflow triggerer
+
+Startup validation ensures that ``core.multi_team`` is enabled and the specified team exists in the database.
+
+Behavior
+^^^^^^^^
+
+- **Team-scoped triggerer** (``--team-name team_x``): Only picks up triggers whose originating Dag belongs to a bundle mapped to ``team_x``.
+- **Global triggerer** (no ``--team-name``): Only picks up triggers whose originating Dag belongs to a bundle with no team assignment.
+- **Multi-Team disabled** (``core.multi_team = False``): ``--team-name`` is rejected. No filtering occurs and all triggerers process all triggers (existing behavior).
+
+Interaction with ``--queues``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Team filtering and queue filtering are orthogonal — they combine as AND conditions. For example, a triggerer started with ``--team-name team_a --queues q1,q2`` only processes triggers that both belong to ``team_a`` and were deferred from tasks in queues ``q1`` or ``q2``.
+
+.. note::
+
+ If you use both ``--team-name`` and ``--queues``, ensure that each combination of team and queue has at least one triggerer running, or triggers matching that combination will remain unassigned until one starts.
+
+Example Deployment
+^^^^^^^^^^^^^^^^^^
+
+A deployment with two teams and a shared set of Dags with no team:
+
+.. code-block:: bash
+
+ # Team-scoped triggerers
+ airflow triggerer --team-name team_a
+ airflow triggerer --team-name team_b
+
+ # Global triggerer for Dags without a team
+ airflow triggerer
+
Review Comment:
This feels superfluous to me, I'd just drop it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org