You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "abhijeets25012-tech (via GitHub)" <gi...@apache.org> on 2026/01/25 18:12:39 UTC
[PR] Enable per-bundle DAG processor deployments in Helm chart [airflow]
abhijeets25012-tech opened a new pull request, #61044:
URL: https://github.com/apache/airflow/pull/61044
This change adds support for deploying separate DAG Processor pods per DAG bundle in the Airflow Helm chart.
- Introduces `deployPerBundle` configuration under `dagProcessor` in `values.yaml`.
- Allows independent scaling, resource allocation, and failure isolation per DAG bundle.
- Uses existing `--bundle-name` option in Airflow core.
- Backward compatible: single deployment is used if `deployPerBundle.enabled` is false.
- Supports bundle-specific overrides for replicas and resources.
Example configuration:
dagProcessor:
deployPerBundle:
enabled: true
args: ["bash", "-c", "exec airflow dag-processor --bundle-name {{ bundleName }}"]
bundleOverrides:
dags-folder:
replicas: 2
resources:
requests:
memory: "2Gi"
cpu: "1000m"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable per-bundle DAG processor deployments in Helm chart [airflow]
Posted by "jedcunningham (via GitHub)" <gi...@apache.org>.
jedcunningham commented on PR #61044:
URL: https://github.com/apache/airflow/pull/61044#issuecomment-3797959920
Duplicate of #61039.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable per-bundle DAG processor deployments in Helm chart [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on PR #61044:
URL: https://github.com/apache/airflow/pull/61044#issuecomment-3797284517
Please open a half-baked PR in draft before it make sense to review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable per-bundle DAG processor deployments in Helm chart [airflow]
Posted by "uplsh580 (via GitHub)" <gi...@apache.org>.
uplsh580 commented on PR #61044:
URL: https://github.com/apache/airflow/pull/61044#issuecomment-3797448844
I'm working on this.
https://github.com/apache/airflow/pull/61039
Related Issue: https://github.com/apache/airflow/issues/61037
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Enable per-bundle DAG processor deployments in Helm chart [airflow]
Posted by "jedcunningham (via GitHub)" <gi...@apache.org>.
jedcunningham closed pull request #61044: Enable per-bundle DAG processor deployments in Helm chart
URL: https://github.com/apache/airflow/pull/61044
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org