You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "abhijeets25012-tech (via GitHub)" <gi...@apache.org> on 2026/01/25 18:12:39 UTC

[PR] Enable per-bundle DAG processor deployments in Helm chart [airflow]

abhijeets25012-tech opened a new pull request, #61044:
URL: https://github.com/apache/airflow/pull/61044

   
   This change adds support for deploying separate DAG Processor pods per DAG bundle in the Airflow Helm chart. 
   
   - Introduces `deployPerBundle` configuration under `dagProcessor` in `values.yaml`.
   - Allows independent scaling, resource allocation, and failure isolation per DAG bundle.
   - Uses existing `--bundle-name` option in Airflow core.
   - Backward compatible: single deployment is used if `deployPerBundle.enabled` is false.
   - Supports bundle-specific overrides for replicas and resources.
   
   Example configuration:
   
   dagProcessor:
     deployPerBundle:
       enabled: true
       args: ["bash", "-c", "exec airflow dag-processor --bundle-name {{ bundleName }}"]
       bundleOverrides:
         dags-folder:
           replicas: 2
           resources:
             requests:
               memory: "2Gi"
               cpu: "1000m"
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Enable per-bundle DAG processor deployments in Helm chart [airflow]

Posted by "jedcunningham (via GitHub)" <gi...@apache.org>.
jedcunningham commented on PR #61044:
URL: https://github.com/apache/airflow/pull/61044#issuecomment-3797959920

   Duplicate of #61039.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Enable per-bundle DAG processor deployments in Helm chart [airflow]

Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on PR #61044:
URL: https://github.com/apache/airflow/pull/61044#issuecomment-3797284517

   Please open a half-baked PR in draft before it make sense to review.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Enable per-bundle DAG processor deployments in Helm chart [airflow]

Posted by "uplsh580 (via GitHub)" <gi...@apache.org>.
uplsh580 commented on PR #61044:
URL: https://github.com/apache/airflow/pull/61044#issuecomment-3797448844

   I'm working on this.
   https://github.com/apache/airflow/pull/61039
   
   Related Issue: https://github.com/apache/airflow/issues/61037


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Enable per-bundle DAG processor deployments in Helm chart [airflow]

Posted by "jedcunningham (via GitHub)" <gi...@apache.org>.
jedcunningham closed pull request #61044: Enable per-bundle DAG processor deployments in Helm chart
URL: https://github.com/apache/airflow/pull/61044


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org