[I] Flaky “Requested bundle is not configured” error during 2 to 3.1+ upgrade with custom Dag bundle names [airflow]

["jason810496 (via GitHub)" <gi...@apache.org>]

jason810496 opened a new issue, #63323:
URL: https://github.com/apache/airflow/issues/63323

   ### Body
   
   
   * related: #47592
   
   ## Why
   
   Some of the users encounter the following flaky error for 2 to 3.1+ upgrade.
   
   If user configure the Dag Bundle with other name than `dags-folder` when running the `Make bundle_name not nullable` migration, which might raise the following `"Requested bundle '{name}' is not configured."` error when triggering DagRun right after the migration.
   
   https://github.com/apache/airflow/blob/450a08e78499fbd6d21d2d6dc2392dff9dbfda98/airflow-core/src/airflow/dag_processing/bundles/manager.py#L333-L345
   
   
   Here're some other edge case that will cause the above error:  
   
   **case 1**: Dag Bundle name not `dags-folder`
   
   ```bash
   AIRFLOW__DAG_PROCESSOR__DAG_BUNDLE_CONFIG_LIST='[{"name": "main", "classpath": "airflow.dag_processing.bundles.local.LocalDagBundle", "kwargs": {"path": "/files/dags", "refresh_interval": 1}}]'
   ```
   
   **case 2**: Multiple Custom Dag Bundle
   
   ```bash
   AIRFLOW__DAG_PROCESSOR__DAG_BUNDLE_CONFIG_LIST='[{"name": "main1", "classpath": "airflow.dag_processing.bundles.local.LocalDagBundle", "kwargs": {"path": "/files/dags", "refresh_interval": 1}},  {"name": "main2", "classpath": "airflow.dag_processing.bundles.local.LocalDagBundle", "kwargs": {"path": "/files/dags", "refresh_interval": 1}}]'
   ```
   
   ## Root Cause
   
   The migration script will only insert `dag-folder` bundle instead of the correct Dag bundle before the fix for all the above cases.
   
   https://github.com/apache/airflow/blob/da0940431b39ad9378215bb0fedd80bc786056df/airflow-core/src/airflow/migrations/versions/0082_3_1_0_make_bundle_name_not_nullable.py#L76-L88
   
   ### Committer
   
   - [x] I acknowledge that I am a maintainer/committer of the Apache Airflow project.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Flaky “Requested bundle is not configured” error during 2 to 3.1+ upgrade with custom Dag bundle names [airflow]

["ephraimbuddy (via GitHub)" <gi...@apache.org>]

ephraimbuddy closed issue #63323: Flaky “Requested bundle is not configured” error during 2 to 3.1+ upgrade with custom Dag bundle names
URL: https://github.com/apache/airflow/issues/63323


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org