You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "nathadfield (via GitHub)" <gi...@apache.org> on 2026/01/26 10:32:56 UTC

[PR] Add configurable bundle version defaults [airflow]

nathadfield opened a new pull request, #61063:
URL: https://github.com/apache/airflow/pull/61063

   # Add configurable bundle version defaults
   
   ## Summary
   
   This PR adds the ability to configure default bundle version behavior at three levels with a clear precedence hierarchy. Users can now set organization-wide, per-DAG, or per-operator defaults for whether DAG runs should use the latest bundle version or the original version, eliminating the need to specify this on every clear/rerun operation.
   
   Additionally, this PR adds the `run_on_latest_version` parameter to `TriggerDagRunOperator`, allowing explicit control when triggering child DAGs programmatically.
   
   ## What Changed
   
   ### 1. Global Configuration
   Added `[core] run_on_latest_version` configuration option that applies organization-wide default behavior.
   
   ```ini
   [core]
   run_on_latest_version = True
   ```
   
   ### 2. DAG-Level Configuration
   Added `run_on_latest_version` parameter to the DAG class that overrides global configuration for specific DAGs.
   
   ```python
   with DAG(dag_id="my_dag", run_on_latest_version=True, ...) as dag:
       ...
   ```
   
   ### 3. Operator-Level Control
   Added `run_on_latest_version` parameter to `TriggerDagRunOperator` with highest precedence, overriding both DAG and global configuration.
   
   ```python
   trigger = TriggerDagRunOperator(
       task_id="trigger_latest",
       trigger_dag_id="target_dag",
       run_on_latest_version=True,
   )
   ```
   
   ### 4. UI Integration
   Configured existing "Run on Latest Version" checkboxes in all three Clear dialogs (Run, Task Instance, Task Group) to respect the configuration hierarchy. Checkboxes now default based on precedence: DAG-level > Global > System.
   
   ### 5. Precedence Hierarchy
   
   ```
   Operator Level > DAG Level > Global Level > System Default (original version)
   ```
   
   ## Implementation Details
   
   ### Backend Changes (9 files)
   - **Configuration**: Added new config option with documentation
   - **DAG Definition**: Added parameter to DAG class and serialization
   - **Resolution Logic**: Implemented precedence hierarchy in `_resolve_bundle_version()`
   - **API Layer**: Exposed configuration and bundle version fields in responses
   - **Error Handling**: Improved logging with INFO/ERROR/WARNING levels based on context
   
   ### Operator Changes (1 file)
   - **TriggerDagRunOperator**: Added `run_on_latest_version` parameter
   - **Supervisor Integration**: Forwards bundle_version to API client
   - **API Client**: Accepts and passes bundle_version parameter
   
   ### UI Changes (3 files)
   - **Clear Dialogs**: Implemented precedence hierarchy for checkbox defaults
   - **Configuration Fetch**: Added `useConfig()` integration
   - **Dynamic Updates**: Added `useEffect` to sync state when config loads
   
   ### Test Coverage (3 files)
   - Added tests for global config API
   - Added tests for DAG-level configuration
   - Added tests for precedence hierarchy resolution
   - Added integration tests for supervisor → API → database flow
   - Fixed SQLAlchemy deprecation warnings
   
   ### Documentation (1 file)
   - Updated dag-bundles.rst with comprehensive examples
   - Documented precedence hierarchy
   - Added configuration reference
   
   ## Use Cases
   
   ### Use Case 1: Organization-Wide Default
   Set global config to True so all DAG runs use the latest bundle version by default.
   
   ### Use Case 2: Critical DAG Exception
   Override global config at DAG level for critical DAGs that must use original version for compliance.
   
   ### Use Case 3: Specific Task Override
   Override at operator level to test original version while other tasks use latest.
   
   ## Testing
   
   ### Unit Tests
   - ✅ Config API tests (2 new tests)
   - ✅ DAG resolution tests (5 new tests)
   - ✅ Supervisor forwarding tests (3 tests including new bundle_version test)
   - ✅ API client tests (3 tests)
   - ✅ Serialization tests
   
   ### Manual Testing
   - ✅ Checkbox defaults respect precedence hierarchy
   - ✅ TriggerDagRunOperator passes bundle version correctly
   - ✅ AF2 compatibility maintained (bundle versioning ignored)
   - ✅ AF3 functionality working
   
   ## Backward Compatibility
   
   **✅ Fully backward compatible**:
   - All new configuration defaults to `False` (preserves current behavior)
   - No database schema changes required
   - No breaking changes to existing APIs
   - UI enhancements are progressive
   - Non-versioned bundles (LocalDagBundle) unaffected
   
   ## Migration
   
   **No migration required!** The feature is completely opt-in:
   1. Default behavior preserved (use original bundle version)
   2. Enable at global, DAG, or operator level as needed
   3. No database changes or data migration
   
   ## Related Issues
   
   - Closes #60880: TriggerDagRunOperator bundle version parameters
   - Closes #60887: Global and DAG-level bundle version defaults
   
   ## Breaking Changes
   
   **None**
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   - [X] Yes (please specify the tool below)
   
   Generated-by: Claude Sonnet 4.5 following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add configurable bundle version defaults [airflow]

Posted by "nathadfield (via GitHub)" <gi...@apache.org>.
nathadfield commented on PR #61063:
URL: https://github.com/apache/airflow/pull/61063#issuecomment-3824049357

   @guan404ming Maybe you could have a read of the associated issues (#60887 and #60880) that I raised where my justification and some thoughts were expressed?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add configurable bundle version defaults [airflow]

Posted by "nathadfield (via GitHub)" <gi...@apache.org>.
nathadfield commented on PR #61063:
URL: https://github.com/apache/airflow/pull/61063#issuecomment-3818597819

   Hey folks! If any of you get the time to review this PR soon I'd be most appreciative. Many thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add configurable bundle version defaults [airflow]

Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on code in PR #61063:
URL: https://github.com/apache/airflow/pull/61063#discussion_r2761637473


##########
providers/standard/src/airflow/providers/standard/operators/trigger_dagrun.py:
##########
@@ -253,40 +265,65 @@ def execute(self, context: Context):
         self.trigger_run_id = run_id
 
         if self.fail_when_dag_is_paused:
-            dag_model = DagModel.get_current(self.trigger_dag_id)
-            if not dag_model:
-                raise ValueError(f"Dag {self.trigger_dag_id} is not found")
-            if dag_model.is_paused:
-                # TODO: enable this when dag state endpoint available from task sdk
-                # if AIRFLOW_V_3_0_PLUS:
-                #     raise DagIsPaused(dag_id=self.trigger_dag_id)
-                raise AirflowException(f"Dag {self.trigger_dag_id} is paused")
+            if AIRFLOW_V_3_0_PLUS:
+                # In Airflow 3.0, use Supervisor API to get DAG info
+                from airflow.sdk.execution_time.comms import DagResult, GetDag
+                from airflow.sdk.execution_time.task_runner import SUPERVISOR_COMMS

Review Comment:
   A better design (atleast imo) would be to fail it on server side, when POST is sent to `"/execution/dag-runs/{dag_id}/{run_id}"` -- and have an error type similar to the plumbing for `reset_dag_run`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add configurable bundle version defaults [airflow]

Posted by "nathadfield (via GitHub)" <gi...@apache.org>.
nathadfield commented on PR #61063:
URL: https://github.com/apache/airflow/pull/61063#issuecomment-3836105003

   @guan404ming I made some changes based on your feedback if you wouldn't mind taking a look.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add configurable bundle version defaults [airflow]

Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on code in PR #61063:
URL: https://github.com/apache/airflow/pull/61063#discussion_r2746685741


##########
airflow-core/src/airflow/ui/src/components/Clear/Run/ClearRunDialog.tsx:
##########
@@ -45,13 +46,33 @@ const ClearRunDialog = ({ dagRun, onClose, open }: Props) => {
   const [note, setNote] = useState<string | null>(dagRun.note);
   const [selectedOptions, setSelectedOptions] = useState<Array<string>>(["existingTasks"]);
   const onlyFailed = selectedOptions.includes("onlyFailed");
-  const [runOnLatestVersion, setRunOnLatestVersion] = useState(false);
 
   // Get current DAG's bundle version to compare with DAG run's bundle version
   const { data: dagDetails } = useDagServiceGetDagDetails({
     dagId,
   });
 
+  // Get global config for run_on_latest_version
+  const globalDefaultRunOnLatestVersion = Boolean(useConfig("run_on_latest_version"));
+
+  // Determine checkbox default based on precedence: DAG-level > Global config > System default (false)
+  const defaultRunOnLatestVersion = useMemo(() => {

Review Comment:
   a single null-check has near-zero cost. Replace with nullish coalescing: `dagDetails?.run_on_latest_version ?? globalDefault`



##########
airflow-core/src/airflow/ui/src/components/Clear/Run/ClearRunDialog.tsx:
##########
@@ -45,13 +46,33 @@ const ClearRunDialog = ({ dagRun, onClose, open }: Props) => {
   const [note, setNote] = useState<string | null>(dagRun.note);
   const [selectedOptions, setSelectedOptions] = useState<Array<string>>(["existingTasks"]);
   const onlyFailed = selectedOptions.includes("onlyFailed");
-  const [runOnLatestVersion, setRunOnLatestVersion] = useState(false);
 
   // Get current DAG's bundle version to compare with DAG run's bundle version
   const { data: dagDetails } = useDagServiceGetDagDetails({
     dagId,
   });
 
+  // Get global config for run_on_latest_version
+  const globalDefaultRunOnLatestVersion = Boolean(useConfig("run_on_latest_version"));
+
+  // Determine checkbox default based on precedence: DAG-level > Global config > System default (false)
+  const defaultRunOnLatestVersion = useMemo(() => {
+    // Level 1: DAG-level configuration
+    if (dagDetails?.run_on_latest_version !== undefined && dagDetails.run_on_latest_version !== null) {
+      return dagDetails.run_on_latest_version;
+    }
+
+    // Level 2: Global configuration (Boolean() always returns true or false)
+    return globalDefaultRunOnLatestVersion;
+  }, [dagDetails?.run_on_latest_version, globalDefaultRunOnLatestVersion]);
+
+  const [runOnLatestVersion, setRunOnLatestVersion] = useState(defaultRunOnLatestVersion);

Review Comment:
   useState + useEffect to sync derived state will cause an extra re-render. Use a nullable override pattern instead: keep override as null until the user interacts, then resolve via `override ?? defaultValue`.



##########
airflow-core/src/airflow/ui/src/components/Clear/Run/ClearRunDialog.tsx:
##########
@@ -45,13 +46,33 @@ const ClearRunDialog = ({ dagRun, onClose, open }: Props) => {
   const [note, setNote] = useState<string | null>(dagRun.note);
   const [selectedOptions, setSelectedOptions] = useState<Array<string>>(["existingTasks"]);
   const onlyFailed = selectedOptions.includes("onlyFailed");
-  const [runOnLatestVersion, setRunOnLatestVersion] = useState(false);
 
   // Get current DAG's bundle version to compare with DAG run's bundle version
   const { data: dagDetails } = useDagServiceGetDagDetails({
     dagId,
   });
 
+  // Get global config for run_on_latest_version
+  const globalDefaultRunOnLatestVersion = Boolean(useConfig("run_on_latest_version"));

Review Comment:
   Boolean("false") evaluates to true since any non-empty string is truthy. Use explicit comparison (raw === "true") instead.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add configurable bundle version defaults [airflow]

Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on code in PR #61063:
URL: https://github.com/apache/airflow/pull/61063#discussion_r2761640225


##########
providers/standard/src/airflow/providers/standard/operators/trigger_dagrun.py:
##########
@@ -253,40 +265,65 @@ def execute(self, context: Context):
         self.trigger_run_id = run_id
 
         if self.fail_when_dag_is_paused:
-            dag_model = DagModel.get_current(self.trigger_dag_id)
-            if not dag_model:
-                raise ValueError(f"Dag {self.trigger_dag_id} is not found")
-            if dag_model.is_paused:
-                # TODO: enable this when dag state endpoint available from task sdk
-                # if AIRFLOW_V_3_0_PLUS:
-                #     raise DagIsPaused(dag_id=self.trigger_dag_id)
-                raise AirflowException(f"Dag {self.trigger_dag_id} is paused")
+            if AIRFLOW_V_3_0_PLUS:
+                # In Airflow 3.0, use Supervisor API to get DAG info
+                from airflow.sdk.execution_time.comms import DagResult, GetDag
+                from airflow.sdk.execution_time.task_runner import SUPERVISOR_COMMS

Review Comment:
   https://github.com/apache/airflow/blob/cec8ba68f19cb21293979a6f042909d5ccbaac11/airflow-core/src/airflow/api_fastapi/execution_api/routes/dag_runs.py#L127-L134



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add configurable bundle version defaults [airflow]

Posted by "nathadfield (via GitHub)" <gi...@apache.org>.
nathadfield commented on PR #61063:
URL: https://github.com/apache/airflow/pull/61063#issuecomment-3824062862

   Here's a video demonstrating the Clear Dag Run dialog when the `run_on_latest_version` configuration option is set.
   
   https://github.com/user-attachments/assets/74730560-8c26-41e8-a914-165e84563518
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add configurable bundle version defaults [airflow]

Posted by "nathadfield (via GitHub)" <gi...@apache.org>.
nathadfield commented on PR #61063:
URL: https://github.com/apache/airflow/pull/61063#issuecomment-3824441801

   @guan404ming Definitely! Hopefully this is a place to start because I can also see value in the ability to choose arbitrary versions too.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add configurable bundle version defaults [airflow]

Posted by "nathadfield (via GitHub)" <gi...@apache.org>.
nathadfield commented on PR #61063:
URL: https://github.com/apache/airflow/pull/61063#issuecomment-3846048540

   > I don't think this is ready yet - needs major revisions unfortunately @nathadfield .
   > 
   > The `dag` endpoint on execution doesn't exist and so everything else will fail. And I don't even think we need that endpoint -- especially since you only need it to know if dag is paused or not.
   > 
   > I'd also recommend separating this PR into two:
   > 
   > 1. Configurable bundle version default
   > 2. TriggerDagRunOperator changes!
   
   Thanks @kaxil! I appreciate the input and work on addressing your issues.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add configurable bundle version defaults [airflow]

Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on code in PR #61063:
URL: https://github.com/apache/airflow/pull/61063#discussion_r2761620609


##########
task-sdk/src/airflow/sdk/api/client.py:
##########
@@ -682,6 +682,20 @@ def get(
         return AssetEventsResponse.model_validate_json(resp.read())
 
 
+class DagOperations:
+    __slots__ = ("client",)
+
+    def __init__(self, client: Client):
+        self.client = client
+
+    def get_detail(self, dag_id: str):
+        """Get details of a DAG from the API server."""
+        resp = self.client.get(f"dags/{dag_id}")
+        # Return a simple dict-like response that can be accessed via attributes
+        data = msgspec.json.decode(resp.read())
+        return type("DagResponse", (), data)

Review Comment:
   We don't have such an endpoint on the execution API !



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add configurable bundle version defaults [airflow]

Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on PR #61063:
URL: https://github.com/apache/airflow/pull/61063#issuecomment-3823489595

   Could you help attach some screenshot or some demo to quick show what have you done in this pr? Thanks!
   
   Also I think your current ui wise change is not that align to your claim in your pr desciption, please help review manually and carefully to ensure all changes are needed and concise. Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add configurable bundle version defaults [airflow]

Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on PR #61063:
URL: https://github.com/apache/airflow/pull/61063#issuecomment-3824383731

   Thank you so much for sharing that practical use case. It is really valuable to hear from the perspective of someone managing live systems. I think that this isn't a '0 or 1' situation but a good start point to make this feature more useful gradually through real world input and more community discussion.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add configurable bundle version defaults [airflow]

Posted by "nathadfield (via GitHub)" <gi...@apache.org>.
nathadfield commented on PR #61063:
URL: https://github.com/apache/airflow/pull/61063#issuecomment-3823595256

   Hi @guan404ming, thanks for reviewing!
   
   The UI changes are minimal and directly implement what's described in the PR. All three Clear dialog files received identical changes:                                                                      
   
   1. Fetch global config via useConfig("run_on_latest_version")
   2. Implement precedence logic (DAG-level → Global → System default)
   3. Apply to checkbox default - Changed from hardcoded useState(false) to useState(defaultRunOnLatestVersion)
   4. Handle async config loading with useEffect
   
   Bug fix included: Changed bundle version comparison from incorrect taskInstance.dag_version?.bundle_version to correct taskInstance.dag_run_bundle_version.
   
   Regarding screenshot/demo, the visual change is subtle - it only affects the default state of the existing "**Run on Latest Version**" checkbox (checked vs unchecked based on config). The checkbox UI itself is unchanged.
   
   - Before: Always defaults to unchecked
   - After: Defaults based on DAG config → Global config → unchecked
   
   I can post a short video that demonstrates that the checkbox is checked automatically when the config is set if that would help?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add configurable bundle version defaults [airflow]

Posted by "nathadfield (via GitHub)" <gi...@apache.org>.
nathadfield commented on PR #61063:
URL: https://github.com/apache/airflow/pull/61063#issuecomment-3824320135

   @guan404ming Thanks for that and I really appreciate your input. Hopefully others will have their say too and in the meantime I'll address the points you raised. From my perspective as someone who manages live Airflow systems, whether you want the default rerun behaviour of a DAG/Task to based on the initial or latest version should be the choice of the administrator.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add configurable bundle version defaults [airflow]

Posted by "nathadfield (via GitHub)" <gi...@apache.org>.
nathadfield closed pull request #61063: Add configurable bundle version defaults
URL: https://github.com/apache/airflow/pull/61063


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add configurable bundle version defaults [airflow]

Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on code in PR #61063:
URL: https://github.com/apache/airflow/pull/61063#discussion_r2761638365


##########
providers/standard/src/airflow/providers/standard/operators/trigger_dagrun.py:
##########
@@ -253,40 +265,65 @@ def execute(self, context: Context):
         self.trigger_run_id = run_id
 
         if self.fail_when_dag_is_paused:
-            dag_model = DagModel.get_current(self.trigger_dag_id)
-            if not dag_model:
-                raise ValueError(f"Dag {self.trigger_dag_id} is not found")
-            if dag_model.is_paused:
-                # TODO: enable this when dag state endpoint available from task sdk
-                # if AIRFLOW_V_3_0_PLUS:
-                #     raise DagIsPaused(dag_id=self.trigger_dag_id)
-                raise AirflowException(f"Dag {self.trigger_dag_id} is paused")
+            if AIRFLOW_V_3_0_PLUS:
+                # In Airflow 3.0, use Supervisor API to get DAG info
+                from airflow.sdk.execution_time.comms import DagResult, GetDag
+                from airflow.sdk.execution_time.task_runner import SUPERVISOR_COMMS

Review Comment:
   https://github.com/apache/airflow/blob/cec8ba68f19cb21293979a6f042909d5ccbaac11/task-sdk/src/airflow/sdk/api/client.py#L706-L713



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add configurable bundle version defaults [airflow]

Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on code in PR #61063:
URL: https://github.com/apache/airflow/pull/61063#discussion_r2761627522


##########
providers/standard/src/airflow/providers/standard/operators/trigger_dagrun.py:
##########
@@ -253,40 +265,65 @@ def execute(self, context: Context):
         self.trigger_run_id = run_id
 
         if self.fail_when_dag_is_paused:
-            dag_model = DagModel.get_current(self.trigger_dag_id)
-            if not dag_model:
-                raise ValueError(f"Dag {self.trigger_dag_id} is not found")
-            if dag_model.is_paused:
-                # TODO: enable this when dag state endpoint available from task sdk
-                # if AIRFLOW_V_3_0_PLUS:
-                #     raise DagIsPaused(dag_id=self.trigger_dag_id)
-                raise AirflowException(f"Dag {self.trigger_dag_id} is paused")
+            if AIRFLOW_V_3_0_PLUS:
+                # In Airflow 3.0, use Supervisor API to get DAG info
+                from airflow.sdk.execution_time.comms import DagResult, GetDag
+                from airflow.sdk.execution_time.task_runner import SUPERVISOR_COMMS

Review Comment:
   Providers should never need to import `SUPERVISOR_COMMS`



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add configurable bundle version defaults [airflow]

Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on PR #61063:
URL: https://github.com/apache/airflow/pull/61063#issuecomment-3824266315

   > Maybe you could have a read of the associated issues (https://github.com/apache/airflow/issues/60887 and https://github.com/apache/airflow/issues/60880) that I raised where my justification and some thoughts were expressed?
   
   To clarify, the points I am raising here represent my own personal opinion. My primary hesitation stems from the practical utility of the feature. Specifically, I feel that restricting the functionality to a standalone `run_on_latest_version` offers relatively limited value to users. In my view, for this feature to be truly useful, it would arguably need to support arbitrary versions. However, I fully share the valid concerns raised by Jed regarding security. So If we were to aim for that level of flexibility (arbitrary versions), doing so while maintaining strict security standards would introduce a significant layer of complexity to the implementation. It is precisely this trade-off between utility and complexity that I wanted to bring up for the community to discuss, rather than implying the idea is incorrect.
   
   I am very open to hearing different perspectives and further discussion on how we might balance these factors. Thank you again for your hard work and contribution!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Add configurable bundle version defaults [airflow]

Posted by "guan404ming (via GitHub)" <gi...@apache.org>.
guan404ming commented on PR #61063:
URL: https://github.com/apache/airflow/pull/61063#issuecomment-3824027622

   Thanks for the detailed explanation. It did resolve my misunderstanding. After thinking it over, I'm not sure we should include this feature as is or maybe we could refine it to let users specify the bundle version they want, rather than just choosing whether to use the latest version. This then could be more useful on my opinion but I'd like to hear others' thoughts!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org