You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "fat-catTW (via GitHub)" <gi...@apache.org> on 2026/08/18 11:17:25 UTC

[PR] Improve deadline diagnostics for null DagRun fields [airflow]

fat-catTW opened a new pull request, #71767:
URL: https://github.com/apache/airflow/pull/71767

    <!-- SPDX-License-Identifier: Apache-2.0
         https://www.apache.org/licenses/LICENSE-2.0 -->
   
   <!--
   Thank you for contributing!
   
   Please provide above a brief description of the changes made in this pull request.
   Write a good git commit message following this guide: https://chris.beams.io/posts/git-commit/
   
   Please make sure that your code changes are covered with tests.
   And in case of new features or big changes remember to adjust the documentation.
   
   For user-facing UI changes, please attach before/after screenshots (or a short
   screen recording) so reviewers can assess the visual impact.
   
   Feel free to ping (in general) for the review if you do not see reaction for a few days
   (72 Hours is the minimum reaction time you can expect from volunteers) - we sometimes miss notifications.
   
   In case of an existing issue, reference it using one of the following:
   
   * closes: #ISSUE
   * related: #ISSUE
   -->
   ## Why
   
   DagRun-based deadline references can evaluate to `None` when the DagRun exists but the referenced timestamp column is unset, such as `logical_date` on manually triggered runs without `-l`.
   
   Before this change, Airflow treated that `None` the same as a missing DagRun and logged a misleading “Could not find DagRun” warning. Deadline creation was then skipped without logging the actual reason, making the configuration hard to diagnose.
   
   relates: #71750
   
   ## Solution
   Distinguish a missing DagRun row from a present DagRun with a `NULL` referenced column.
   
   When deadline creation is skipped because the reference evaluates to `None`, log structured context including the Dag id, run id, deadline alert id, reference type, and the DagRun column required by built-in references.
   
   
   
   ##### Was generative AI tooling used to co-author this PR?
   
   <!--
   If generative AI tooling has been used in the process of authoring this PR, please
   change below checkbox to `[X]` followed by the name of the tool, uncomment the "Generated-by".
   -->
   
   - `[X]` Yes (please specify the tool below)
   
   
   Generated-by: [Codex] following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
   
   
   ---
   
   * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
   * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
   * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
   * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Improve deadline diagnostics for null DagRun fields [airflow]

Posted by "ColtenOuO (via GitHub)" <gi...@apache.org>.
ColtenOuO commented on code in PR #71767:
URL: https://github.com/apache/airflow/pull/71767#discussion_r3807710607


##########
airflow-core/src/airflow/serialization/definitions/dag.py:
##########
@@ -786,6 +786,19 @@ def _process_dagrun_deadline_alerts(
                         "deadline_alerts.deadline_created",
                         tags=prune_dict({"dag_id": self.dag_id, "team_name": team_name}),
                     )
+                else:
+                    required_dagrun_column = {
+                        SerializedReferenceModels.DagRunLogicalDateDeadline: "logical_date",
+                        SerializedReferenceModels.DagRunQueuedAtDeadline: "queued_at",
+                    }.get(type(deserialized_deadline_alert.reference))
+                    log.warning(
+                        "skipping deadline alert because the deadline reference evaluated to None",
+                        dag_id=self.dag_id,
+                        run_id=orm_dagrun.run_id,
+                        deadline_alert_id=deadline_alert.id,
+                        reference_type=deserialized_deadline_alert.reference.reference_name,
+                        required_dagrun_column=required_dagrun_column,
+                    )

Review Comment:
   This `else` branch fires for every `SerializedReferenceModels.TYPES.DAGRUN` reference type, not just `DagRunLogicalDateDeadline`/`DagRunQueuedAtDeadline` — `TYPES.DAGRUN` also includes `AverageRuntimeDeadline`, `FixedDatetimeDeadline`, and `SerializedCustomReference`.
   
   `AverageRuntimeDeadline._evaluate_with` legitimately returns `None` while the Dag hasn't accumulated `min_runs` completed runs yet, and it already logs its own `"Not enough completed runs..."` info message for that case. With this change, every such call *also* emits a spurious WARNING here, misleadingly implying the reference evaluated to `None` because of a null DagRun column — even though `required_dagrun_column` resolves to `None` for this type (the dict only maps `DagRunLogicalDateDeadline`/`DagRunQueuedAtDeadline`) and this reference doesn't read any DagRun column at all. The same false-positive applies to any `SerializedCustomReference` that legitimately returns `None` for its own reasons.
   
   ```python
   with DAG(
       dag_id="demo",
       schedule=None,
       deadline=DeadlineAlert(
           reference=DeadlineReference.AVERAGE_RUNTIME(max_runs=10, min_runs=5),
           interval=timedelta(minutes=5),
           callback=AsyncCallback(empty_callback),
       ),
   ):
       ...
   ```
   
   Trigger it before 5 successful runs exist, and the api-server log shows both lines back to back:
   
   ```
   [info]    Not enough completed runs to calculate average runtime for dag_id=... (found 1, need 5)
   [warning] skipping deadline alert because the deadline reference evaluated to None
             reference_type=AverageRuntimeDeadline required_dagrun_column=None
   ```
   
   The second line is a false positive — it repeats on every trigger until the Dag accumulates enough run history, so it reads as an ongoing warning-level problem for something that's actually expected behavior.
   
   Suggest scoping the warning to only the two reference types it's documented for:
   
   ```python
   if type(deserialized_deadline_alert.reference) in (
       SerializedReferenceModels.DagRunLogicalDateDeadline,
       SerializedReferenceModels.DagRunQueuedAtDeadline,
   ):
       log.warning(...)
   ```
   
   so `AverageRuntimeDeadline` / `FixedDatetimeDeadline` / `SerializedCustomReference` returning `None` don't get this misleading warning attached.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Improve deadline diagnostics for null DagRun fields [airflow]

Posted by "ColtenOuO (via GitHub)" <gi...@apache.org>.
ColtenOuO commented on code in PR #71767:
URL: https://github.com/apache/airflow/pull/71767#discussion_r3807710607


##########
airflow-core/src/airflow/serialization/definitions/dag.py:
##########
@@ -786,6 +786,19 @@ def _process_dagrun_deadline_alerts(
                         "deadline_alerts.deadline_created",
                         tags=prune_dict({"dag_id": self.dag_id, "team_name": team_name}),
                     )
+                else:
+                    required_dagrun_column = {
+                        SerializedReferenceModels.DagRunLogicalDateDeadline: "logical_date",
+                        SerializedReferenceModels.DagRunQueuedAtDeadline: "queued_at",
+                    }.get(type(deserialized_deadline_alert.reference))
+                    log.warning(
+                        "skipping deadline alert because the deadline reference evaluated to None",
+                        dag_id=self.dag_id,
+                        run_id=orm_dagrun.run_id,
+                        deadline_alert_id=deadline_alert.id,
+                        reference_type=deserialized_deadline_alert.reference.reference_name,
+                        required_dagrun_column=required_dagrun_column,
+                    )

Review Comment:
   This `else` branch fires for every `SerializedReferenceModels.TYPES.DAGRUN` reference type, not just `DagRunLogicalDateDeadline`/`DagRunQueuedAtDeadline` — `TYPES.DAGRUN` also includes `AverageRuntimeDeadline`, `FixedDatetimeDeadline`, and `SerializedCustomReference`.
   
   `AverageRuntimeDeadline._evaluate_with` legitimately returns `None` while the Dag hasn't accumulated `min_runs` completed runs yet, and it already logs its own `"Not enough completed runs..."` info message for that case. With this change, every such call also emits a spurious WARNING here, misleadingly implying the reference evaluated to `None` because of a null DagRun column — even though `required_dagrun_column` resolves to `None` for this type (the dict only maps `DagRunLogicalDateDeadline`/`DagRunQueuedAtDeadline`) and this reference doesn't read any DagRun column at all. The same false-positive applies to any `SerializedCustomReference` that legitimately returns `None` for its own reasons.
   
   ```python
   with DAG(
       dag_id="demo",
       schedule=None,
       deadline=DeadlineAlert(
           reference=DeadlineReference.AVERAGE_RUNTIME(max_runs=10, min_runs=5),
           interval=timedelta(minutes=5),
           callback=AsyncCallback(empty_callback),
       ),
   ):
       ...
   ```
   
   Trigger it before 5 successful runs exist, and the api-server log shows both lines back to back:
   
   ```
   [info]    Not enough completed runs to calculate average runtime for dag_id=... (found 1, need 5)
   [warning] skipping deadline alert because the deadline reference evaluated to None
             reference_type=AverageRuntimeDeadline required_dagrun_column=None
   ```
   
   The second line is a false positive — it repeats on every trigger until the Dag accumulates enough run history, so it reads as an ongoing warning-level problem for something that's actually expected behavior.
   
   Suggest scoping the warning to only the two reference types it's documented for:
   
   ```python
   if type(deserialized_deadline_alert.reference) in (
       SerializedReferenceModels.DagRunLogicalDateDeadline,
       SerializedReferenceModels.DagRunQueuedAtDeadline,
   ):
       log.warning(...)
   ```
   
   so `AverageRuntimeDeadline` / `FixedDatetimeDeadline` / `SerializedCustomReference` returning `None` don't get this misleading warning attached.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Improve deadline diagnostics for null DagRun fields [airflow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #71767:
URL: https://github.com/apache/airflow/pull/71767#issuecomment-5603730608

   ### Backport successfully created: v3-3-test
   
   Note: As of [Merging PRs targeted for Airflow 3.X](https://github.com/apache/airflow/blob/main/dev/README_AIRFLOW3_DEV.md#merging-prs-targeted-for-airflow-3x)
   the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.
   
   In matter of doubt please ask in [#release-management](https://apache-airflow.slack.com/archives/C03G9H97MM2) Slack channel.
   
   <table>
           <tr>
           <th>Status</th>
           <th>Branch</th>
           <th>Result</th>
       </tr>
       <tr>
           <td>✅</td>
           <td>v3-3-test</td>
           <td><a href="https://github.com/apache/airflow/pull/72812"><img src="https://img.shields.io/badge/PR-72812-blue" alt="PR Link"></a></td>
       </tr>
   </table>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Improve deadline diagnostics for null DagRun fields [airflow]

Posted by "ferruzzi (via GitHub)" <gi...@apache.org>.
ferruzzi commented on PR #71767:
URL: https://github.com/apache/airflow/pull/71767#issuecomment-5459118912

   Can you add a 1-line bugfix newsfragment?  Other than that I think this LGTM and I want to get it merged before a handful of others while need to rebase on top of this fix. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Improve deadline diagnostics for null DagRun fields [airflow]

Posted by "ferruzzi (via GitHub)" <gi...@apache.org>.
ferruzzi commented on code in PR #71767:
URL: https://github.com/apache/airflow/pull/71767#discussion_r3963352074


##########
airflow-core/src/airflow/serialization/definitions/deadline.py:
##########
@@ -373,10 +373,13 @@ def _fetch_from_db(column, *, session: Session, dag_id: str, run_id: str) -> dat
     """
     from airflow.models import DagRun
 

Review Comment:
   I may be missing it, but are there any tests actually running this change?  The only one I see that hits `_fetch_from_db` is `test_deadline_database_integration`, which mocks it out, so the change is skipped entirely.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Improve deadline diagnostics for null DagRun fields [airflow]

Posted by "fat-catTW (via GitHub)" <gi...@apache.org>.
fat-catTW commented on code in PR #71767:
URL: https://github.com/apache/airflow/pull/71767#discussion_r3964502582


##########
airflow-core/src/airflow/serialization/definitions/dag.py:
##########
@@ -789,6 +789,18 @@ def _process_dagrun_deadline_alerts(
                         "deadline_alerts.deadline_created",
                         tags=prune_dict({"dag_id": self.dag_id, "team_name": team_name}),
                     )
+                elif required_dagrun_column := {
+                    SerializedReferenceModels.DagRunLogicalDateDeadline: "logical_date",
+                    SerializedReferenceModels.DagRunQueuedAtDeadline: "queued_at",
+                }.get(type(deserialized_deadline_alert.reference)):

Review Comment:
   Done, moved this to a module-level `_DAGRUN_REFERENCE_REQUIRED_COLUMNS` constant.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Improve deadline diagnostics for null DagRun fields [airflow]

Posted by "ferruzzi (via GitHub)" <gi...@apache.org>.
ferruzzi commented on code in PR #71767:
URL: https://github.com/apache/airflow/pull/71767#discussion_r3963357730


##########
airflow-core/src/airflow/serialization/definitions/dag.py:
##########
@@ -789,6 +789,18 @@ def _process_dagrun_deadline_alerts(
                         "deadline_alerts.deadline_created",
                         tags=prune_dict({"dag_id": self.dag_id, "team_name": team_name}),
                     )
+                elif required_dagrun_column := {
+                    SerializedReferenceModels.DagRunLogicalDateDeadline: "logical_date",
+                    SerializedReferenceModels.DagRunQueuedAtDeadline: "queued_at",
+                }.get(type(deserialized_deadline_alert.reference)):

Review Comment:
   Nit, only if you are making other changes:  May as well move this dict into a module-level constant, no?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Improve deadline diagnostics for null DagRun fields [airflow]

Posted by "fat-catTW (via GitHub)" <gi...@apache.org>.
fat-catTW commented on code in PR #71767:
URL: https://github.com/apache/airflow/pull/71767#discussion_r3964510746


##########
airflow-core/src/airflow/serialization/definitions/deadline.py:
##########
@@ -373,10 +373,13 @@ def _fetch_from_db(column, *, session: Session, dag_id: str, run_id: str) -> dat
     """
     from airflow.models import DagRun
 

Review Comment:
   Thanks for review!
   You were right, the existing integration test mocked out `_fetch_from_db`, so it did not directly exercise this change.
   
   I added direct DB-backed coverage for the serialized `_fetch_from_db` helper:
   - a DagRun exists but the selected `logical_date` column is NULL, returning `None` without logging `Could not find DagRun`
   - no DagRun exists, preserving the existing missing-DagRun warning
   
   I also kept the higher-level DagRun deadline tests for the call-site warning behavior.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Improve deadline diagnostics for null DagRun fields [airflow]

Posted by "fat-catTW (via GitHub)" <gi...@apache.org>.
fat-catTW commented on code in PR #71767:
URL: https://github.com/apache/airflow/pull/71767#discussion_r3813297467


##########
airflow-core/src/airflow/serialization/definitions/dag.py:
##########
@@ -786,6 +786,19 @@ def _process_dagrun_deadline_alerts(
                         "deadline_alerts.deadline_created",
                         tags=prune_dict({"dag_id": self.dag_id, "team_name": team_name}),
                     )
+                else:
+                    required_dagrun_column = {
+                        SerializedReferenceModels.DagRunLogicalDateDeadline: "logical_date",
+                        SerializedReferenceModels.DagRunQueuedAtDeadline: "queued_at",
+                    }.get(type(deserialized_deadline_alert.reference))
+                    log.warning(
+                        "skipping deadline alert because the deadline reference evaluated to None",
+                        dag_id=self.dag_id,
+                        run_id=orm_dagrun.run_id,
+                        deadline_alert_id=deadline_alert.id,
+                        reference_type=deserialized_deadline_alert.reference.reference_name,
+                        required_dagrun_column=required_dagrun_column,
+                    )

Review Comment:
   Good catch! Thanks for the review.
   I narrowed this warning to only `DagRunLogicalDateDeadline` and `DagRunQueuedAtDeadline`, since those are the references that read nullable DagRun timestamp columns.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Improve deadline diagnostics for null DagRun fields [airflow]

Posted by "ferruzzi (via GitHub)" <gi...@apache.org>.
ferruzzi commented on code in PR #71767:
URL: https://github.com/apache/airflow/pull/71767#discussion_r3963357730


##########
airflow-core/src/airflow/serialization/definitions/dag.py:
##########
@@ -789,6 +789,18 @@ def _process_dagrun_deadline_alerts(
                         "deadline_alerts.deadline_created",
                         tags=prune_dict({"dag_id": self.dag_id, "team_name": team_name}),
                     )
+                elif required_dagrun_column := {
+                    SerializedReferenceModels.DagRunLogicalDateDeadline: "logical_date",
+                    SerializedReferenceModels.DagRunQueuedAtDeadline: "queued_at",
+                }.get(type(deserialized_deadline_alert.reference)):

Review Comment:
   May as well move this dict into a module-level constant, no?



##########
airflow-core/src/airflow/serialization/definitions/dag.py:
##########
@@ -789,6 +789,18 @@ def _process_dagrun_deadline_alerts(
                         "deadline_alerts.deadline_created",
                         tags=prune_dict({"dag_id": self.dag_id, "team_name": team_name}),
                     )
+                elif required_dagrun_column := {
+                    SerializedReferenceModels.DagRunLogicalDateDeadline: "logical_date",
+                    SerializedReferenceModels.DagRunQueuedAtDeadline: "queued_at",
+                }.get(type(deserialized_deadline_alert.reference)):

Review Comment:
   Nit:  May as well move this dict into a module-level constant, no?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Improve deadline diagnostics for null DagRun fields [airflow]

Posted by "vatsrahul1001 (via GitHub)" <gi...@apache.org>.
vatsrahul1001 merged PR #71767:
URL: https://github.com/apache/airflow/pull/71767


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org