You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "amoghrajesh (via GitHub)" <gi...@apache.org> on 2026/02/16 14:12:22 UTC
[PR] Add missing ti.start and ti.finish metrics in Airflow 3 [airflow]
amoghrajesh opened a new pull request, #62019:
URL: https://github.com/apache/airflow/pull/62019
<!--
Thank you for contributing!
Please provide above a brief description of the changes made in this pull request.
Write a good git commit message following this guide: http://chris.beams.io/posts/git-commit/
Please make sure that your code changes are covered with tests.
And in case of new features or big changes remember to adjust the documentation.
Feel free to ping (in general) for the review if you do not see reaction for a few days
(72 Hours is the minimum reaction time you can expect from volunteers) - we sometimes miss notifications.
In case of an existing issue, reference it using one of the following:
* closes: #ISSUE
* related: #ISSUE
-->
---
##### Was generative AI tooling used to co-author this PR?
<!--
If generative AI tooling has been used in the process of authoring this PR, please
change below checkbox to `[X]` followed by the name of the tool, uncomment the "Generated-by".
-->
- [x] No
<!--
Generated-by: [Tool Name] following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
-->
closes: https://github.com/apache/airflow/issues/62015
Some metrics like `ti.start` and `ti.finish` were not ported over to Airflow 3.
This PR intends to add those to Airflow 3 and emit it from task execution.
### Testing
Deployed Airflow with breeze with `--integration statsd`
#### Earlier:
<img width="2560" height="564" alt="image (67)" src="https://github.com/user-attachments/assets/9814cd22-cc87-45dc-b16c-acaa9d1fcd6b" />
<img width="2560" height="564" alt="image (68)" src="https://github.com/user-attachments/assets/c23b7e50-e4d0-4183-b1a7-a29221afe1a8" />
#### After:
##### Prometheus
<img width="2560" height="564" alt="image" src="https://github.com/user-attachments/assets/4e73c549-9b46-4aec-88ce-231de4e3a86d" />
<img width="2560" height="564" alt="image" src="https://github.com/user-attachments/assets/c3cce2e7-4de5-4062-9a8b-059f221d1041" />
##### Statsd
Finish:
```shell
➜ ~ curl -s http://localhost:29102/metrics | grep -i "finish"
# HELP airflow_ti_finish Metric autogenerated by statsd_exporter.
# TYPE airflow_ti_finish counter
airflow_ti_finish 2
# HELP airflow_ti_finish_sleeping_sleep_success Metric autogenerated by statsd_exporter.
# TYPE airflow_ti_finish_sleeping_sleep_success counter
airflow_ti_finish_sleeping_sleep_success 2
```
Start:
```shell
➜ ~ curl -s http://localhost:29102/metrics | grep -i "start"
# HELP airflow_job_start Metric autogenerated by statsd_exporter.
# TYPE airflow_job_start counter
airflow_job_start 2
# HELP airflow_ti_start Metric autogenerated by statsd_exporter.
# TYPE airflow_ti_start counter
airflow_ti_start 4
# HELP airflow_ti_start_sleeping_sleep Metric autogenerated by statsd_exporter.
# TYPE airflow_ti_start_sleeping_sleep counter
airflow_ti_start_sleeping_sleep 4
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.77124692275e+09
```
---
* Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment: `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add missing ti.start and ti.finish metrics in Airflow 3 [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #62019:
URL: https://github.com/apache/airflow/pull/62019#discussion_r2816436513
##########
task-sdk/src/airflow/sdk/execution_time/task_runner.py:
##########
@@ -1283,6 +1288,13 @@ def _on_term(signum, frame):
msg, state = _handle_current_task_failed(ti)
error = e
finally:
+ Stats.incr(
+ f"ti.finish.{ti.dag_id}.{ti.task_id}.{state.value}",
+ tags=stats_tags,
+ )
+ # Same metric with tagging
+ Stats.incr("ti.finish", tags={**stats_tags, "state": state.value})
Review Comment:
As per our offline chat, you could handle it later :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add missing ti.start and ti.finish metrics in Airflow 3 [airflow]
Posted by "kacpermuda (via GitHub)" <gi...@apache.org>.
kacpermuda commented on PR #62019:
URL: https://github.com/apache/airflow/pull/62019#issuecomment-3908807869
> I think you are involved with metrics and could help with reviews
I'm not, but the code looks good, left an approve, thanks !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add missing ti.start and ti.finish metrics in Airflow 3 [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on PR #62019:
URL: https://github.com/apache/airflow/pull/62019#issuecomment-3908722881
cc @kacpermuda @xBis7 apologies if wrongly tagged, but I think you are involved with metrics and could help with reviews
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add missing ti.start and ti.finish metrics in Airflow 3 [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #62019:
URL: https://github.com/apache/airflow/pull/62019#discussion_r2812863527
##########
task-sdk/src/airflow/sdk/execution_time/task_runner.py:
##########
@@ -1283,6 +1288,13 @@ def _on_term(signum, frame):
msg, state = _handle_current_task_failed(ti)
error = e
finally:
+ Stats.incr(
+ f"ti.finish.{ti.dag_id}.{ti.task_id}.{state.value}",
+ tags=stats_tags,
+ )
+ # Same metric with tagging
+ Stats.incr("ti.finish", tags={**stats_tags, "state": state.value})
Review Comment:
Now, I realize that it's hard for anyone reading the code to figure out what's going on with the `DualStatsManager` params. Some comments will be useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add missing ti.start and ti.finish metrics in Airflow 3 [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #62019:
URL: https://github.com/apache/airflow/pull/62019#discussion_r2812822658
##########
task-sdk/src/airflow/sdk/execution_time/task_runner.py:
##########
@@ -1173,6 +1173,11 @@ def _on_term(signum, frame):
state: TaskInstanceState
error: BaseException | None = None
+ stats_tags = {"dag_id": ti.dag_id, "task_id": ti.task_id}
+ Stats.incr(f"ti.start.{ti.dag_id}.{ti.task_id}", tags=stats_tags)
+ # Same metric with tagging
+ Stats.incr("ti.start", tags=stats_tags)
Review Comment:
You should use the `DualStatsManager` instead of `Stats` because of the legacy metric.
`ti.start` is the regular metric name and the `ti.start.{ti.dag_id}.{ti.task_id}` is the legacy name.
```diff
- Stats.incr(f"ti.start.{ti.dag_id}.{ti.task_id}", tags=stats_tags)
- # Same metric with tagging
- Stats.incr("ti.start", tags=stats_tags)
+ DualStatsManager.incr("ti.start", extra_tags=stats_tags)
```
There is a new config option to disable legacy metrics. The `DualStatsManager` will check the option and skip the legacy name if enabled. Also it will read the `metrics_tamplate.yaml` and get the legacy name and the name tags from there
https://github.com/astronomer/airflow/blob/ff0c4bd334d215a1f2d717f1f0f2e1c25985c257/shared/observability/src/airflow_shared/observability/metrics/metrics_template.yaml#L167-L179
##########
task-sdk/src/airflow/sdk/execution_time/task_runner.py:
##########
@@ -1283,6 +1288,13 @@ def _on_term(signum, frame):
msg, state = _handle_current_task_failed(ti)
error = e
finally:
+ Stats.incr(
+ f"ti.finish.{ti.dag_id}.{ti.task_id}.{state.value}",
+ tags=stats_tags,
+ )
+ # Same metric with tagging
+ Stats.incr("ti.finish", tags={**stats_tags, "state": state.value})
Review Comment:
Similarly here
```diff
- Stats.incr(
- f"ti.finish.{ti.dag_id}.{ti.task_id}.{state.value}",
- tags=stats_tags,
- )
- # Same metric with tagging
- Stats.incr("ti.finish", tags={**stats_tags, "state": state.value})
+ DualStatsManager.incr("ti.finish", tags={"state": state.value}, extra_tags=stats_tags)
```
`tags` are used for both new and legacy metrics. `extra_tags` are referring to the tags on the legacy metric name.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add missing ti.start and ti.finish metrics in Airflow 3 [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh merged PR #62019:
URL: https://github.com/apache/airflow/pull/62019
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add missing ti.start and ti.finish metrics in Airflow 3 [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #62019:
URL: https://github.com/apache/airflow/pull/62019#discussion_r2815495131
##########
task-sdk/src/airflow/sdk/execution_time/task_runner.py:
##########
@@ -1173,6 +1173,11 @@ def _on_term(signum, frame):
state: TaskInstanceState
error: BaseException | None = None
+ stats_tags = {"dag_id": ti.dag_id, "task_id": ti.task_id}
+ Stats.incr(f"ti.start.{ti.dag_id}.{ti.task_id}", tags=stats_tags)
+ # Same metric with tagging
+ Stats.incr("ti.start", tags=stats_tags)
Review Comment:
@xBis7 thanks for that, I assessed whether to use `DualStatsManager` or not and decided to not because this is a bugfix for `3.1.8`, and 3.1.x doesn't have the shared observability library: https://github.com/apache/airflow/tree/v3-1-test/shared
Hence I decided to use the classic way of doing things to not divert the code during cherry pick. We can certainly follow up in a later PR to migrate all `Stats` usages in task sdk to use `DualStatsManager`, what do you think about that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add missing ti.start and ti.finish metrics in Airflow 3 [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #62019:
URL: https://github.com/apache/airflow/pull/62019#issuecomment-3919229007
### Backport successfully created: v3-1-test
Note: As of [Merging PRs targeted for Airflow 3.X](https://github.com/apache/airflow/blob/main/dev/README_AIRFLOW3_DEV.md#merging-prs-targeted-for-airflow-3x)
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.
In matter of doubt please ask in [#release-management](https://apache-airflow.slack.com/archives/C03G9H97MM2) Slack channel.
<table>
<tr>
<th>Status</th>
<th>Branch</th>
<th>Result</th>
</tr>
<tr>
<td>✅</td>
<td>v3-1-test</td>
<td><a href="https://github.com/apache/airflow/pull/62110"><img src="https://img.shields.io/badge/PR-62110-blue" alt="PR Link"></a></td>
</tr>
</table>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add missing ti.start and ti.finish metrics in Airflow 3 [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #62019:
URL: https://github.com/apache/airflow/pull/62019#discussion_r2852476334
##########
task-sdk/src/airflow/sdk/execution_time/task_runner.py:
##########
@@ -1173,6 +1173,11 @@ def _on_term(signum, frame):
state: TaskInstanceState
error: BaseException | None = None
+ stats_tags = {"dag_id": ti.dag_id, "task_id": ti.task_id}
+ Stats.incr(f"ti.start.{ti.dag_id}.{ti.task_id}", tags=stats_tags)
+ # Same metric with tagging
+ Stats.incr("ti.start", tags=stats_tags)
Review Comment:
@amoghrajesh DId you do this part?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add missing ti.start and ti.finish metrics in Airflow 3 [airflow]
Posted by "xBis7 (via GitHub)" <gi...@apache.org>.
xBis7 commented on code in PR #62019:
URL: https://github.com/apache/airflow/pull/62019#discussion_r2812863527
##########
task-sdk/src/airflow/sdk/execution_time/task_runner.py:
##########
@@ -1283,6 +1288,13 @@ def _on_term(signum, frame):
msg, state = _handle_current_task_failed(ti)
error = e
finally:
+ Stats.incr(
+ f"ti.finish.{ti.dag_id}.{ti.task_id}.{state.value}",
+ tags=stats_tags,
+ )
+ # Same metric with tagging
+ Stats.incr("ti.finish", tags={**stats_tags, "state": state.value})
Review Comment:
I realize now that it's hard for anyone reading the code to figure out what's going on with the `DualStatsManager` params. Some comments will be useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org