You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/11/28 21:51:46 UTC
[GitHub] [airflow] ashb opened a new pull request #12695: Remove deprecated dagbag metrics
ashb opened a new pull request #12695:
URL: https://github.com/apache/airflow/pull/12695
These were deprecated in 1.10.6 via #6157, so we should remove them
before 2.0 rolls around.
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #12695: Remove deprecated dagbag metrics
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #12695:
URL: https://github.com/apache/airflow/pull/12695#discussion_r532112742
##########
File path: airflow/models/dagbag.py
##########
@@ -467,17 +466,7 @@ def collect_dags(
except Exception as e: # pylint: disable=broad-except
self.log.exception(e)
- end_dttm = timezone.utcnow()
- durations = (end_dttm - start_dttm).total_seconds()
- Stats.gauge('collect_dags', durations, 1)
- Stats.gauge('dagbag_size', len(self.dags), 1)
Review comment:
Hmmm curious, I _did_ remove this in that PR. How did it come back..
Oh. It got added back in in #5743. Whoops. So metric was fixed in 1.10.6 and then broken again in 1.10.7 :(
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #12695: Remove deprecated dagbag metrics
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #12695:
URL: https://github.com/apache/airflow/pull/12695#discussion_r532112396
##########
File path: airflow/models/dagbag.py
##########
@@ -467,17 +466,7 @@ def collect_dags(
except Exception as e: # pylint: disable=broad-except
self.log.exception(e)
- end_dttm = timezone.utcnow()
- durations = (end_dttm - start_dttm).total_seconds()
- Stats.gauge('collect_dags', durations, 1)
- Stats.gauge('dagbag_size', len(self.dags), 1)
Review comment:
(It was probably a bug in my linked PR #6157 that this was still emitted here!)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on a change in pull request #12695: Remove deprecated dagbag metrics
Posted by GitBox <gi...@apache.org>.
turbaszek commented on a change in pull request #12695:
URL: https://github.com/apache/airflow/pull/12695#discussion_r532111071
##########
File path: airflow/models/dagbag.py
##########
@@ -467,17 +466,7 @@ def collect_dags(
except Exception as e: # pylint: disable=broad-except
self.log.exception(e)
- end_dttm = timezone.utcnow()
- durations = (end_dttm - start_dttm).total_seconds()
- Stats.gauge('collect_dags', durations, 1)
- Stats.gauge('dagbag_size', len(self.dags), 1)
Review comment:
Should we remove it from https://github.com/apache/airflow/blob/master/docs/logging-monitoring/metrics.rst ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #12695: Remove deprecated dagbag metrics
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #12695:
URL: https://github.com/apache/airflow/pull/12695#discussion_r532112332
##########
File path: airflow/models/dagbag.py
##########
@@ -467,17 +466,7 @@ def collect_dags(
except Exception as e: # pylint: disable=broad-except
self.log.exception(e)
- end_dttm = timezone.utcnow()
- durations = (end_dttm - start_dttm).total_seconds()
- Stats.gauge('collect_dags', durations, 1)
- Stats.gauge('dagbag_size', len(self.dags), 1)
Review comment:
It's still emitted here https://github.com/apache/airflow/blob/0a1b434d2895c3245f02e33cec40a64a92777876/airflow/utils/dag_processing.py#L1142-L1151
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #12695: Remove deprecated dagbag metrics
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #12695:
URL: https://github.com/apache/airflow/pull/12695#issuecomment-735376401
The PR needs to run all tests because it modifies core of Airflow! Please rebase it to latest master or ask committer to re-run it!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb merged pull request #12695: Remove deprecated dagbag metrics
Posted by GitBox <gi...@apache.org>.
ashb merged pull request #12695:
URL: https://github.com/apache/airflow/pull/12695
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #12695: Remove deprecated dagbag metrics
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #12695:
URL: https://github.com/apache/airflow/pull/12695#issuecomment-735377175
[The Workflow run](https://github.com/apache/airflow/actions/runs/389822268) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org