You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/07/21 14:08:29 UTC
[GitHub] [airflow] SuperQ opened a new pull request, #25219: Update statsd-exporter setup
SuperQ opened a new pull request, #25219:
URL: https://github.com/apache/airflow/pull/25219
In examing our use of the statsd_exporter in airflow, we found a couple
issues. We couldn't easily override the fork of the exporter image,
becuase it requires pushing statsd.extraMappings. We also found that the
exporter produces huge metric names (over 250 chars long) due to metrics
that are not matched in the mappings config.
* Always use the ConfigMap. This allows for easier use of the official
upstream Prometheus container image.
* Upgrade the statsd-exporter to the latest release.
* Update exporter config to use the new defaults section.
* Add a "drop unknown mappings" to reduce metrics spam.
Signed-off-by: SuperQ <su...@gmail.com>
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SuperQ commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
SuperQ commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1198374305
Ahh, you're right, I misremembered that the statsd_exporter expects a list of float64, not strings.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SuperQ commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
SuperQ commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1191648719
So far as I've seen, OTel is great for tracing. But everything else being tacked on is turning it into a bloated kitchen sink of a project. Trying to solve too many problems. And a lot of it is design by committee, trying to make every vendor happy. This makes no vendors happy.
Yes, Kamil is good people.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #25219:
URL: https://github.com/apache/airflow/pull/25219#discussion_r932450270
##########
chart/values.schema.json:
##########
@@ -3751,11 +3751,207 @@
"$ref": "#/definitions/io.k8s.api.core.v1.TopologySpreadConstraint"
}
},
+ "histogramBuckets": {
+ "description": "A list of histogram buckets for the StatsD exporter.",
+ "type": "array",
+ "default": [
+ ".005",
+ ".01",
+ ".025",
+ ".05",
+ ".1",
+ ".25",
+ ".5",
+ "1",
+ "2.5"
+ ],
+ "items": {
+ "type": "string"
+ }
+ },
+ "defaultMappings": {
+ "description": "Default mappings for StatsD exporter.",
+ "type": "array",
+ "default": [
+ {
+ "match": "airflow.dagrun.dependency-check.*.*",
+ "name": "airflow_dagrun_dependency_check",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.ti_failures",
+ "name": "airflow_ti_tasks_failures_total"
+ },
+ {
+ "match": "airflow.ti_successes",
+ "name": "airflow_ti_tasks_succeeded_total"
+ },
+ {
+ "match": "airflow.ti.start.*.*",
+ "name": "airflow_ti_tasks_started_total",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2"
+ }
+ },
+ {
+ "match": "airflow.ti.finish.*.*.*",
+ "name": "airflow_ti_tasks_finished_total",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2",
+ "task_state": "$3"
+ }
+ },
+ {
+ "match": "airflow.operator_successes_(.*)",
+ "match_type": "regex",
+ "name": "airflow_operator_successes",
+ "labels": {
+ "operator": "$1"
+ }
+ },
+ {
+ "match": "airflow.operator_failures_(.*)",
+ "match_type": "regex",
+ "name": "airflow_operator_failures",
+ "labels": {
+ "operator": "$1"
+ }
+ },
+ {
+ "match": "airflow.scheduler_heartbeat",
+ "match_type": "regex",
+ "name": "airflow_scheduler_heartbeat",
+ "labels": {
+ "type": "counter"
+ }
+ },
+ {
+ "match": "airflow.dag.*.*.duration",
+ "name": "airflow_task_duration",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2"
+ }
+ },
+ {
+ "match": "airflow.dagrun.duration.success.*",
+ "name": "airflow_dagrun_duration",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dagrun.duration.failed.*",
+ "name": "airflow_dagrun_failed",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dagrun.schedule_delay.*",
+ "name": "airflow_dagrun_schedule_delay",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dag_processing.last_runtime.*",
+ "name": "airflow_dag_processing_last_runtime",
+ "labels": {
+ "dag_file": "$1"
+ }
+ },
+ {
+ "match": "airflow.dag_processing.last_run.seconds_ago.*",
+ "name": "airflow_dag_processing_last_run_seconds_ago",
+ "labels": {
+ "dag_file": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.open_slots.*",
+ "name": "airflow_pool_open_slots",
+ "labels": {
+ "pool": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.used_slots.*",
+ "name": "airflow_pool_used_slots",
+ "labels": {
+ "pool": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.starving_tasks.*",
+ "name": "airflow_pool_starving_tasks",
+ "labels": {
+ "pool": "$1"
+ }
+ }
+ ],
+ "items": {
+ "type": "object",
+ "description": "Prometheus statsd_exporter mapping configuration",
+ "properties": {
Review Comment:
@SuperQ - FYI. these are standard JSON-Schema validations. IntelliJ for one (And I guess VScode and others) have great support for those and provides all the autocompletes and what-nots (which I found extremely useful to add validations without learning the whole of it).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SuperQ commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
SuperQ commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1195915282
Fixed a dupe issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SuperQ commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
SuperQ commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1194620682
Added more test fixes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SuperQ commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
SuperQ commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1195195795
Oops, fixed one small indenting mistake in the configmap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SuperQ commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
SuperQ commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1192771679
Yea, looks like I added a bug in the configmap. I'll fix it up soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SuperQ commented on a diff in pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
SuperQ commented on code in PR #25219:
URL: https://github.com/apache/airflow/pull/25219#discussion_r932453300
##########
chart/values.schema.json:
##########
@@ -3751,11 +3751,207 @@
"$ref": "#/definitions/io.k8s.api.core.v1.TopologySpreadConstraint"
}
},
+ "histogramBuckets": {
+ "description": "A list of histogram buckets for the StatsD exporter.",
+ "type": "array",
+ "default": [
+ ".005",
+ ".01",
+ ".025",
+ ".05",
+ ".1",
+ ".25",
+ ".5",
+ "1",
+ "2.5"
+ ],
+ "items": {
+ "type": "string"
+ }
+ },
+ "defaultMappings": {
+ "description": "Default mappings for StatsD exporter.",
+ "type": "array",
+ "default": [
+ {
+ "match": "airflow.dagrun.dependency-check.*.*",
+ "name": "airflow_dagrun_dependency_check",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.ti_failures",
+ "name": "airflow_ti_tasks_failures_total"
+ },
+ {
+ "match": "airflow.ti_successes",
+ "name": "airflow_ti_tasks_succeeded_total"
+ },
+ {
+ "match": "airflow.ti.start.*.*",
+ "name": "airflow_ti_tasks_started_total",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2"
+ }
+ },
+ {
+ "match": "airflow.ti.finish.*.*.*",
+ "name": "airflow_ti_tasks_finished_total",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2",
+ "task_state": "$3"
+ }
+ },
+ {
+ "match": "airflow.operator_successes_(.*)",
+ "match_type": "regex",
+ "name": "airflow_operator_successes",
+ "labels": {
+ "operator": "$1"
+ }
+ },
+ {
+ "match": "airflow.operator_failures_(.*)",
+ "match_type": "regex",
+ "name": "airflow_operator_failures",
+ "labels": {
+ "operator": "$1"
+ }
+ },
+ {
+ "match": "airflow.scheduler_heartbeat",
+ "match_type": "regex",
+ "name": "airflow_scheduler_heartbeat",
+ "labels": {
+ "type": "counter"
+ }
+ },
+ {
+ "match": "airflow.dag.*.*.duration",
+ "name": "airflow_task_duration",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2"
+ }
+ },
+ {
+ "match": "airflow.dagrun.duration.success.*",
+ "name": "airflow_dagrun_duration",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dagrun.duration.failed.*",
+ "name": "airflow_dagrun_failed",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dagrun.schedule_delay.*",
+ "name": "airflow_dagrun_schedule_delay",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dag_processing.last_runtime.*",
+ "name": "airflow_dag_processing_last_runtime",
+ "labels": {
+ "dag_file": "$1"
+ }
+ },
+ {
+ "match": "airflow.dag_processing.last_run.seconds_ago.*",
+ "name": "airflow_dag_processing_last_run_seconds_ago",
+ "labels": {
+ "dag_file": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.open_slots.*",
+ "name": "airflow_pool_open_slots",
+ "labels": {
+ "pool": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.used_slots.*",
+ "name": "airflow_pool_used_slots",
+ "labels": {
+ "pool": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.starving_tasks.*",
+ "name": "airflow_pool_starving_tasks",
+ "labels": {
+ "pool": "$1"
+ }
+ }
+ ],
+ "items": {
+ "type": "object",
+ "description": "Prometheus statsd_exporter mapping configuration",
+ "properties": {
Review Comment:
Yea, things I don't use :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1191532307
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:
- Pay attention to the quality of your code (flake8, mypy and type annotations). Our [pre-commits]( https://github.com/apache/airflow/blob/main/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks) will help you with that.
- In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/main/docs/apache-airflow/howto/custom-operator.rst) Consider adding an example DAG that shows how users should use it.
- Consider using [Breeze environment](https://github.com/apache/airflow/blob/main/BREEZE.rst) for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
- Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
- Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
- Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#coding-style-and-best-practices).
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SuperQ commented on a diff in pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
SuperQ commented on code in PR #25219:
URL: https://github.com/apache/airflow/pull/25219#discussion_r932469014
##########
chart/values.schema.json:
##########
@@ -3751,11 +3751,207 @@
"$ref": "#/definitions/io.k8s.api.core.v1.TopologySpreadConstraint"
}
},
+ "histogramBuckets": {
+ "description": "A list of histogram buckets for the StatsD exporter.",
+ "type": "array",
+ "default": [
+ ".005",
+ ".01",
+ ".025",
+ ".05",
+ ".1",
+ ".25",
+ ".5",
+ "1",
+ "2.5"
+ ],
+ "items": {
+ "type": "string"
+ }
+ },
+ "defaultMappings": {
+ "description": "Default mappings for StatsD exporter.",
+ "type": "array",
+ "default": [
+ {
+ "match": "airflow.dagrun.dependency-check.*.*",
+ "name": "airflow_dagrun_dependency_check",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.ti_failures",
+ "name": "airflow_ti_tasks_failures_total"
+ },
+ {
+ "match": "airflow.ti_successes",
+ "name": "airflow_ti_tasks_succeeded_total"
+ },
+ {
+ "match": "airflow.ti.start.*.*",
+ "name": "airflow_ti_tasks_started_total",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2"
+ }
+ },
+ {
+ "match": "airflow.ti.finish.*.*.*",
+ "name": "airflow_ti_tasks_finished_total",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2",
+ "task_state": "$3"
+ }
+ },
+ {
+ "match": "airflow.operator_successes_(.*)",
+ "match_type": "regex",
+ "name": "airflow_operator_successes",
+ "labels": {
+ "operator": "$1"
+ }
+ },
+ {
+ "match": "airflow.operator_failures_(.*)",
+ "match_type": "regex",
+ "name": "airflow_operator_failures",
+ "labels": {
+ "operator": "$1"
+ }
+ },
+ {
+ "match": "airflow.scheduler_heartbeat",
+ "match_type": "regex",
+ "name": "airflow_scheduler_heartbeat",
+ "labels": {
+ "type": "counter"
+ }
+ },
+ {
+ "match": "airflow.dag.*.*.duration",
+ "name": "airflow_task_duration",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2"
+ }
+ },
+ {
+ "match": "airflow.dagrun.duration.success.*",
+ "name": "airflow_dagrun_duration",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dagrun.duration.failed.*",
+ "name": "airflow_dagrun_failed",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dagrun.schedule_delay.*",
+ "name": "airflow_dagrun_schedule_delay",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dag_processing.last_runtime.*",
+ "name": "airflow_dag_processing_last_runtime",
+ "labels": {
+ "dag_file": "$1"
+ }
+ },
+ {
+ "match": "airflow.dag_processing.last_run.seconds_ago.*",
+ "name": "airflow_dag_processing_last_run_seconds_ago",
+ "labels": {
+ "dag_file": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.open_slots.*",
+ "name": "airflow_pool_open_slots",
+ "labels": {
+ "pool": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.used_slots.*",
+ "name": "airflow_pool_used_slots",
+ "labels": {
+ "pool": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.starving_tasks.*",
+ "name": "airflow_pool_starving_tasks",
+ "labels": {
+ "pool": "$1"
+ }
+ }
+ ],
+ "items": {
+ "type": "object",
+ "description": "Prometheus statsd_exporter mapping configuration",
+ "properties": {
Review Comment:
I spend most of my time contributing to random projects. Mostly Go code. So this is a bit out of the ordinary for my workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on a diff in pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
jedcunningham commented on code in PR #25219:
URL: https://github.com/apache/airflow/pull/25219#discussion_r939123022
##########
chart/values.yaml:
##########
@@ -1314,9 +1314,96 @@ statsd:
priorityClassName: ~
+ histogramBuckets:
+ - 0.005
+ - 0.01
+ - 0.025
+ - 0.05
+ - 0.1
+ - 0.25
+ - 0.5
+ - 1
+ - 2.5
+
+ defaultMappings:
Review Comment:
I wonder if we should call this '`mappings' instead`?
##########
chart/values.yaml:
##########
@@ -1314,9 +1314,96 @@ statsd:
priorityClassName: ~
+ histogramBuckets:
+ - 0.005
+ - 0.01
+ - 0.025
+ - 0.05
+ - 0.1
+ - 0.25
+ - 0.5
+ - 1
+ - 2.5
+
+ defaultMappings:
+ # Map dot separated stats to labels
+ - match: airflow.dagrun.dependency-check.*.*
+ name: "airflow_dagrun_dependency_check"
+ labels:
+ dag_id: "$1"
+ - match: airflow.ti_failures
+ name: "airflow_ti_tasks_failures_total"
+ - match: airflow.ti_successes
+ name: "airflow_ti_tasks_succeeded_total"
+ - match: airflow.ti.start.*.*
+ name: "airflow_ti_tasks_started_total"
+ labels:
+ dag_id: "$1"
+ task_id: "$2"
+ - match: airflow.ti.finish.*.*.*
+ name: "airflow_ti_tasks_finished_total"
+ labels:
+ dag_id: "$1"
+ task_id: "$2"
+ task_state: "$3"
Review Comment:
These are all new, right? I think this will also break backwards compatibility, no?
##########
chart/templates/configmaps/statsd-configmap.yaml:
##########
@@ -34,6 +34,20 @@ metadata:
{{- end }}
data:
mappings.yml: |-
-{{ .Files.Get "dockerfiles/statsd-exporter/mappings.yml" | indent 4 }}
+ defaults:
+ observer_type: histogram
+ histogram_options:
+ buckets:
+{{ toYaml .Values.statsd.histogramBuckets | indent 10 }}
Review Comment:
So looking at this from a backwards compatibility lens, I wonder if we should default histogramBuckets to null and only add this if a user elects to.
I'm not familiar enough to know if this is a breaking change or not in prometheus.
For example, before it was like this:
```
airflow_dag_processing_last_duration_hello_world{quantile="0.5"} 0.122281
airflow_dag_processing_last_duration_hello_world{quantile="0.9"} 0.146598
airflow_dag_processing_last_duration_hello_world{quantile="0.99"} 0.234025
```
With this change, it looks like this:
```
airflow_dag_processing_last_duration_hello_world_bucket{le="0.005"} 0
airflow_dag_processing_last_duration_hello_world_bucket{le="0.01"} 0
...
airflow_dag_processing_last_duration_hello_world_bucket{le="2.5"} 23
airflow_dag_processing_last_duration_hello_world_bucket{le="+Inf"} 23
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1304938691
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1191641848
> Eh, I would prefer to just implement Prometheus/OpenMetrics native metrics. ;-)
>
> I'm not really a fan of OpenTelemetry.
Why ? Srsly - we have not yet implemented it but any insight from someone whe works on Prometheus might be valuable.
BTW. I think you might know Kamil Trzcinski, who is a dear friend of mine :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SuperQ commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
SuperQ commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1191639636
Eh, I would prefer to just implement Prometheus/OpenMetrics native metrics. ;-)
I'm not really a fan of OpenTelemetry.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SuperQ commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
SuperQ commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1191592232
@potiuk Yes, there is no change to the actual mapped metrics. It's just moving the config from embedded in the image to always using a ConfigMap.
It _will_ drop unknown statsd metrics. But this is really necessary because without mappings, it can completely hose a Prometheus instance with thousands of randomly named metrics.
Looking at our metadata endpoint, I see over 18,000 metric names coming from airflow right now. That's over half of our total metric name metadata. Also due to the length of the random names, it's 75% of the data by bytes (over 2MiB of json output into Grafana).
I'm happy to contribute additional mappings to make sure people don't miss out on things exposed over statsd.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SuperQ commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
SuperQ commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1195092893
Ok, I think I fixed the last of the test issues. I didn't realize the values.yaml schema valdation was that complicated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on a diff in pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
jedcunningham commented on code in PR #25219:
URL: https://github.com/apache/airflow/pull/25219#discussion_r932333388
##########
chart/values.schema.json:
##########
@@ -3751,11 +3751,207 @@
"$ref": "#/definitions/io.k8s.api.core.v1.TopologySpreadConstraint"
}
},
+ "histogramBuckets": {
+ "description": "A list of histogram buckets for the StatsD exporter.",
+ "type": "array",
+ "default": [
+ ".005",
+ ".01",
+ ".025",
+ ".05",
+ ".1",
+ ".25",
+ ".5",
+ "1",
+ "2.5"
+ ],
+ "items": {
+ "type": "string"
+ }
+ },
+ "defaultMappings": {
+ "description": "Default mappings for StatsD exporter.",
+ "type": "array",
+ "default": [
+ {
+ "match": "airflow.dagrun.dependency-check.*.*",
+ "name": "airflow_dagrun_dependency_check",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.ti_failures",
+ "name": "airflow_ti_tasks_failures_total"
+ },
+ {
+ "match": "airflow.ti_successes",
+ "name": "airflow_ti_tasks_succeeded_total"
+ },
+ {
+ "match": "airflow.ti.start.*.*",
+ "name": "airflow_ti_tasks_started_total",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2"
+ }
+ },
+ {
+ "match": "airflow.ti.finish.*.*.*",
+ "name": "airflow_ti_tasks_finished_total",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2",
+ "task_state": "$3"
+ }
+ },
+ {
+ "match": "airflow.operator_successes_(.*)",
+ "match_type": "regex",
+ "name": "airflow_operator_successes",
+ "labels": {
+ "operator": "$1"
+ }
+ },
+ {
+ "match": "airflow.operator_failures_(.*)",
+ "match_type": "regex",
+ "name": "airflow_operator_failures",
+ "labels": {
+ "operator": "$1"
+ }
+ },
+ {
+ "match": "airflow.scheduler_heartbeat",
+ "match_type": "regex",
+ "name": "airflow_scheduler_heartbeat",
+ "labels": {
+ "type": "counter"
+ }
+ },
+ {
+ "match": "airflow.dag.*.*.duration",
+ "name": "airflow_task_duration",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2"
+ }
+ },
+ {
+ "match": "airflow.dagrun.duration.success.*",
+ "name": "airflow_dagrun_duration",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dagrun.duration.failed.*",
+ "name": "airflow_dagrun_failed",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dagrun.schedule_delay.*",
+ "name": "airflow_dagrun_schedule_delay",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dag_processing.last_runtime.*",
+ "name": "airflow_dag_processing_last_runtime",
+ "labels": {
+ "dag_file": "$1"
+ }
+ },
+ {
+ "match": "airflow.dag_processing.last_run.seconds_ago.*",
+ "name": "airflow_dag_processing_last_run_seconds_ago",
+ "labels": {
+ "dag_file": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.open_slots.*",
+ "name": "airflow_pool_open_slots",
+ "labels": {
+ "pool": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.used_slots.*",
+ "name": "airflow_pool_used_slots",
+ "labels": {
+ "pool": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.starving_tasks.*",
+ "name": "airflow_pool_starving_tasks",
+ "labels": {
+ "pool": "$1"
+ }
+ }
+ ],
+ "items": {
+ "type": "object",
+ "description": "Prometheus statsd_exporter mapping configuration",
+ "properties": {
Review Comment:
```suggestion
"additionalProperties": false,
"properties": {
```
Probably want this as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1191556353
If we can completely get rid of the custom image - I am all for it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1251699785
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] BobDu commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
BobDu commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1255118674
I had the same problem
When I try to modify some default mapping configuration, if just add a new mapping item, using extraMapping can fully needs,
but, if add it conflicts with the default mapping item, custom configuration does not take effect.
Because now default configuration always takes higher priority.
```
mappings.yml: |-
{{ .Files.Get "dockerfiles/statsd-exporter/mappings.yml" | indent 4 }}
{{ toYaml .Values.statsd.extraMappings | indent 6 }}
{{- end }}
```
There are only two options for it.
1. fork and change the helm chart
2. custom build my statsd expoorter image with my mappings file.
both bad practice.
I believe this PR can reslove this probelm.
But, By reading reviews comment, I realized because this want to add some new metrics mapping,
reviewers worry will break backwards compatibility.
So, it needs to be more carefully justified.
I'm trying to split it into two steps.
First, only provided a optional choose: overideMapping, no change an default behavior, it should be easier and faster to merge.
Then, we continue to carefully add metrics in this PR.
So, I create a new PR #26598 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SuperQ commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
SuperQ commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1191551156
Thinking about this some more, what if we completely eliminated the airflow image fork and embedded mappings config. We could move the current mappings to `values.yaml` as `statsd.defaultMappings`. This would make it even easier to override things in the default setup.
We can still keep the `statsd.extraMappings` for easy extension of the defaults.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1191570993
Just loudly thinking - are the metrics exported now batckwards-compatible with the old ones? @jedcunningham @ephraimbuddy you might want to take a look and see if it also works for Astroonomer (I believe it came from Astronomer's Chart /image originally)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #25219:
URL: https://github.com/apache/airflow/pull/25219#discussion_r932455969
##########
chart/values.schema.json:
##########
@@ -3751,11 +3751,207 @@
"$ref": "#/definitions/io.k8s.api.core.v1.TopologySpreadConstraint"
}
},
+ "histogramBuckets": {
+ "description": "A list of histogram buckets for the StatsD exporter.",
+ "type": "array",
+ "default": [
+ ".005",
+ ".01",
+ ".025",
+ ".05",
+ ".1",
+ ".25",
+ ".5",
+ "1",
+ "2.5"
+ ],
+ "items": {
+ "type": "string"
+ }
+ },
+ "defaultMappings": {
+ "description": "Default mappings for StatsD exporter.",
+ "type": "array",
+ "default": [
+ {
+ "match": "airflow.dagrun.dependency-check.*.*",
+ "name": "airflow_dagrun_dependency_check",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.ti_failures",
+ "name": "airflow_ti_tasks_failures_total"
+ },
+ {
+ "match": "airflow.ti_successes",
+ "name": "airflow_ti_tasks_succeeded_total"
+ },
+ {
+ "match": "airflow.ti.start.*.*",
+ "name": "airflow_ti_tasks_started_total",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2"
+ }
+ },
+ {
+ "match": "airflow.ti.finish.*.*.*",
+ "name": "airflow_ti_tasks_finished_total",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2",
+ "task_state": "$3"
+ }
+ },
+ {
+ "match": "airflow.operator_successes_(.*)",
+ "match_type": "regex",
+ "name": "airflow_operator_successes",
+ "labels": {
+ "operator": "$1"
+ }
+ },
+ {
+ "match": "airflow.operator_failures_(.*)",
+ "match_type": "regex",
+ "name": "airflow_operator_failures",
+ "labels": {
+ "operator": "$1"
+ }
+ },
+ {
+ "match": "airflow.scheduler_heartbeat",
+ "match_type": "regex",
+ "name": "airflow_scheduler_heartbeat",
+ "labels": {
+ "type": "counter"
+ }
+ },
+ {
+ "match": "airflow.dag.*.*.duration",
+ "name": "airflow_task_duration",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2"
+ }
+ },
+ {
+ "match": "airflow.dagrun.duration.success.*",
+ "name": "airflow_dagrun_duration",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dagrun.duration.failed.*",
+ "name": "airflow_dagrun_failed",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dagrun.schedule_delay.*",
+ "name": "airflow_dagrun_schedule_delay",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dag_processing.last_runtime.*",
+ "name": "airflow_dag_processing_last_runtime",
+ "labels": {
+ "dag_file": "$1"
+ }
+ },
+ {
+ "match": "airflow.dag_processing.last_run.seconds_ago.*",
+ "name": "airflow_dag_processing_last_run_seconds_ago",
+ "labels": {
+ "dag_file": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.open_slots.*",
+ "name": "airflow_pool_open_slots",
+ "labels": {
+ "pool": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.used_slots.*",
+ "name": "airflow_pool_used_slots",
+ "labels": {
+ "pool": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.starving_tasks.*",
+ "name": "airflow_pool_starving_tasks",
+ "labels": {
+ "pool": "$1"
+ }
+ }
+ ],
+ "items": {
+ "type": "object",
+ "description": "Prometheus statsd_exporter mapping configuration",
+ "properties": {
Review Comment:
Maybe you should :P . I also STILL use vim when I want to do something quick - but IntelliJ is something I happily pay for for years and I get Yearly subscription in ~ 1 day back in form of saved hours
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SuperQ commented on a diff in pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
SuperQ commented on code in PR #25219:
URL: https://github.com/apache/airflow/pull/25219#discussion_r932445610
##########
chart/values.schema.json:
##########
@@ -3751,11 +3751,207 @@
"$ref": "#/definitions/io.k8s.api.core.v1.TopologySpreadConstraint"
}
},
+ "histogramBuckets": {
+ "description": "A list of histogram buckets for the StatsD exporter.",
+ "type": "array",
+ "default": [
+ ".005",
+ ".01",
+ ".025",
+ ".05",
+ ".1",
+ ".25",
+ ".5",
+ "1",
+ "2.5"
+ ],
+ "items": {
+ "type": "string"
+ }
+ },
+ "defaultMappings": {
+ "description": "Default mappings for StatsD exporter.",
+ "type": "array",
+ "default": [
+ {
+ "match": "airflow.dagrun.dependency-check.*.*",
+ "name": "airflow_dagrun_dependency_check",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.ti_failures",
+ "name": "airflow_ti_tasks_failures_total"
+ },
+ {
+ "match": "airflow.ti_successes",
+ "name": "airflow_ti_tasks_succeeded_total"
+ },
+ {
+ "match": "airflow.ti.start.*.*",
+ "name": "airflow_ti_tasks_started_total",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2"
+ }
+ },
+ {
+ "match": "airflow.ti.finish.*.*.*",
+ "name": "airflow_ti_tasks_finished_total",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2",
+ "task_state": "$3"
+ }
+ },
+ {
+ "match": "airflow.operator_successes_(.*)",
+ "match_type": "regex",
+ "name": "airflow_operator_successes",
+ "labels": {
+ "operator": "$1"
+ }
+ },
+ {
+ "match": "airflow.operator_failures_(.*)",
+ "match_type": "regex",
+ "name": "airflow_operator_failures",
+ "labels": {
+ "operator": "$1"
+ }
+ },
+ {
+ "match": "airflow.scheduler_heartbeat",
+ "match_type": "regex",
+ "name": "airflow_scheduler_heartbeat",
+ "labels": {
+ "type": "counter"
+ }
+ },
+ {
+ "match": "airflow.dag.*.*.duration",
+ "name": "airflow_task_duration",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2"
+ }
+ },
+ {
+ "match": "airflow.dagrun.duration.success.*",
+ "name": "airflow_dagrun_duration",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dagrun.duration.failed.*",
+ "name": "airflow_dagrun_failed",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dagrun.schedule_delay.*",
+ "name": "airflow_dagrun_schedule_delay",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dag_processing.last_runtime.*",
+ "name": "airflow_dag_processing_last_runtime",
+ "labels": {
+ "dag_file": "$1"
+ }
+ },
+ {
+ "match": "airflow.dag_processing.last_run.seconds_ago.*",
+ "name": "airflow_dag_processing_last_run_seconds_ago",
+ "labels": {
+ "dag_file": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.open_slots.*",
+ "name": "airflow_pool_open_slots",
+ "labels": {
+ "pool": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.used_slots.*",
+ "name": "airflow_pool_used_slots",
+ "labels": {
+ "pool": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.starving_tasks.*",
+ "name": "airflow_pool_starving_tasks",
+ "labels": {
+ "pool": "$1"
+ }
+ }
+ ],
+ "items": {
+ "type": "object",
+ "description": "Prometheus statsd_exporter mapping configuration",
+ "properties": {
Review Comment:
Thanks, I'm still trying to figure out the exact syntax/use of these validators. I'm attempting to run some of the presubmit scripts, but since I don't work on this every day it's probably not setup correctly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SuperQ commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
SuperQ commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1251876246
!stale, just need to find time to get back to this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SuperQ commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
SuperQ commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1191563135
Done!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1192753651
Some tests are failing :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1191670866
Thanks for the insights @SuperQ . I think might be good input for @howardyoo :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1191679294
BTW. Some of our helm tests need fixing :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1195185222
> Ok, I think I fixed the last of the test issues. I didn't realize the values.yaml schema valdation was that complicated.
Or rather has so many safeguards :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
potiuk commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1191634642
This is cool. I think though rather than tags, we should focus rather quickly on implementing Open Telemetry https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-49+OpenTelemetry+Support+for+Apache+Airflow
It's already been approved, we just need to implement it :)
If you would like to help I think @howardyoo might make a note that you could test/help in that effort?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SuperQ commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
SuperQ commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1191593399
And while I'm musing about, long-term it would be helpful if the airflow statsd included tagged data, rather than traditional statsd dotted trees only. This would make it far simpler to convert between monitoring systems.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] howardyoo commented on pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
howardyoo commented on PR #25219:
URL: https://github.com/apache/airflow/pull/25219#issuecomment-1191723459
Yes, thanks for the feedback.
I believe in general, metrics side of OpenTelemetry has gotten much better
(I mean metrics are kind of simple by nature) - so still think the
specification has a good potential. We'll just have to see how it goes!
Howard
On Thu, Jul 21, 2022 at 11:05 AM Jarek Potiuk ***@***.***>
wrote:
> Thanks for the insights @SuperQ <https://github.com/SuperQ> . I think
> might be good input for @howardyoo <https://github.com/howardyoo> :)
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/airflow/pull/25219#issuecomment-1191670866>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AHZNLLVNOFUQ5R4HSAFBXN3VVFYLLANCNFSM54HZEBTA>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a diff in pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
potiuk commented on code in PR #25219:
URL: https://github.com/apache/airflow/pull/25219#discussion_r932450270
##########
chart/values.schema.json:
##########
@@ -3751,11 +3751,207 @@
"$ref": "#/definitions/io.k8s.api.core.v1.TopologySpreadConstraint"
}
},
+ "histogramBuckets": {
+ "description": "A list of histogram buckets for the StatsD exporter.",
+ "type": "array",
+ "default": [
+ ".005",
+ ".01",
+ ".025",
+ ".05",
+ ".1",
+ ".25",
+ ".5",
+ "1",
+ "2.5"
+ ],
+ "items": {
+ "type": "string"
+ }
+ },
+ "defaultMappings": {
+ "description": "Default mappings for StatsD exporter.",
+ "type": "array",
+ "default": [
+ {
+ "match": "airflow.dagrun.dependency-check.*.*",
+ "name": "airflow_dagrun_dependency_check",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.ti_failures",
+ "name": "airflow_ti_tasks_failures_total"
+ },
+ {
+ "match": "airflow.ti_successes",
+ "name": "airflow_ti_tasks_succeeded_total"
+ },
+ {
+ "match": "airflow.ti.start.*.*",
+ "name": "airflow_ti_tasks_started_total",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2"
+ }
+ },
+ {
+ "match": "airflow.ti.finish.*.*.*",
+ "name": "airflow_ti_tasks_finished_total",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2",
+ "task_state": "$3"
+ }
+ },
+ {
+ "match": "airflow.operator_successes_(.*)",
+ "match_type": "regex",
+ "name": "airflow_operator_successes",
+ "labels": {
+ "operator": "$1"
+ }
+ },
+ {
+ "match": "airflow.operator_failures_(.*)",
+ "match_type": "regex",
+ "name": "airflow_operator_failures",
+ "labels": {
+ "operator": "$1"
+ }
+ },
+ {
+ "match": "airflow.scheduler_heartbeat",
+ "match_type": "regex",
+ "name": "airflow_scheduler_heartbeat",
+ "labels": {
+ "type": "counter"
+ }
+ },
+ {
+ "match": "airflow.dag.*.*.duration",
+ "name": "airflow_task_duration",
+ "labels": {
+ "dag_id": "$1",
+ "task_id": "$2"
+ }
+ },
+ {
+ "match": "airflow.dagrun.duration.success.*",
+ "name": "airflow_dagrun_duration",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dagrun.duration.failed.*",
+ "name": "airflow_dagrun_failed",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dagrun.schedule_delay.*",
+ "name": "airflow_dagrun_schedule_delay",
+ "labels": {
+ "dag_id": "$1"
+ }
+ },
+ {
+ "match": "airflow.dag_processing.last_runtime.*",
+ "name": "airflow_dag_processing_last_runtime",
+ "labels": {
+ "dag_file": "$1"
+ }
+ },
+ {
+ "match": "airflow.dag_processing.last_run.seconds_ago.*",
+ "name": "airflow_dag_processing_last_run_seconds_ago",
+ "labels": {
+ "dag_file": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.open_slots.*",
+ "name": "airflow_pool_open_slots",
+ "labels": {
+ "pool": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.used_slots.*",
+ "name": "airflow_pool_used_slots",
+ "labels": {
+ "pool": "$1"
+ }
+ },
+ {
+ "match": "airflow.pool.starving_tasks.*",
+ "name": "airflow_pool_starving_tasks",
+ "labels": {
+ "pool": "$1"
+ }
+ }
+ ],
+ "items": {
+ "type": "object",
+ "description": "Prometheus statsd_exporter mapping configuration",
+ "properties": {
Review Comment:
@SuperQ - BTW. these are standard JSON-Schema validations. IntelliJ for one (And I guess VScode and others) have great support for those and provides all the autocompletes and what-nots (which I found extremely useful to add validations without learning the whole of it).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] closed pull request #25219: Update statsd-exporter setup
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed pull request #25219: Update statsd-exporter setup
URL: https://github.com/apache/airflow/pull/25219
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org