You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Jeoffreybauvin (via GitHub)" <gi...@apache.org> on 2023/12/12 16:07:32 UTC
[I] dag_processing.processes should be a gauge, not a counter [airflow]
Jeoffreybauvin opened a new issue, #36188:
URL: https://github.com/apache/airflow/issues/36188
### What do you see as an issue?
I wasn't sure if it's a doc issue or a code issue.
For me, dag_processing.processes should be considered as a gauge, not a counter, due to negative values.
`statsd | ts=2023-12-12T15:55:25.099Z caller=exporter.go:126 level=debug msg="counter must be non-negative value" metric=airflow_dag_processing_processes event_value=-1`
### Solving the problem
Define dag_processing.processes as a gauge.
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] dag_processing.processes should be a gauge, not a counter [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #36188:
URL: https://github.com/apache/airflow/issues/36188#issuecomment-2661287098
This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activity on this issue. Kindly asking to recheck the report against latest Airflow version and let us know if the issue is reproducible. The issue will be closed in next 30 days if no further activity occurs from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] dag_processing.processes should be a gauge, not a counter [airflow]
Posted by "Bowrna (via GitHub)" <gi...@apache.org>.
Bowrna commented on issue #36188:
URL: https://github.com/apache/airflow/issues/36188#issuecomment-1870204105
could i pick up this issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] dag_processing.processes should be a gauge, not a counter [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #36188:
URL: https://github.com/apache/airflow/issues/36188#issuecomment-2746059384
This issue has been closed because it has not received response from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] dag_processing.processes should be a gauge, not a counter [airflow]
Posted by "dirrao (via GitHub)" <gi...@apache.org>.
dirrao commented on issue #36188:
URL: https://github.com/apache/airflow/issues/36188#issuecomment-1870790987
> could i pick up this issue?
Yes. you can pick this task.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] dag_processing.processes should be a gauge, not a counter [airflow]
Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala commented on issue #36188:
URL: https://github.com/apache/airflow/issues/36188#issuecomment-1859223334
I just checked OTEL code source and found that it doesn't allow negative results in all its counters (including `UpDownCounter`, which we use); maybe we need to implement a class to support relative Gauges with increase/decrease methods.
@ferruzzi WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] dag_processing.processes should be a gauge, not a counter [airflow]
Posted by "dirrao (via GitHub)" <gi...@apache.org>.
dirrao commented on issue #36188:
URL: https://github.com/apache/airflow/issues/36188#issuecomment-1868745845
I think we can publish the current number of running parsing processors as a gauge. I see it being updated if there is a new processor added or stopped/terminated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] dag_processing.processes should be a gauge, not a counter [airflow]
Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala commented on issue #36188:
URL: https://github.com/apache/airflow/issues/36188#issuecomment-1859218428
@Jeoffreybauvin Defining `dag_processing.processes` as a gauge is not simple because it's relative, so we use `increment` and `decrement` to update it, and these methods are not available for gauge.
Also, it's fine to have negative values in the counter, but it looks like the exception is raised in Airflow. I will check if there is a problem in the counter, or we just need to remove the validation check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] dag_processing.processes should be a gauge, not a counter [airflow]
Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala commented on issue #36188:
URL: https://github.com/apache/airflow/issues/36188#issuecomment-1928444128
Yes, a new metric will be better to avoid introducing a breaking change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] dag_processing.processes should be a gauge, not a counter [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #36188: dag_processing.processes should be a gauge, not a counter
URL: https://github.com/apache/airflow/issues/36188
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] dag_processing.processes should be a gauge, not a counter [airflow]
Posted by "Bowrna (via GitHub)" <gi...@apache.org>.
Bowrna commented on issue #36188:
URL: https://github.com/apache/airflow/issues/36188#issuecomment-1925168423
@dirrao @potiuk @ferruzzi @hussein-awala Do you think we have to fix in the existing key and edit the docs to or write this in new key and have a backward compatibility and slowly deprecate it in the upcoming release?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] dag_processing.processes should be a gauge, not a counter [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #36188:
URL: https://github.com/apache/airflow/issues/36188#issuecomment-1869506421
> I think we can publish the current number of running parsing processors as a gauge. I see it being updated if there is a new processor added or stopped/terminated.
SGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] dag_processing.processes should be a gauge, not a counter [airflow]
Posted by "Bowrna (via GitHub)" <gi...@apache.org>.
Bowrna commented on issue #36188:
URL: https://github.com/apache/airflow/issues/36188#issuecomment-1948134350
cool let me start this work by this weekend. its a tough past 3 weeks. hope to finish it soon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] dag_processing.processes should be a gauge, not a counter [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #36188:
URL: https://github.com/apache/airflow/issues/36188#issuecomment-1853104463
cc: @ferruzzi ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] dag_processing.processes should be a gauge, not a counter [airflow]
Posted by "ferruzzi (via GitHub)" <gi...@apache.org>.
ferruzzi commented on issue #36188:
URL: https://github.com/apache/airflow/issues/36188#issuecomment-1863513364
Sorry for the delay, I've been dealing with laptop issues and may be slow to reply.
From the [Airflow Metrics docs}(https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/logging-monitoring/metrics.html), `dag_processing.processes` represents:
```Relative number of currently running DAG parsing processes (ie this delta is negative when, since the last metric was sent, processes have completed)```
In OTel, a Counter is an incremental value, like the odometer in your car, it can only go up. A Gauge is like your speedometer, it only shows the current value.
Neither of these feel too accurate to the intent of the metric. The [OTel specifications](https://opentelemetry.io/docs/specs/otel/metrics/api/#updowncounter), doesn't say an UpDownCounter must be positive. If that is actually true in the python implementation, that may be a bug.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] dag_processing.processes should be a gauge, not a counter [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #36188:
URL: https://github.com/apache/airflow/issues/36188#issuecomment-1928121937
We need a new metric, I think and deprecate the old one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org