You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "sebastianliebscher (via GitHub)" <gi...@apache.org> on 2023/11/02 17:40:24 UTC
[PR] refactor(logging): Clean up `statsd` logging [superset]
sebastianliebscher opened a new pull request, #25847:
URL: https://github.com/apache/superset/pull/25847
### SUMMARY
This PR removes the `log_to_statsd=` functionality from the `@event_logger` and replaces usage with already existing `@statsd_metrics`.
Currently, there are two decorators for basically the same use case:
- `@event_logger.log_this_with_context(log_to_statsd=False)`
- `@statsd_metrics`
The `@event_logger` just increments a statsd counter, while the `@statsd_metrics` additionally measures the execution time of a function.
**Note**: Most API endpoint already use `@statsd_metrics`. In a case where this decorator is added to replace `log_to_statsd=True`, the statsd metric
- `superset_<Endpoint>`
will be replaced by
- `superset_<Class>_<Endpoint>_success|warning|error`
- `superset_<Class>_<Endpoint>_time`
- `superset_<Class>_<Endpoint>_time_count`
- `superset_<Class>_<Endpoint>_time_sum`
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
<!--- Skip this if not applicable -->
### TESTING INSTRUCTIONS
<!--- Required! What steps can be taken to manually verify the changes? -->
### ADDITIONAL INFORMATION
<!--- Check any relevant boxes with "x" -->
<!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue -->
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] refactor(logging): Clean up `statsd` logging [superset]
Posted by "sebastianliebscher (via GitHub)" <gi...@apache.org>.
sebastianliebscher commented on code in PR #25847:
URL: https://github.com/apache/superset/pull/25847#discussion_r1380574022
##########
superset/async_events/api.py:
##########
@@ -33,6 +34,7 @@ class AsyncEventsRestApi(BaseSupersetApi):
allow_browser_login = True
@expose("/", methods=("GET",))
+ @statsd_metrics
@event_logger.log_this
@protect()
Review Comment:
Most API endpoint already use `@statsd_metrics`. In a case where this decorator is added to replace implicit `log_to_statsd=True`, the statsd metric
- `superset_<Endpoint>`
will be replaced by
- `superset_<Class>_<Endpoint>_success|warning|error`
- `superset_<Class>_<Endpoint>_time`
- `superset_<Class>_<Endpoint>_time_count`
- `superset_<Class>_<Endpoint>_time_sum`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] refactor(logging): Clean up `statsd` logging [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on PR #25847:
URL: https://github.com/apache/superset/pull/25847#issuecomment-2702152399
This PR hasn't seen any activity/events in roughly seven months. I'll reluctantly close it, but would love to see it reopened/revisited if anyone here has interest or bandwidth.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] refactor(logging): Clean up `statsd` logging [superset]
Posted by "sebastianliebscher (via GitHub)" <gi...@apache.org>.
sebastianliebscher commented on code in PR #25847:
URL: https://github.com/apache/superset/pull/25847#discussion_r1380574022
##########
superset/async_events/api.py:
##########
@@ -33,6 +34,7 @@ class AsyncEventsRestApi(BaseSupersetApi):
allow_browser_login = True
@expose("/", methods=("GET",))
+ @statsd_metrics
@event_logger.log_this
@protect()
Review Comment:
Most API endpoints already use `@statsd_metrics`. When this decorator is added to replace implicit `log_to_statsd=True`, the statsd metric
- `superset_<Endpoint>`
will be replaced by
- `superset_<Class>_<Endpoint>_success|warning|error`
- `superset_<Class>_<Endpoint>_time`
- `superset_<Class>_<Endpoint>_time_count`
- `superset_<Class>_<Endpoint>_time_sum`
So it is kind of a breaking change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] refactor(logging): Clean up `statsd` logging [superset]
Posted by "sebastianliebscher (via GitHub)" <gi...@apache.org>.
sebastianliebscher commented on code in PR #25847:
URL: https://github.com/apache/superset/pull/25847#discussion_r1380574022
##########
superset/async_events/api.py:
##########
@@ -33,6 +34,7 @@ class AsyncEventsRestApi(BaseSupersetApi):
allow_browser_login = True
@expose("/", methods=("GET",))
+ @statsd_metrics
@event_logger.log_this
@protect()
Review Comment:
Most API endpoints already use `@statsd_metrics`. In a case where this decorator is added to replace implicit `log_to_statsd=True`, the statsd metric
- `superset_<Endpoint>`
will be replaced by
- `superset_<Class>_<Endpoint>_success|warning|error`
- `superset_<Class>_<Endpoint>_time`
- `superset_<Class>_<Endpoint>_time_count`
- `superset_<Class>_<Endpoint>_time_sum`
So it is kind of a breaking change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] refactor(logging): Clean up `statsd` logging [superset]
Posted by "sebastianliebscher (via GitHub)" <gi...@apache.org>.
sebastianliebscher commented on code in PR #25847:
URL: https://github.com/apache/superset/pull/25847#discussion_r1380574022
##########
superset/async_events/api.py:
##########
@@ -33,6 +34,7 @@ class AsyncEventsRestApi(BaseSupersetApi):
allow_browser_login = True
@expose("/", methods=("GET",))
+ @statsd_metrics
@event_logger.log_this
@protect()
Review Comment:
Most API endpoints already use `@statsd_metrics`. In a case where this decorator is added to replace implicit `log_to_statsd=True`, the statsd metric
- `superset_<Endpoint>`
will be replaced by
- `superset_<Class>_<Endpoint>_success|warning|error`
- `superset_<Class>_<Endpoint>_time`
- `superset_<Class>_<Endpoint>_time_count`
- `superset_<Class>_<Endpoint>_time_sum`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] refactor(logging): Clean up `statsd` logging [superset]
Posted by "michael-s-molina (via GitHub)" <gi...@apache.org>.
michael-s-molina commented on PR #25847:
URL: https://github.com/apache/superset/pull/25847#issuecomment-1792895159
> One could double log the metric (which does pollute the metric space) and add deprecation logic in 3.1 (slated for late Q4 2023) and then remove the double logging in 4.0.
@john-bodley This seems like too much for something that is not urgent. I think it would be better to just wait for the breaking window.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] refactor(logging): Clean up `statsd` logging [superset]
Posted by "john-bodley (via GitHub)" <gi...@apache.org>.
john-bodley commented on PR #25847:
URL: https://github.com/apache/superset/pull/25847#issuecomment-1792881661
Thanks @sebastianliebscher for the refactor/cleanup. Given (as you mentioned) this is a breaking change, i.e., the logged statsD metric name will change it seems like we would have to wait until the breaking window opens for 4.0 (circa Q1 2024) before merging.
In the interim I was wondering whether there is a workaround? One could double log the metric (which does pollute the metric space) and add deprecation logic in 3.1 (slated for late Q4 2023) and then remove the double logging in 4.0.
cc @michael-s-molina
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] refactor(logging): Clean up `statsd` logging [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas closed pull request #25847: refactor(logging): Clean up `statsd` logging
URL: https://github.com/apache/superset/pull/25847
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] refactor(logging): Clean up `statsd` logging [superset]
Posted by "sebastianliebscher (via GitHub)" <gi...@apache.org>.
sebastianliebscher commented on code in PR #25847:
URL: https://github.com/apache/superset/pull/25847#discussion_r1380574022
##########
superset/async_events/api.py:
##########
@@ -33,6 +34,7 @@ class AsyncEventsRestApi(BaseSupersetApi):
allow_browser_login = True
@expose("/", methods=("GET",))
+ @statsd_metrics
@event_logger.log_this
@protect()
Review Comment:
Most API endpoints already use `@statsd_metrics`. When this decorator is added to replace implicit `log_to_statsd=True` like in this example, the statsd metric
- `superset_<Endpoint>`
will be replaced by
- `superset_<Class>_<Endpoint>_success|warning|error`
- `superset_<Class>_<Endpoint>_time`
- `superset_<Class>_<Endpoint>_time_count`
- `superset_<Class>_<Endpoint>_time_sum`
So it is kind of a breaking change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] refactor(logging): Clean up `statsd` logging [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on PR #25847:
URL: https://github.com/apache/superset/pull/25847#issuecomment-2311087915
Putting this one on our 5.0 project board as well. This one looks like it needs a big ol' rebase, though, at some point before October-ish.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] refactor(logging): Clean up `statsd` logging [superset]
Posted by "rusackas (via GitHub)" <gi...@apache.org>.
rusackas commented on PR #25847:
URL: https://github.com/apache/superset/pull/25847#issuecomment-2311088598
I'll just convert it to Draft for now, but please feel free to mark it as "Ready for Review" when it's rebased.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org