[GitHub] gianm commented on issue #6559: Backpressure metric should not be emitted by default

[GitBox <gi...@apache.org>]

gianm commented on issue #6559: Backpressure metric should not be emitted by default
URL: https://github.com/apache/incubator-druid/issues/6559#issuecomment-438438378
 
 
   For 0.13.0's default config I am fine keeping the new metric, or removing it (either way I'm going to enable it at my site). But after 0.13.0 I think we need a better way of handling this. I don't think it's user-friendly to ask users, especially new users, to write private extensions or even long configuration files in order to get metrics emission to work. There are two questions packed in here.
   
   First, what metrics do we emit by default? It could be some "reasonable" level, it could be everything, or it could be nothing. @jon-wei and I have advocated for "reasonable". @leventov has said he would prefer "nothing", but in a concession to backwards compatibility, is ok with "the level that was in place when the QueryMetrics system was set up".
   
   Second, how do cluster operators customize metrics emitting? Today, they have to write a private extension. It sounds like everyone things configuration would be a good option, but it's not implemented yet.
   
   I would like to propose the following.
   
   - Adopt the "reasonable" standard for new metrics (if a new user comes in, with zero configuration, they get this "reasonable" set). The names of these metrics and metric-dimensions are part of Druid's public API. Other metrics and metric-dimensions, that are not in the default set, are _not_ in Druid's public API.
   - Continue to support QueryMetrics extensions as a way for advanced operators to have full control over how metrics emission works. It will work the same way as it works now. If a new kind of metric or metric-dimension is added, then QueryMetrics extension authors will still be 'notified' and have the opportunity to decide what to do about it.
   - Consider building a configuration-based approach that lets intermediate users do some customizations through config only.
   
   I feel that this approach will provide stability for a core set of most-useful metrics, while still allowing customizability, and allowing some mechanism to add "advanced" metrics that aren't part of the public API, if we want to do that for some reason.
   
   Btw, with regard to the suggestion below for how configuration works, I don't see how it adds value beyond making the name "query/node/backpressure" part of the public API of Druid. It eliminates that, but introduces a new thing "nodeBackPressureTime" that _is_ part of the public API (we can't change it without breaking old configs).
   
   ```
   druid.query.queryMetrics.default.metrics.nodeBackPressureTime.name=query/node/backpressure
   druid.query.queryMetrics.default.metrics.nodeBackPressureTime.unit=nanoseconds
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org