[GitHub] [incubator-pinot] sunithabeeram opened a new pull request #4356: Emit metrics to track performance against specific Service-Level-Objectives (SLO)

[GitBox <gi...@apache.org>]

sunithabeeram opened a new pull request #4356: Emit metrics to track performance against specific Service-Level-Objectives (SLO)
URL: https://github.com/apache/incubator-pinot/pull/4356
 
 
   This change builds on issue #4007. The main contribution of this PR is to emit metrics that can track performance of queries against a table for a given set of service-level objectives. We break the tracking into 5 categories: responses that are errors, responses that have partial-data, responses that don't meet latency objectives (latentResponses), responses that don't meet freshness objectives (staleResponses) - all others are considered "healthy", as in - they meet the SLO. The availability of the system can then be measured as: (healthyResponses)/(healthyResponses+partialResponses+errorResponses+latentResponses+staleResponses). This PR does not address the availability calculation as this can be computed outside of the system - and can be computed by assigning differnt weights for different categories. For e.g., the above equation assigns 0 healthyWeight for partialResponses - however, operators can choose to penalize partialResponses lesser and so on.
   
   The SLO are now part of table-config that the broker fetches in order to emit the metrics natively.
   
   Changes in this PR include:
   - Updates to TableConfig to expose SloConfig
   - Support a Lazy-loading Table-Config on the broker side to allow emitting the metrics.
   - Categorize different scenarios into the specified response type classifications.
   - Name change:  numConsumingSegmentsQueried -> numConsumingSegmentsProcessed
   
   Testing done:
   - Basic unit tests to cover newly added functionality
   - Manual integration tests to cover metric emission and validations.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org