You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2020/12/22 22:48:07 UTC
[GitHub] [incubator-pinot] chenboat opened a new issue #6378: Create a dashboard for most expensive queries per table
chenboat opened a new issue #6378:
URL: https://github.com/apache/incubator-pinot/issues/6378
In many production settings, a Pinot cluster can handle thousand of queries per second. Expensive queries can greatly impact the overall system performance. Right now, users of Pinot have to rely on external mechanisms of pinpoint these costly queries. Namely, one has to export the broker query logs with query stats and then rely on tools like Kibana to filter and find those queries.
We propose to add query monitoring to Pinot controller console. The query monitoring console should show the most expensive (according to query stats like number of docs/entries scanned) queries per table in the past N days. One can use the dashboard to:
1. Find out the most expensive queries per table.
2. Help to find missing index config for the corresponding table.
3. (Nice to have) allow users to create the indexes to improve query performance.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] mcvsubbu commented on issue #6378: Create a dashboard for most expensive queries per table
Posted by GitBox <gi...@apache.org>.
mcvsubbu commented on issue #6378:
URL: https://github.com/apache/incubator-pinot/issues/6378#issuecomment-754138688
> why not write the query response back to Pinot. Have a logger that logs the query response to kafka topic and have that ingested into Pinot. We can build multiple visualizations from that
We do that in Linkedin. See RequestStatistics. Feel free to add fields to it, but please do not rename the existing members :-)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] kishoreg commented on issue #6378: Create a dashboard for most expensive queries per table
Posted by GitBox <gi...@apache.org>.
kishoreg commented on issue #6378:
URL: https://github.com/apache/incubator-pinot/issues/6378#issuecomment-752684947
why not write the query response back to Pinot. Have a logger that logs the query response to kafka topic and have that ingested into Pinot. We can build multiple visualizations from that
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org