[GitHub] [incubator-pinot] mayankshriv opened a new issue #6179: Framework for performance testing

[GitBox <gi...@apache.org>]

mayankshriv opened a new issue #6179:
URL: https://github.com/apache/incubator-pinot/issues/6179


   As the community grows, the amount of contributions also grow. However, not everyone runs Pinot for similar workload, so it is possible that some changes that improve performance for a certain kind of workload may end up degrading performance for other kinds (we have noticed this in the past).
   
   It would be good to build a performance testing framework that helps us validate commits against any regressions in performance. A good starting point would be:
   
   - Data and query generator: This utility would be able to generate different types of workloads with configurable data and query workloads (e.g. cardinality, data type, query type, etc).
   
   - Perf driver: This would setup the table in a distributed fashion, bootstrap the table with data, and run queries at specified qps.
   
   - Overall driver: This would be the overall driver responsible for setting up everything and running end-to-end, along with generating statistical reports comparing metrics (throughput/latency) between commits (or commit vs master).
   
   - We would also need a list of pre-defined workloads, as well as a place to run these tests (preferably on the cloud).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org