[GitHub] [incubator-pinot] mayankshriv commented on pull request #6225: Perf optimization for SQL GROUP BY ORDER BY

[GitBox <gi...@apache.org>]

mayankshriv commented on pull request #6225:
URL: https://github.com/apache/incubator-pinot/pull/6225#issuecomment-721375606


   > If there is no resize during the upsert, how do you prevent server from running out of memory?
   > One big difference between PQL and SQL is that for PQL we limit the groups in the map (trimming during upsert is desired, but not possible for PQL semantic). For SQL we want to trim the result during upsert to have the accurate result and also prevent OOM. We can increase the trimming threshold to reduce the number of trimmings.
   
   +1, eliminating resizing during upsert is dangerous and can cause OOM. PQL has a config to bypass the truncation that certain use cases are using, perhaps the same can be used for reducing num resize during upserts (defaulting to a tuned value that does not require a general case to explicitly set a value).


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org