[GitHub] [incubator-druid] leerho commented on issue #6743: IncrementalIndex generally overestimates theta sketch size

[GitBox <gi...@apache.org>]

leerho commented on issue #6743: IncrementalIndex generally overestimates theta sketch size
URL: https://github.com/apache/incubator-druid/issues/6743#issuecomment-485046057
 
 
   @gianm 
   This topic has been inactive for a while, but the issue still exists.  As we have been integrating our C++ versions of our sketches into PostgreSQL we have learned about how they solve this problem.   Custom UDFs perform memory allocations and destruction using the Postgres supplied "palloc" and "pfree" functions. This way, Postgres can track and manage the allocation and free processes in the context of queries or transactions.  This way, when the contexts are closed, all the allocated memory can be guaranteed to be freed.  
   
   Inside a specific aggregation function every allocation for space (for an array for example) is performed via these palloc and pfree functions.  The aggregation function does not have to live in a contiguous chunk of memory, which make the design of the aggregation function much easier.  
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org