[GitHub] AlexanderSaydakov commented on issue #6743: IncrementalIndex generally overestimates theta sketch size

[GitBox <gi...@apache.org>]

AlexanderSaydakov commented on issue #6743: IncrementalIndex generally overestimates theta sketch size
URL: https://github.com/apache/incubator-druid/issues/6743#issuecomment-448028867
 
 
   I don't see any way around this given the existing protocol. Druid asks the maximum size. And there is no distinction between the size in memory (slot in BufferAggregator) and serialized size in the segment. The 250KB is the absolute maximum size in memory, so that sketches always fit into those BufferAggregator slots. The method to get this size is getMaxIntermediateSize() in AggregatorFactory (note "intermediate" in the name).
   A serialized result should be about 130KB max. If the intention is to have the serialized size in IncrementalIndex, then, perhaps, another method in AggregatorFactory can be introduced (say, getMaxSerializedSize() or something like that)

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org