[GitHub] leerho commented on issue #6743: IncrementalIndex generally overestimates theta sketch size

[GitBox <gi...@apache.org>]

leerho commented on issue #6743: IncrementalIndex generally overestimates theta sketch size
URL: https://github.com/apache/incubator-druid/issues/6743#issuecomment-451328299
 
 
   @gianm 
   
   > I'd like to take you up on working out a way to migrate users to the new sketch. It is somewhat delicate since the new one is in an extension and the old one is in core. And also because they don't have the same on-disk format, so we'd need to support both forever (we don't want to drop compatibility with on-disk data). We need to get the migration right to avoid user confusion.
   
   Understood and I agree that you will have to keep it around, but once we have an easy way for users to substitute in the DS HLL sketch, we should deprecate the old one with documentation on how to use the DS HLL sketch instead. The docs will have to include the warning that since the two sketches use different hash functions, using the DS-HLL sketch is starting from scratch.  If the user is relying oh historically stored sketch images, they are stuck with the old sketch, and its error problems.
   
   Just wondering, is there a way we could move the DS library to core?
   
   I am still on vacation this week, but will start looking at this next week :)
    

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org