[GitHub] gianm edited a comment on issue #6869: [Proposal] Deprecating "approximate histogram" in favor of new sketches

[GitBox <gi...@apache.org>]

gianm edited a comment on issue #6869: [Proposal] Deprecating "approximate histogram" in favor of new sketches
URL: https://github.com/apache/incubator-druid/issues/6869#issuecomment-454665859
 
 
   This proposal looks good to me. approxHistogram had a good run.
   
   > Provide advice on how to choose between the quantiles sketch or the moments sketch
   
   Do you have thoughts on what kind of advice we'd include here?
   
   > Is it possible/valid for the newer sketch aggs to be able to operate on an old "approximate histogram"? If so, this would make migration easier for users.
   
   My guess is no, since the algorithms are all different.
   
   > Change the APPROX_QUANTILE Druid SQL function to use whatever aggregator type is stored in a segment, and use a default option when used on a numeric column
   
   This will require a bit of an enhancement to the SQL metadata layer. Check out RowSignature, in particular this part:
   
   ```java
             case COMPLEX:
               // Loses information about exactly what kind of complex column this is.
               type = Calcites.createSqlTypeWithNullability(typeFactory, SqlTypeName.OTHER, true);
               break;
   ```
   
   For `APPROX_QUANTILE` to be able to behave differently based on the type of sketch stored in the underlying column, we'll need to figure out some way to encode that information in the type. Or some way to make it accessible to the `SqlAggregator` conversion out-of-band. Encoding it in the type somehow is preferable, though, if possible, because avoiding the need for out-of-band info is always nice. It would also make it easier to expose it through the INFORMATION_SCHEMA.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org