[jira] [Commented] (CALCITE-1787) thetaSketch Support for Druid Adapter

["Julian Hyde (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CALCITE-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16008330#comment-16008330 ] 

Julian Hyde commented on CALCITE-1787:
--------------------------------------

Does it need to be a new type? Can it not just be a new expression (of type varbinary or whatever)? There's some precedent for this. We have an aggregate function called HISTOGRAM_AGG which represents a partially evaluated aggregate containing a sorted set. From this result you can derive other aggregates: min, max, median. Partial aggregates are common in databases so I'd rather have a simpler approach than adding a new type for each one.

> thetaSketch Support for Druid Adapter
> -------------------------------------
>
>                 Key: CALCITE-1787
>                 URL: https://issues.apache.org/jira/browse/CALCITE-1787
>             Project: Calcite
>          Issue Type: New Feature
>          Components: druid
>    Affects Versions: 1.12.0
>            Reporter: Zain Humayun
>            Assignee: Julian Hyde
>            Priority: Minor
>             Fix For: 1.12.0
>
>
> Currently, the Druid adapter does not support the [thetaSketch|http://druid.io/docs/latest/development/extensions-core/datasketches-aggregators.html] aggregate type, which is used to measure the cardinality of a column quickly. Many Druid instances support theta sketches, so I think it would be a nice feature to have.
> I've been looking at the Druid adapter, and propose we add a new DruidType called {{thetaSketch}} and then add logic in the {{getJsonAggregation}} method in class {{DruidQuery}} to generate the {{thetaSketch}} aggregate. This will require accessing information about the columns (what data type they are) so that the thetaSketch aggregate is only produced if the column's type is {{thetaSketch}}. 
> Also, I've noticed that a {{hyperUnique}} DruidType is currently defined, but a {{hyperUnique}} aggregate is never produced. Since both are approximate aggregators, I could also couple in the logic for {{hyperUnique}}.
> I'd love to hear your thoughts on my approach, and any suggestions you have for this feature.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)