You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/02/20 03:04:00 UTC
[jira] [Commented] (IMPALA-10467) Implement ds_theta_union()
function
[ https://issues.apache.org/jira/browse/IMPALA-10467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17287468#comment-17287468 ]
ASF subversion and git services commented on IMPALA-10467:
----------------------------------------------------------
Commit 7f60990028cb7fba71a29b9f3f518f5cc8d43a0b in impala's branch refs/heads/master from Fucun Chu
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=7f60990 ]
IMPALA-10467: Implement ds_theta_union() function
This function receives a set of serialized Apache DataSketches Theta
sketches produced by ds_theta_sketch() and merges them into a single
sketch.
An example usage is to create a sketch for each partition of a table,
write these sketches to a separate table and based on which partition
the user is interested of the relevant sketches can be union-ed
together to get an estimate. E.g.:
SELECT
ds_theta_estimate(ds_theta_union(sketch_col))
FROM sketch_tbl
WHERE partition_col=1 OR partition_col=5;
Testing:
- Apart from the automated tests I added to this patch I also
tested ds_theta_union() on a bigger dataset to check that
serialization, deserialization and merging steps work well. I
took TPCH25.linelitem, created a number of sketches with grouping
by l_shipdate and called ds_theta_union() on those sketches
Change-Id: I91baf58c76eb43748acd5245047edac8c66761b2
Reviewed-on: http://gerrit.cloudera.org:8080/17048
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Implement ds_theta_union() function
> -----------------------------------
>
> Key: IMPALA-10467
> URL: https://issues.apache.org/jira/browse/IMPALA-10467
> Project: IMPALA
> Issue Type: New Feature
> Reporter: Fucun Chu
> Assignee: Fucun Chu
> Priority: Major
>
> Similarly to https://issues.apache.org/jira/browse/IMPALA-9633 DataSketches Theta also has union functionality.
> ds_theta_union() expects a dataset of STRINGs (e.g. column of a table) that represents serialized Theta sketches and returns a single STRING that is the received sketches unioned and serialized.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org