[jira] [Commented] (IMPALA-10687) Implement ds_cpc_union() function

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-10687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344718#comment-17344718 ] 

ASF subversion and git services commented on IMPALA-10687:
----------------------------------------------------------

Commit b1326f7eff62dd521989132d650fa8b53057085b in impala's branch refs/heads/master from Fucun Chu
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b1326f7 ]

IMPALA-10687: Implement ds_cpc_union() function

This function receives a set of serialized Apache DataSketches CPC
sketches produced by ds_cpc_sketch() and merges them into a single
sketch.

An example usage is to create a sketch for each partition of a table,
write these sketches to a separate table and based on which partition
the user is interested of the relevant sketches can be union-ed
together to get an estimate. E.g.:
  SELECT
      ds_cpc_estimate(ds_cpc_union(sketch_col))
  FROM sketch_tbl
  WHERE partition_col=1 OR partition_col=5;

Testing:
  - Apart from the automated tests I added to this patch I also
    tested ds_cpc_union() on a bigger dataset to check that
    serialization, deserialization and merging steps work well. I
    took TPCH25.linelitem, created a number of sketches with grouping
    by l_shipdate and called ds_cpc_union() on those sketches

Change-Id: Ib94b45ae79efcc11adc077dd9df9b9868ae82cb6
Reviewed-on: http://gerrit.cloudera.org:8080/17372
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Implement ds_cpc_union() function
> ---------------------------------
>
>                 Key: IMPALA-10687
>                 URL: https://issues.apache.org/jira/browse/IMPALA-10687
>             Project: IMPALA
>          Issue Type: New Feature
>            Reporter: Fucun Chu
>            Assignee: Fucun Chu
>            Priority: Major
>
> Similarly to https://issues.apache.org/jira/browse/IMPALA-9633 DataSketches CPC also has union functionality.
> ds_cpc_union() expects a dataset of STRINGs (e.g. column of a table) that represents serialized CPC sketches and returns a single STRING that is the received sketches unioned and serialized.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org