[jira] [Comment Edited] (CALCITE-3221) Add a sort-merge union algorithm

["Vladimir Sitnikov (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CALCITE-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272805#comment-17272805 ] 

Vladimir Sitnikov edited comment on CALCITE-3221 at 1/27/21, 12:02 PM:
-----------------------------------------------------------------------

{quote} But this does not collide with the current ticket's purpose: a new MergeUnion operator that keeps the collation from its (sorted) inputs{quote}
Please update the issue description so the intention matches the implementation. Currently, the half of the description mentions Enumerable#union which is misleading


was (Author: vladimirsitnikov):
{quote} But this does not collide with the current ticket's purpose: a new MergeUnion operator that keeps the collation from its (sorted) inputs{quote}
Please update the issue description so the intention matches the implementation.

> Add a sort-merge union algorithm
> --------------------------------
>
>                 Key: CALCITE-3221
>                 URL: https://issues.apache.org/jira/browse/CALCITE-3221
>             Project: Calcite
>          Issue Type: Improvement
>          Components: core
>    Affects Versions: 1.19.0
>            Reporter: Stamatis Zampetakis
>            Assignee: Ruben Q L
>            Priority: Minor
>              Labels: pull-request-available
>         Attachments: screenshot-1.png
>
>          Time Spent: 11h 20m
>  Remaining Estimate: 0h
>
> Currently, the union operation offered by Calcite is based on a {{HashSet}} (see [EnumerableDefaults.union|https://github.com/apache/calcite/blob/d98856bf1a5f5c151d004b769e14bdd368a67234/linq4j/src/main/java/org/apache/calcite/linq4j/EnumerableDefaults.java#L2747]) and necessitates reading in memory all rows before returning a single result.   
> Apart from increased memory consumption the operator is blocking and also destroys the order of its inputs.  
> The goal of this issue is to add a new union algorithm (EnumerableMergeUnion ?) exploiting the fact that the inputs are sorted which consumes less memory and retains the order of its inputs.   
> Most likely the implementation of the merge join can be useful.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)