[jira] [Created] (HIVE-29449) Avoid ANY distribution in HiveSortExchange RelNode

["Stamatis Zampetakis (Jira)" <ji...@apache.org>]

Stamatis Zampetakis created HIVE-29449:
------------------------------------------

             Summary: Avoid ANY distribution in HiveSortExchange RelNode
                 Key: HIVE-29449
                 URL: https://issues.apache.org/jira/browse/HIVE-29449
             Project: Hive
          Issue Type: Task
            Reporter: Stamatis Zampetakis
            Assignee: Stamatis Zampetakis


Currently, we are creating a {{HiveSortExchange}} operator with [ANY|https://github.com/apache/calcite/blob/869fe15f36fa5579f2a1cf289958fa763af36a9b/core/src/main/java/org/apache/calcite/rel/RelDistribution.java#L101] distribution for queries with a SORT BY clause (and no DISTRIBUTE BY).

+Example+
{code:sql}
explain cbo select * from t1 sort by a
{code}

{noformat}
CBO PLAN:
HiveSortExchange(distribution=[any], collation=[[0]])
  HiveProject(t1.a=[$0], t1.b=[$1])
    HiveTableScan(table=[[default, t1]], table:alias=[t1])
{noformat}

However, ANY is not a valid distribution as per Javadoc and there are even some assertions in place to prevent its usage when [creating an Exchange|https://github.com/apache/calcite/blob/869fe15f36fa5579f2a1cf289958fa763af36a9b/core/src/main/java/org/apache/calcite/rel/core/Exchange.java#L67] operator. The assertion has been circumvented in HIVE-28572 by using the HiveRelDistribution wrapper but by doing so we have a plan that is in a semi-invalid state. 

An alternative way for representing SORT BY queries (with no DISTRIBUTE BY) would be to use a HASH distribution with empty keys. 

{noformat}
CBO PLAN:
HiveSortExchange(distribution=[hash], collation=[[0]])
  HiveProject(t1.a=[$0], t1.b=[$1])
    HiveTableScan(table=[[default, t1]], table:alias=[t1])
{noformat}

The plan would still not reflect what really happens at the physical layer but it will contain a valid distribution type.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)