[jira] [Commented] (IMPALA-14940) Calcite planner: Handle Broadcast and Shuffle hints

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18126524#comment-18126524 ] 

ASF subversion and git services commented on IMPALA-14940:
----------------------------------------------------------

Commit d13a226823d57fe7f9b64cde5fdb8fd65967cd66 in impala's branch refs/heads/master from Steve Carlin
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=d13a22682 ]

IMPALA-14940: Calcite planner: handle broadcast and shuffle hints.

These hints have been added to the parser, rules, and physical node
converter in a similar way that the straight join was processed.

Other added changes to handle all hints include:

- The straight join was only being handled at the beginning of the query,
e.g. "select straight_join ...".  The broadcast and shuffle hints are
placed near the JOIN keyword. Calcite had this mechanism in place, but
it needed to be added and placed in the correct position for the validator
to handle it.

- The Calcite hints are handled with "[]" or "/*+ */", and this code was
also added.

- Some joins were not parsing correctly that are now fixed. Impala allows
joins that do not have the ON clause or even a comma separator. For instance,
this SQL statement now passes in Impala that did not pass in Calcite:
    "select min(t1.`1` + t2.`1`) from t t1 join t t2;"
Some logic was added to the parser to handle this. The "joinType" is treated
as a "COMMA" join type since any join condition will exist in the WHERE clause
and not attached to the SqlJoin directly.

- There is an issue that the hint gets applied too broadly when multiple joins
are in a "from" clause. Impala syntax allows a hint to be specified on a
specific join between two tables. IMPALA-15251 has been filed to address this
at a later date.

- One note on implementation: In order to honor the hint, the join ordering
is disabled in this iteration. The idea here is that the hint is specific
to the joining of two specific tables, and join ordering could make the
hint irrelevant.

Testing: Added one direct test to make sure cross join syntax works.

While no direct tests are added for the hints, this is one step among many that
will ensure we can do a Jenkins run without the original planner as the
fallback planner. Any test that tests shuffle and broadcast will succeed now.
Also, the above query without the ON clause is in the test framework and this
query, among others, will also no longer fallback to the original planner.

Change-Id: Ie5a932cc09d3d370aa627228d0e226a92fa03168
Reviewed-on: http://gerrit.cloudera.org:8080/24257
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Calcite planner: Handle Broadcast and Shuffle hints
> ---------------------------------------------------
>
>                 Key: IMPALA-14940
>                 URL: https://issues.apache.org/jira/browse/IMPALA-14940
>             Project: IMPALA
>          Issue Type: Sub-task
>            Reporter: Steve Carlin
>            Assignee: Steve Carlin
>            Priority: Major
>             Fix For: Impala 5.0.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org