[Impala-ASF-CR] IMPALA-8718: project out collection slots in analytic's sort tuple

["Quanlong Huang (Code Review)" <ge...@cloudera.org>]

Hello Tim Armstrong, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/14135

to look at the new patch set (#2).

Change subject: IMPALA-8718: project out collection slots in analytic's sort tuple
......................................................................

IMPALA-8718: project out collection slots in analytic's sort tuple

Subplan node is mainly used to extract collection values. It evaluates
its right plan tree (usually a nested loop join) for every row from its
left child (usually a scan producing tuples with collection values), and
returns those rows produced by the right child. Each row (TupleRow)
produced by the join node consists of several tuples from the join
operands. So the scan node tuple that contains collection values will be
part of the output of the join node, then become part of the output of
the subplan node.

When generating analytic plan, a TupleDescriptor for sort is created
based on the materialized slots of the input. If the input comes from a
subplan node, there are collection slots in it. These collection slots
will be picked out into the sort tuple, and occur in the smap of it.
Then the output smap of the analytic plan will contain the collection
slot consequently. This causes IllegalStateException if the analytic
plan is the nullable side of an outer join. The exception is thrown when
we are checking the necessary of adding a TupleIsNullPredicate for each
output slot.

We should project out the collection slots in creating the sort tuple of
analytic plan to avoid causing such an exception. Projecting out them is
safe since outputs of the analytic node must be in the select list of
the block with the analytic, and we don't allow collection types to be
returned from a select block, and also don't support any builtin or UDF
functions that take collection types as an argument.

Tests
 - Add Planner test in analytic-fns.test with VALIDATE_CARDINALITY
 enabled. Also fix some incorrect row-sizes of existing tests.
 - Add e2e test in nested-types-runtime.test to verify that collection
 slots are projected out.

Change-Id: I7edf74ff0f603dfd33ff546e61545bc724990655
---
M fe/src/main/java/org/apache/impala/analysis/TupleIsNullPredicate.java
M fe/src/main/java/org/apache/impala/planner/AnalyticPlanner.java
M fe/src/test/java/org/apache/impala/planner/PlannerTest.java
M testdata/workloads/functional-planner/queries/PlannerTest/analytic-fns.test
M testdata/workloads/functional-query/queries/QueryTest/nested-types-runtime.test
5 files changed, 137 insertions(+), 14 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/14135/2
-- 
To view, visit http://gerrit.cloudera.org:8080/14135
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7edf74ff0f603dfd33ff546e61545bc724990655
Gerrit-Change-Number: 14135
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>