[Impala-ASF-CR] IMPALA-9330: Support resolving unmasked nested columns in masked tables

["Quanlong Huang (Code Review)" <ge...@cloudera.org>]

Hello Anurag Mantripragada, Fang-Yu Rao, Vihang Karajgaonkar, Kurt Deschler, Csaba Ringhofer, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/15108

to look at the new patch set (#9).

Change subject: IMPALA-9330: Support resolving unmasked nested columns in masked tables
......................................................................

IMPALA-9330: Support resolving unmasked nested columns in masked tables

Column masking policies on primitive columns of a table which contains
nested types (though they won't be masked) will cause query failures.
To be specifit, if tableA(id int, int_array array<int>) has a masking
policy on column "id", all queries on "tableA" will fail, e.g.
  select id from tableA;
  select t.id, a.item from tableA t, t.int_array a;

Column masking is implemented by wrapping the underlying table/view with
a table masking view. However, as we don't support nested types in
SelectList, the table masking view can't expose nested columns of the
masked table, which causes collection refs not being resolved correctly.

This patch fixes the issue by 2 steps:
1) Expose nested columns of the underlying table in the output Type of
   the table masking view (see InlineViewRef#createTupleDescriptor()).
   So nested Paths in the original query block can be resolved.
2) For such kind of Paths, resolved them again inside the table masking
   view. So they can point to the underlying table as what they mean
   (see Analyzer#resolvePathWithMasking()). TupleDescriptor of such kind
   of table masking view won't be materialized since the view is simple
   enough that its query plan is just a ScanNode of the underlying
   table. The whole query plan can be stitched as if the table is not
   masked.
Note that one day when we support nested columns in SelectList, we may
don't need these 2 hacks.

This patch also adds some TRACE level loggings to improve debuggability.

Test changes in TestRanger.test_column_masking:
 - Add column masking policy on a table containing nested types.
 - Add queries on the masked tables. Some queries are borrowed from
   existing tests for nested types.

Tests:
 - Run CORE tests.

Change-Id: I1cc5565c64c1a4a56445b8edde59b1168f387791
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/BaseTableRef.java
M fe/src/main/java/org/apache/impala/analysis/CollectionTableRef.java
M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java
M fe/src/main/java/org/apache/impala/analysis/Path.java
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/analysis/SlotRef.java
M fe/src/main/java/org/apache/impala/analysis/TableRef.java
M fe/src/main/java/org/apache/impala/authorization/TableMask.java
M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java
M testdata/workloads/functional-query/queries/QueryTest/ranger_column_masking.test
M tests/authorization/test_ranger.py
12 files changed, 527 insertions(+), 18 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/08/15108/9
-- 
To view, visit http://gerrit.cloudera.org:8080/15108
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1cc5565c64c1a4a56445b8edde59b1168f387791
Gerrit-Change-Number: 15108
Gerrit-PatchSet: 9
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anurag Mantripragada <an...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Kurt Deschler <kd...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>