[jira] [Commented] (IMPALA-10484) Support using unrelated collection columns of a table containing row-filtering policies

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-10484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17304548#comment-17304548 ] 

ASF subversion and git services commented on IMPALA-10484:
----------------------------------------------------------

Commit 98de1c5436415c270901e4af76e0ec06deee1f32 in impala's branch refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=98de1c5 ]

IMPALA-9234: Support Ranger row filtering policies

Ranger row filtering policies provide customized expressions to filter
out rows for specific users when reading from a table. This patch adds
support for this feature. A new feature flag, enable_row_filtering, is
added to disable this experimental feature. It defaults to be true so
the feature is enabled by default. Enabling row-filtering requires
--enable_column_masking=true since it depends on the column masking
implementation.

Note that row filtering policies take effects prior to any column
masking policies, because column masking policies apply on result data.

Implementation:
The existing table masking view infrastructure can be extended to
support row filtering. Currently when analyzing a table with column
masking policies, we replace the TableRef with an InlineViewRef which
contains a SelectStmt wrapping the columns with masking expressions.
This patch adds the row filtering expressions to the WhereClause of the
SelectStmt.

Limitations:
 - Expressions using subqueries are not supported (IMPALA-10483).
 - Row filtering policies on nested tables will not be applied when
   nested collection columns are used directly in the FROM clause. This
   will leak data so we forbid such kinds of queries until IMPALA-10484
   is resolved.

Tests:
 - Add FE test for error message when disabling row filtering.
 - Add e2e test with row filtering policies.
 - Add e2e test with column masking and row filtering policies both take
   place.
 - Verified audits in a CDP cluster with Ranger and Solr set up.

Change-Id: I580517be241225ca15e45686381b78890178d7cc
Reviewed-on: http://gerrit.cloudera.org:8080/16976
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>


> Support using unrelated collection columns of a table containing row-filtering policies
> ---------------------------------------------------------------------------------------
>
>                 Key: IMPALA-10484
>                 URL: https://issues.apache.org/jira/browse/IMPALA-10484
>             Project: IMPALA
>          Issue Type: New Feature
>          Components: Security
>    Affects Versions: Impala 4.0
>            Reporter: Quanlong Huang
>            Priority: Major
>
> This is a follow-up for IMPALA-9234. Row-filtering policies are applied as the WHERE clause of the table masking view of the base table/view. E.g. if table "tblA" contains a row-filtering policy "id=0", the original query "{{select * from tblA join tblB on (id)}}" will be analyzed as
> {code:sql}
> select * from (
>   select col1, col2, ..., colN from tblA where id = 0
> ) v join tblB on (id)
> {code}
> Tables containing complex types are more complex in analyzing. Let's say "tblA" contains two columns:
> ||id int||int_array array<int>||
> |0|[1,2,3]|
> |1|[4,5]|
> The following queries gets all items of the array column:
> {code:sql}
> -- Good for applying table masking view on tblA
> select a.item from tblA t, t.int_array a;
> -- Hard to apply table masking view on tblA
> select item from tblA.int_array;
> {code}
> The second query uses unrelated collection column directly. So we can't apply the row-filtering policies on the base table. This will leak data, so IMPALA-9234 forbids them.
> We can support this by rewritting the second query to the first one. Luckily, we already have a rewritter containing this ability: {{AcidRewriter}} in {{StmtRewriter}}. With some refactor it can be reused. However, there is a bug (IMPALA-10482) needs to be fixed first.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org