You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/01/15 23:36:00 UTC
[jira] [Commented] (IMPALA-10387) Implement missing overloads of
mask functions used in Ranger default masking policies
[ https://issues.apache.org/jira/browse/IMPALA-10387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17266411#comment-17266411 ]
ASF subversion and git services commented on IMPALA-10387:
----------------------------------------------------------
Commit 9bb7157bf014282c95ab3e233b80d77e00c95b52 in impala's branch refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=9bb7157 ]
IMPALA-10387: Add missing overloads of mask functions used in Ranger default masking policies
The mask functions in Hive are implemented through GenericUDFs which can
accept an infinite number of function signatures. Impala currently don't
support GenericUDFs. So we provide builtin mask functions with limited
overloads.
This patch adds some missing overloads that could be used by Ranger
default masking policies, e.g. MASK_HASH, MASK_SHOW_LAST_4,
MASK_DATE_SHOW_YEAR, etc.
Tests:
- Add test coverage on all default masking policies applied on all
supported types.
Change-Id: Icf3e70fd7aa9f3b6d6b508b776696e61ec1fcc2e
Reviewed-on: http://gerrit.cloudera.org:8080/16930
Reviewed-by: Quanlong Huang <hu...@gmail.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Implement missing overloads of mask functions used in Ranger default masking policies
> -------------------------------------------------------------------------------------
>
> Key: IMPALA-10387
> URL: https://issues.apache.org/jira/browse/IMPALA-10387
> Project: IMPALA
> Issue Type: New Feature
> Components: Frontend
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
>
> Some meaningless overloads of the mask functions are not implemented in Impala since Hive always returns NULL on them. It's expected to use the mask_null function (Nullify policy) directly.
> However, it's common for users to configure Redact policy (mask) on all columns. If there are float/double/decimal columns, they will hit errors like this in Impala:
> {code:java}
> I1208 22:38:03.880369 8354 jni-util.cc:288] b8461c997916681b:83957ca300000000] org.apache.impala.common.AnalysisException: No matching function with signature: mask(FLOAT).
> at org.apache.impala.analysis.FunctionCallExpr.analyzeImpl(FunctionCallExpr.java:555)
> at org.apache.impala.analysis.Expr.analyze(Expr.java:438)
> at org.apache.impala.analysis.Expr.analyze(Expr.java:432)
> at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.analyzeSelectClause(SelectStmt.java:276)
> at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.analyze(SelectStmt.java:215)
> at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.access$100(SelectStmt.java:199)
> at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:192)
> at org.apache.impala.analysis.InlineViewRef.analyze(InlineViewRef.java:218)
> at org.apache.impala.analysis.FromClause.analyze(FromClause.java:78)
> at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.analyze(SelectStmt.java:213)
> at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.access$100(SelectStmt.java:199)
> at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:192)
> at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:473)
> at org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:437)
> at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1530)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1497)
> at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1467)
> at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:154) {code}
> We'd better add these overloads to avoid hitting such errors in simple setup. These patterns are used by Ranger default masking policies:
> * mask(col)
> * mask_show_last_n(col, 4, 'x', 'x', 'x', -1, '1')
> * mask_show_first_n(col, 4, 'x', 'x', 'x', -1, '1')
> * mask_hash(col)
> * mask(col, 'x', 'x', 'x', -1, '1', 1, 0, -1)
> We should have overloads for {{col}} to be any types.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org