You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/03/27 23:02:00 UTC
[jira] [Commented] (IMPALA-11845) Select STAR with table ref is not resolved correctly on column/row masked views
[ https://issues.apache.org/jira/browse/IMPALA-11845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17705719#comment-17705719 ]
ASF subversion and git services commented on IMPALA-11845:
----------------------------------------------------------
Commit faae4a513c504b28c83b0417b368f2eff8f4c91c in impala's branch refs/heads/branch-4.1.2 from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=faae4a513 ]
IMPALA-11845: (Addendum) Don't specify db name in the new struct tests
Some new tests are added for STAR expansion on struct types when the
table is masked by Ranger masking policies. They are tested on both
Parquet and ORC tables. However, some tests explicitly use
'functional_parquet' as the db name, which lose the coverage on ORC
tables. This patch removes the explicit db names.
Change-Id: I8efea5cc2e10d8ae50ee6c1201e325932cb27fbf
Reviewed-on: http://gerrit.cloudera.org:8080/19470
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Select STAR with table ref is not resolved correctly on column/row masked views
> -------------------------------------------------------------------------------
>
> Key: IMPALA-11845
> URL: https://issues.apache.org/jira/browse/IMPALA-11845
> Project: IMPALA
> Issue Type: Bug
> Components: Security
> Affects Versions: Impala 4.0.0, Impala 3.4.0, Impala 3.4.1, Impala 4.1.0, Impala 4.2.0, Impala 4.1.1
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
> Fix For: Impala 4.3.0
>
>
> When a catalog view is applied with Ranger column-masking or row-filtering policies, select STAR on it doesn't work if the table ref is used, e.g. this doesn't work:
> {code:sql}
> select v.* from masked_view v;
> ERROR: IllegalStateException: null
> {code}
> Note that selecting STAR directly or selecting any columns still work:
> {code:sql}
> select * from masked_view;
> select c1, c2 from masked_view;
> {code}
> The stacktrace:
> {noformat}
> I0117 10:50:36.894714 21116 jni-util.cc:288] f940f87cdea03526:3c48d0f300000000] java.lang.IllegalStateException
> at com.google.common.base.Preconditions.checkState(Preconditions.java:486)
> at org.apache.impala.analysis.Analyzer.resolvePathWithMasking(Analyzer.java:1216)
> at org.apache.impala.analysis.Analyzer.resolvePathWithMasking(Analyzer.java:1177)
> at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.analyzeStarPath(SelectStmt.java:785)
> at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.collectStarExpandedPaths(SelectStmt.java:879)
> at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.analyze(SelectStmt.java:333)
> at org.apache.impala.analysis.SelectStmt$SelectAnalyzer.access$100(SelectStmt.java:280)
> at org.apache.impala.analysis.SelectStmt.analyze(SelectStmt.java:272)
> at org.apache.impala.analysis.AnalysisContext.reAnalyze(AnalysisContext.java:622)
> at org.apache.impala.analysis.AnalysisContext.analyze(AnalysisContext.java:553)
> at org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:468)
> at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:2059)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1967)
> at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1789)
> at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:164)
> {noformat}
> Related code:
> {code:java}
> 1180 public Path resolvePathWithMasking(List<String> rawPath, PathType pathType,
> 1181 TimeTravelSpec timeTravelSpec) throws AnalysisException, TableLoadingException {
> 1182 Path resolvedPath = resolvePath(rawPath, pathType, timeTravelSpec);
> 1183 // Skip normal resolution cases that don't relate to nested types.
> 1184 if (pathType == PathType.TABLE_REF) {
> 1185 if (resolvedPath.destTable() != null || !resolvedPath.isRootedAtTuple()) {
> 1186 return resolvedPath;
> 1187 }
> 1188 } else if (pathType == PathType.SLOT_REF) {
> 1189 if (!resolvedPath.getMatchedTypes().get(0).isStructType()) {
> 1190 return resolvedPath;
> 1191 }
> 1192 } else if (pathType == PathType.STAR) {
> 1193 if (!resolvedPath.destType().isStructType() || !resolvedPath.isRootedAtTuple()) {
> 1194 return resolvedPath;
> 1195 }
> 1196 }
> 1197 // In this case, resolvedPath is resolved on a nested column. Check if it's resolved
> 1198 // on a table masking view. The root TableRef(table/view) could be at a parent query
> 1199 // block (correlated case, e.g. "t.int_array" in query
> 1200 // "SELECT ... FROM tbl t, (SELECT * FROM t.int_array) a" roots at "tbl t" which is
> 1201 // in the parent block), so we should find the parent block first then we can find
> 1202 // the root TableRef.
> 1203 TupleId rootTupleId = resolvedPath.getRootDesc().getId();
> 1204 Analyzer parentAnalyzer = findAnalyzer(rootTupleId);
> 1205 TableRef rootTblRef = parentAnalyzer.getTableRef(rootTupleId);
> 1206 Preconditions.checkNotNull(rootTblRef);
> 1207 if (!rootTblRef.isTableMaskingView()) return resolvedPath;
> 1208 // resolvedPath is resolved on a nested column of a table masking view. The view
> 1209 // won't produce results of nested columns. It just exposes the nested columns of the
> 1210 // underlying BaseTableRef in the fields of its output type. (See more in
> 1211 // InlineViewRef#createTupleDescriptor()). We need to resolve 'rawPath' inside the
> 1212 // view as if the underlying table is not masked. So the resolved path can point to
> 1213 // the real table and be used to create materialized slot in the TupleDescriptor of
> 1214 // the real table.
> 1215 InlineViewRef tableMaskingView = (InlineViewRef) rootTblRef;
> 1216 Preconditions.checkState(
> 1217 tableMaskingView.getUnMaskedTableRef() instanceof BaseTableRef);
> 1218 // Resolve rawPath inside the table masking view to point to the real table.
> 1219 Path maskedPath = tableMaskingView.inlineViewAnalyzer_.resolvePath(
> 1220 rawPath, pathType, timeTravelSpec);
> 1221 maskedPath.setPathBeforeMasking(resolvedPath);
> 1222 return maskedPath;
> 1223 }
> {code}
> https://github.com/apache/impala/blob/b0009db40b7a532694987a0f4280b322d72f07b7/fe/src/main/java/org/apache/impala/analysis/Analyzer.java#L1217
> The 'pathType' is STAR and the 'rawPath' is the view ref. It's not a nested struct column so should be handled in the if-branches at the begining.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org