You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@submarine.apache.org by GitBox <gi...@apache.org> on 2021/12/17 06:51:08 UTC
[GitHub] [submarine] Junyewu opened a new issue #838: [Submarine Spark Security] when run sql have expression will return ”UnresolvedAttribute.exprId“ error
Junyewu opened a new issue #838:
URL: https://github.com/apache/submarine/issues/838
env:
spark-3.1.1
ranger-2.1
submarine-security-0.6.0
sql code
```
spark-sql> create or replace temporary view user_city_temp as
select id,case when if(city_zip_active >0,city_zip_active,if(city_zip_check >0 ,city_zip_check,city_zip_phone)) in ('0100','0200','0300','0400') then if(city_zip_active >0,city_zip_active,if(city_zip_check >0 ,city_zip_check,city_zip_phone)) else 'other' end as city from user_city;
colum_name type
city_zip_phone int NULL
city_zip_check int NULL
city_zip_active int NULL
```
it will return a error
```
org.apache.spark.sql.catalyst.analysis.UnresolvedException: Invalid call to exprId on unresolved object, tree: 'city_zip_active
at org.apache.spark.sql.catalyst.analysis.UnresolvedAttribute.exprId(unresolved.scala:157)
at org.apache.spark.sql.catalyst.optimizer.SubmarineDataMaskingExtension$$anonfun$collectAllAliases$1$$anonfun$applyOrElse$2.$anonfun$applyOrElse$3(SubmarineDataMaskingExtension.scala:191)
at scala.collection.mutable.HashMap.getOrElseUpdate(HashMap.scala:86)
at org.apache.spark.sql.catalyst.optimizer.SubmarineDataMaskingExtension$$anonfun$collectAllAliases$1$$anonfun$applyOrElse$2.applyOrElse(SubmarineDataMaskingExtension.scala:191)
at org.apache.spark.sql.catalyst.optimizer.SubmarineDataMaskingExtension$$anonfun$collectAllAliases$1$$anonfun$applyOrElse$2.applyOrElse(SubmarineDataMaskingExtension.scala:189)
at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformUp$2(TreeNode.scala:352)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] harsha-tenneti commented on issue #838: [Submarine Spark Security] when run sql have expression will return ”UnresolvedAttribute.exprId“ error
Posted by GitBox <gi...@apache.org>.
harsha-tenneti commented on issue #838:
URL: https://github.com/apache/submarine/issues/838#issuecomment-1008136345
Do you have any masking policies on any of the columns in the table? If yes, what are the columns and their masking strategy?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] Junyewu commented on issue #838: [Submarine Spark Security] when run sql have expression will return ”UnresolvedAttribute.exprId“ error
Posted by GitBox <gi...@apache.org>.
Junyewu commented on issue #838:
URL: https://github.com/apache/submarine/issues/838#issuecomment-997512637
@yaooqinn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] Junyewu commented on issue #838: [Submarine Spark Security] when run sql have expression will return ”UnresolvedAttribute.exprId“ error
Posted by GitBox <gi...@apache.org>.
Junyewu commented on issue #838:
URL: https://github.com/apache/submarine/issues/838#issuecomment-996506540
I try to make `CreateViewCommand` is not doMasking，and it is run ok
org.apache.spark.sql.catalyst.optimizer.scala
```
override def apply(plan: LogicalPlan): LogicalPlan = plan match {
case c: Command => c match {
case c: CreateDataSourceTableAsSelectCommand => c.copy(query = doMasking(c.query))
case c: CreateHiveTableAsSelectCommand => c.copy(query = doMasking(c.query))
// case c: CreateViewCommand => c.copy(child = doMasking(c.child))
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] Junyewu commented on issue #838: [Submarine Spark Security] when run sql have expression will return ”UnresolvedAttribute.exprId“ error
Posted by GitBox <gi...@apache.org>.
Junyewu commented on issue #838:
URL: https://github.com/apache/submarine/issues/838#issuecomment-1009659931
yes，table have masking
mask column：city_zip_active
masking strategy：redact
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] xunliu closed issue #838: [Submarine Spark Security] when run sql have expression will return ”UnresolvedAttribute.exprId“ error
Posted by GitBox <gi...@apache.org>.
xunliu closed issue #838:
URL: https://github.com/apache/submarine/issues/838
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] xunliu commented on issue #838: [Submarine Spark Security] when run sql have expression will return ”UnresolvedAttribute.exprId“ error
Posted by GitBox <gi...@apache.org>.
xunliu commented on issue #838:
URL: https://github.com/apache/submarine/issues/838#issuecomment-1059931945
The [Submarine Spark Security] functionality has been moved to the apache/incubator-kyuubi standalone project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [submarine] Junyewu removed a comment on issue #838: [Submarine Spark Security] when run sql have expression will return ”UnresolvedAttribute.exprId“ error
Posted by GitBox <gi...@apache.org>.
Junyewu removed a comment on issue #838:
URL: https://github.com/apache/submarine/issues/838#issuecomment-997512637
@yaooqinn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@submarine.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org