[jira] [Commented] (HIVE-20628) Parsing error when using a complex map data type under dynamic column masking

["Alessandro Solimando (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-20628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563830#comment-17563830 ] 

Alessandro Solimando commented on HIVE-20628:
---------------------------------------------

Complex types are not supported by Hive, the current error could be improved and be more explicit about what is happening.

> Parsing error when using a complex map data type under dynamic column masking
> -----------------------------------------------------------------------------
>
>                 Key: HIVE-20628
>                 URL: https://issues.apache.org/jira/browse/HIVE-20628
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive, HiveServer2, Parser, Security
>    Affects Versions: 2.1.0
>         Environment: The error can be simulated using HDP 2.6.4 sandbox
>            Reporter: Darryl Dutton
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> When trying to use the map complex data type as part of dynamic column mask, Hive throws a parsing error as it is expecting a primitive type (see trace pasted below). The use case is trying to apply masking to elements within a map type by applying a custom hive UDF (to apply the mask) using Ranger. Expect Hive to support complex data types for masking in addition to the primitive types. The expectation occurs when Hive need to evaluate the UDF or apply a standard mask (pass-through works as expected). You can recreate the problem by creating a simple table with a map data type column, then applying the masking to that column through a Ranger resource based policy and  a custom function (you can use a standard Hive UDF  str_to_map('F4','XXXX') to simulate returning a map). 
> CREATE  TABLE `mask_test`(
>  `key` string, 
>  `value` map<string,string>)
> STORED AS INPUTFORMAT 
>  'org.apache.hadoop.mapred.TextInputFormat'
>  
> INSERT INTO TABLE mask_test
> SELECT 'AAA' as key, 
> map('F1','2022','F2','2222','F3','3333333') as value
> FROM (select 1 ) as temp;
>  
>  
> Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.parse.SemanticException:org.apache.hadoop.hive.ql.parse.ParseException: line 1:57 cannot recognize input near 'map' '<' 'string' in primitive type specification
>  at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.rewriteASTWithMaskAndFilter(SemanticAnalyzer.java:10370)
>  at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10486)
>  at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:219)
>  at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
>  at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:465)
>  at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:321)
>  at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1224)
>  at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1218)
>  at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:146)
>  ... 15 more
> Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.parse.ParseException:line 1:57 cannot recognize input near 'map' '<' 'string' in primitive type specification
>  at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:214)
>  at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:171)
>  at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:166)
>  at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.rewriteASTWithMaskAndFilter(SemanticAnalyzer.java:10368)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)