You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by "Rajkumar Singh (Jira)" <ji...@apache.org> on 2020/09/23 23:54:00 UTC
[jira] [Updated] (RANGER-3009) Query with column mask fail with
ParseException if column name has special char
[ https://issues.apache.org/jira/browse/RANGER-3009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rajkumar Singh updated RANGER-3009:
-----------------------------------
Description:
hive query with column masking failed with ParseException
Table DDL
{code:java}
CREATE TABLE `emp`( `id` string, `name#` string);
{code}
The following query failed
{code:java}
select `emp`.`id`, `emp`.`name#` from (SELECT `id`, CAST(mask_show_first_n(name#, 4, 'x', 'x', 'x', -1, '1') AS string) AS `name#`, BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM `default`.`emp` )`emp`;
{code}
Error: Error while compiling statement: FAILED: ParseException line 1:79 character '#' not supported here (state=42000,code=40000)
quoting manually helped
{code:java}
select `emp`.`id`, `emp`.`name#` from (SELECT `id`, CAST(mask_show_first_n(`name#`, 4, 'x', 'x', 'x', -1, '1') AS string) AS `name#`, BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM `default`.`emp` )`emp`;
{code}
manual query change will not work for Ranger authorizer as following query
{code:java}
select * from emp;
{code}
will be rewritten to
{code:java}
select `emp`.`id`, `emp`.`name#` from (SELECT `id`, CAST(mask_show_first_n(name#, 4, 'x', 'x', 'x', -1, '1') AS string) AS `name#`, BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM `default`.`emp` )`emp`;
{code}
Ranger apply the transformer for column RangerHiveAuthorizer#applyRowFilterAndColumnMasking so we should consider the enclosing the column names in the back-ticks to make it work
https://github.com/apache/ranger/blob/master/hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java#L1332
was:
hive query with column masking failed with ParseException
Table DDL
{code:java}
CREATE TABLE `emp`( `id` string, `name#` string);
{code}
The following query failed
{code:java}
select `emp`.`id`, `emp`.`name#` from (SELECT `id`, CAST(mask_show_first_n(name#, 4, 'x', 'x', 'x', -1, '1') AS string) AS `name#`, BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM `default`.`emp` )`emp`;
{code}
Error: Error while compiling statement: FAILED: ParseException line 1:79 character '#' not supported here (state=42000,code=40000)
quoting manually helped
{code:java}
select `emp`.`id`, `emp`.`name#` from (SELECT `id`, CAST(mask_show_first_n(`name#`, 4, 'x', 'x', 'x', -1, '1') AS string) AS `name#`, BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM `default`.`emp` )`emp`;
{code}
manual query change will not work for Ranger authorizer as following query
{code:java}
select * from emp;
{code}
will be rewritten to
{code:java}
select `emp`.`id`, `emp`.`name#` from (SELECT `id`, CAST(mask_show_first_n(name#, 4, 'x', 'x', 'x', -1, '1') AS string) AS `name#`, BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM `default`.`emp` )`emp`;
{code}
Ranger apply the transformer for column here so we should consider the enclosing the column names in the back-ticks to make it work
https://github.com/apache/ranger/blob/master/hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java#L1332
> Query with column mask fail with ParseException if column name has special char
> -------------------------------------------------------------------------------
>
> Key: RANGER-3009
> URL: https://issues.apache.org/jira/browse/RANGER-3009
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
> Affects Versions: 2.1.0
> Reporter: Rajkumar Singh
> Priority: Major
>
> hive query with column masking failed with ParseException
> Table DDL
> {code:java}
> CREATE TABLE `emp`( `id` string, `name#` string);
> {code}
> The following query failed
> {code:java}
> select `emp`.`id`, `emp`.`name#` from (SELECT `id`, CAST(mask_show_first_n(name#, 4, 'x', 'x', 'x', -1, '1') AS string) AS `name#`, BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM `default`.`emp` )`emp`;
> {code}
> Error: Error while compiling statement: FAILED: ParseException line 1:79 character '#' not supported here (state=42000,code=40000)
> quoting manually helped
> {code:java}
> select `emp`.`id`, `emp`.`name#` from (SELECT `id`, CAST(mask_show_first_n(`name#`, 4, 'x', 'x', 'x', -1, '1') AS string) AS `name#`, BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM `default`.`emp` )`emp`;
> {code}
> manual query change will not work for Ranger authorizer as following query
> {code:java}
> select * from emp;
> {code}
> will be rewritten to
> {code:java}
> select `emp`.`id`, `emp`.`name#` from (SELECT `id`, CAST(mask_show_first_n(name#, 4, 'x', 'x', 'x', -1, '1') AS string) AS `name#`, BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM `default`.`emp` )`emp`;
> {code}
> Ranger apply the transformer for column RangerHiveAuthorizer#applyRowFilterAndColumnMasking so we should consider the enclosing the column names in the back-ticks to make it work
> https://github.com/apache/ranger/blob/master/hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizer.java#L1332
--
This message was sent by Atlassian Jira
(v8.3.4#803005)