[jira] [Assigned] (HIVE-23172) Quoted Backtick Columns Are Not Parsing Correctly

["David Mollitor (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-23172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

David Mollitor reassigned HIVE-23172:
-------------------------------------


> Quoted Backtick Columns Are Not Parsing Correctly
> -------------------------------------------------
>
>                 Key: HIVE-23172
>                 URL: https://issues.apache.org/jira/browse/HIVE-23172
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: David Mollitor
>            Assignee: David Mollitor
>            Priority: Critical
>
> I recently came across a weird behavior while examining failures of {{special_character_in_tabnames_2.q}} while working on HIVE-23150. I was surprised to see it fail because I couldn't see of any reason why it should... it's doing pretty standard SQL statements just like every other test, but for some reason this test is just a *little bit* differently than most others and it brought this issue to light.
> Turns out,... the parsing of table names is pretty much wrong across the board.
> The statement that caught my attention was this:
> {code:sql}
> DROP TABLE IF EXISTS `s/c`;
> {code}
> And here is the relevant grammar:
> {code:none}
> fragment
> RegexComponent
>     : 'a'..'z' | 'A'..'Z' | '0'..'9' | '_'
>     | PLUS | STAR | QUESTION | MINUS | DOT
>     | LPAREN | RPAREN | LSQUARE | RSQUARE | LCURLY | RCURLY
>     | BITWISEXOR | BITWISEOR | DOLLAR | '!'
>     ;
> Identifier
>     :
>     (Letter | Digit) (Letter | Digit | '_')*
>     | {allowQuotedId()}? QuotedIdentifier  /* though at the language level we allow all Identifiers to be QuotedIdentifiers; 
>                                               at the API level only columns are allowed to be of this form */
>     | '`' RegexComponent+ '`'
>     ;
> fragment    
> QuotedIdentifier 
>     :
>     '`'  ( '``' | ~('`') )* '`' { setText(StringUtils.replace(getText().substring(1, getText().length() -1 ), "``", "`")); }
>     ;
> {code}
> The mystery for me was that, for some reason, this String {{`s/c`}} was being stripped of its back-ticks. Every other test I investigated did not have this behavior, the back ticks were always preserved around the table name. The main Hive Java code base would see the back-ticks and deal with it internally. For HIVE-23150, I introduced some sanity checks and they were failing because they were expecting the back ticks to be present.
> With the help of HIVE-23171 I finally figured it out. So, what I discovered is that pretty much every table name is hitting the {{RegexComponent}} rule and the back ticks are carried forward. However, {{`s/c`}} the forward slash `/` is not allowable in {{RegexComponent}} so it hits on {{QuotedIdentifier}} rule which is trimming the back ticks.
> I validated this by disabling {{QuotedIdentifier}}. When I did this, {{`s/c`}} fails in error but {{`sc`}} parses successfully... because {{`sc`}} is being treated as a {{RegexComponent}}.
> So, if you have {{allowQuotedId}} disabled, table names can only use the characters defined in the {{RegexComponent}} rule (otherwise it errors), and it will *not* strip the back ticks. If you have {{allowQuotedId}} enabled, then if the table name has a character not specified in {{RegexComponent}}, it will identify it as a table name and it *will* strip the back ticks, if all the characters are part of {{RegexComponent}} then it will *not* strip the back ticks.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)