[jira] [Created] (NIFI-5095) PutHiveQL should not log warning message when it fails to parse SET property command

["Koji Kawamura (JIRA)" <ji...@apache.org>]

Koji Kawamura created NIFI-5095:
-----------------------------------

             Summary: PutHiveQL should not log warning message when it fails to parse SET property command
                 Key: NIFI-5095
                 URL: https://issues.apache.org/jira/browse/NIFI-5095
             Project: Apache NiFi
          Issue Type: Bug
          Components: Extensions
    Affects Versions: 1.5.0
            Reporter: Koji Kawamura
            Assignee: Koji Kawamura


PutHiveQL can accept multiple queries separated by a specified delimiter string, ';' by default. It supports users to specify Hive parameters by 'SET' statement. E.g. set 'hive.exec.dynamic.partition.mode'=nonstrict

PutHiveQL also parses each query string with Hive ParseDriver, in order to find input/output table names within queries. However, the aforementioned 'SET' command is not a valid Hive query. The only query can start with 'SET' is 'SET ROLE'. [https://raw.githubusercontent.com/apache/hive/master/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g]
 When set property statement is parsed, following warning message is logged and shown in the NiFi UI:
{code:java}
2018-04-19 05:34:05,616 WARN [Timer-Driven Process Thread-8] o.apache.nifi.processors.hive.PutHiveQL PutHiveQL[id=db408703-0162-1000-0000-000073ad3455] Failed to parse hiveQL: set hive.exec.dynamic.partition.mode=nonstrict due to org.apache.hadoop.hive.ql.parse.ParseException: line 1:4 missing KW_ROLE at 'hive' near 'hive' line 1:8 missing EOF at '.' near 'hive':
{code}
In case there are other DML statements such as 'INSERT ...' in the same FlowFile content, those queries are performed successfully regardless of having above parse failure. However, the warning message is mis-leading, it looks as if queries have failed. We should not show such warning message for set property commands.

We can short-circuit query parse logic if statement starts with 'set', since 'set role' does not have any target table. As a reference Hive HCatCli.java has the similar filtering logic.
 [https://github.com/apache/hive/blob/master/hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/HCatCli.java#L283]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)