[jira] [Commented] (NIFI-5044) SelectHiveQL accept only one statement

["Matt Burgess (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-5044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427226#comment-16427226 ] 

Matt Burgess commented on NIFI-5044:
------------------------------------

The comments only mention some improvements to both processors, the Statement Delimiter property is only added to (and supported by) PutHiveQL. Changing this Jira to an Improvement to support multiple statements in SelectHiveQL.

This can be useful for cases such as your example when you need to set properties on the session and then issue a query, but how would we handle multiple statements that each return result sets? Would we process each statement "normally" in turn, issuing flow files that would presumably have very different schemas?

If we only need to support the "set X=Y" use case, then perhaps we could offer user-defined properties in SelectHiveQL, and we would use the key value pairs to issue "set X=Y" commands in the session on the user's behalf, before executing the query.  What do you think [~disoardi]?

> SelectHiveQL accept only one statement
> --------------------------------------
>
>                 Key: NIFI-5044
>                 URL: https://issues.apache.org/jira/browse/NIFI-5044
>             Project: Apache NiFi
>          Issue Type: Improvement
>    Affects Versions: 1.2.0
>            Reporter: Davide Isoardi
>            Priority: Critical
>
> In [this |[https://github.com/apache/nifi/commit/bbc714e73ba245de7bc32fd9958667c847101f7d] ] commit claims to add support to running multiple statements both on SelectHiveQL and PutHiveQL; instead, it adds only the support to PutHiveQL, so SelectHiveQL still lacks this important feature. @Matt Burgess, I saw that you worked on that, is there any reason for this? If not, can we support it?
> If I try to execute this query:
> {quote}set hive.vectorized.execution.enabled = false; SELECT * FROM table_name
> {quote}
> I have this error:
>  
> {quote}2018-04-05 13:35:40,572 ERROR [Timer-Driven Process Thread-146] o.a.nifi.processors.hive.SelectHiveQL SelectHiveQL[id=243d4c17-b1fe-14af-ffff-ffffee8ce15e] Unable to execute HiveQL select query set hive.vectorized.execution.enabled = false; SELECT * FROM table_name for StandardFlowFileRecord[uuid=0e035558-07ce-473b-b0d4-ac00b8b1df93,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1522824912161-2753, container=default, section=705], offset=838441, length=25],offset=0,name=cliente_attributi.csv,size=25] due to org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException: The query did not generate a result set!; routing to failure: {}
>  org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException: The query did not generate a result set!
>  at org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:305)
>  at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2529)
>  at org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:275)
>  at org.apache.nifi.processors.hive.SelectHiveQL.lambda$onTrigger$0(SelectHiveQL.java:215)
>  at org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:114)
>  at org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:106)
>  at org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:215)
>  at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
>  at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
>  at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
>  at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
>  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  at java.lang.Thread.run(Thread.java:745)
>  Caused by: java.sql.SQLException: The query did not generate a result set!
>  at org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:438)
>  at org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
>  at org.apache.commons.dbcp.DelegatingStatement.executeQuery(DelegatingStatement.java:208)
>  at org.apache.nifi.processors.hive.SelectHiveQL$2.process(SelectHiveQL.java:293)
> {quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)