[jira] [Created] (NIFI-4302) QueryDatabaseTable bug

["Valeriy Plyashko (JIRA)" <ji...@apache.org>]

Valeriy Plyashko created NIFI-4302:
--------------------------------------

             Summary: QueryDatabaseTable bug
                 Key: NIFI-4302
                 URL: https://issues.apache.org/jira/browse/NIFI-4302
             Project: Apache NiFi
          Issue Type: Bug
          Components: Extensions
    Affects Versions: 1.3.0
            Reporter: Valeriy Plyashko


Hello! Please fix

{noformat}
org.apache.nifi.processors.standard.QueryDatabaseTable#onTrigger
org.apache.nifi.processors.standard.util.JdbcCommon#convertToAvroStream(java.sql.ResultSet, java.io.OutputStream, org.apache.nifi.processors.standard.util.JdbcCommon.AvroConversionOptions, org.apache.nifi.processors.standard.util.JdbcCommon.ResultSetRowCallback)
{noformat}

to prevent calling java.sql.ResultSet#next after it already returned false.
----
Motivation:

I found that usage of java.sql.ResultSet at org/apache/nifi/processors/standard/QueryDatabaseTable.java:287
doesn't fully match java.sql.ResultSet documentation.

Docs for java.sql.ResultSet#next:
{noformat}
     *  If the result set type is <code>TYPE_FORWARD_ONLY</code>, it is vendor specified
     * whether their JDBC driver implementation will return <code>false</code> or
     *  throw an <code>SQLException</code> on a
     * subsequent call to <code>next</code>.
{noformat}
But loop org/apache/nifi/processors/standard/QueryDatabaseTable.java:278 doesn't care if resultSet returned false. If there were any records in past iteration and we have got not enough fragments, loop continues and calls java.sql.ResultSet#next at least once more. It is ignored that java.sql.ResultSet#next returned false in past iteration.

On some jdbc drivers it's ok. But, for example, using

{code:xml}
<dependency>
    <groupId>ru.yandex.clickhouse</groupId>
    <artifactId>clickhouse-jdbc</artifactId>
    <version>0.1.26</version>
</dependency>
{code}

causes SQLException when you call java.sql.ResultSet#next after it already returned false.

The result is:

{noformat}
2017-08-16 15:47:56,478 ERROR [Timer-Driven Process Thread-3] o.a.n.p.standard.QueryDatabaseTable QueryDatabaseTable[id=e4f48906-015d-1000-a318-4584f2755c75] Unable to execute SQL select query SELECT * FROM abc due to org.apache.nifi.processor.exception.ProcessException: Error during database query or conversion of records to Avro.: {}
org.apache.nifi.processor.exception.ProcessException: Error during database query or conversion of records to Avro.
	at org.apache.nifi.processors.standard.QueryDatabaseTable.lambda$onTrigger$13(QueryDatabaseTable.java:305)
	at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2529)
	at org.apache.nifi.processors.standard.QueryDatabaseTable.onTrigger(QueryDatabaseTable.java:299)
	at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
	at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
	at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
	at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.sql.SQLException: java.io.IOException: Attempted read on closed stream.
	at ru.yandex.clickhouse.response.ClickHouseResultSet.hasNext(ClickHouseResultSet.java:114)
	at ru.yandex.clickhouse.response.ClickHouseResultSet.next(ClickHouseResultSet.java:124)
	at org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java:207)
	at org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java:207)
	at org.apache.nifi.processors.standard.util.JdbcCommon.convertToAvroStream(JdbcCommon.java:252)
	at org.apache.nifi.processors.standard.QueryDatabaseTable.lambda$onTrigger$13(QueryDatabaseTable.java:303)
	... 13 common frames omitted
Caused by: java.io.IOException: Attempted read on closed stream.
	at org.apache.http.conn.EofSensorInputStream.isReadAllowed(EofSensorInputStream.java:109)
	at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:118)
	at ru.yandex.clickhouse.response.ClickHouseLZ4Stream.readNextBlock(ClickHouseLZ4Stream.java:82)
	at ru.yandex.clickhouse.response.ClickHouseLZ4Stream.checkNext(ClickHouseLZ4Stream.java:74)
	at ru.yandex.clickhouse.response.ClickHouseLZ4Stream.read(ClickHouseLZ4Stream.java:50)
	at ru.yandex.clickhouse.response.StreamSplitter.readFromStream(StreamSplitter.java:85)
	at ru.yandex.clickhouse.response.StreamSplitter.next(StreamSplitter.java:47)
	at ru.yandex.clickhouse.response.ClickHouseResultSet.hasNext(ClickHouseResultSet.java:108)
	... 18 common frames omitted
{noformat}




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)