You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Matt Burgess (Jira)" <ji...@apache.org> on 2021/02/23 21:50:00 UTC
[jira] [Updated] (NIFI-8119) ExecuteSQL does not properly free
database ressources
[ https://issues.apache.org/jira/browse/NIFI-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matt Burgess updated NIFI-8119:
-------------------------------
Affects Version/s: (was: 1.11.2)
Status: Patch Available (was: Open)
> ExecuteSQL does not properly free database ressources
> -----------------------------------------------------
>
> Key: NIFI-8119
> URL: https://issues.apache.org/jira/browse/NIFI-8119
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Reporter: Christian Gumpert
> Priority: Minor
> Time Spent: 40m
> Remaining Estimate: 0h
>
> We are using Nifi to ingest data from a Teradata database into our S3-based data lake using a typical pattern of GenerateTableFetch and ExecuteSQL processors. Our Teradata database tables contain columns of type CLOB (which contains some JSON data).
> We have installed the Teradata JDBC driver from the Teradata Tools and Utilities package version 16.10.26.00 as described in this [Cloudera community article|https://community.cloudera.com/t5/Community-Articles/Using-Teradata-JDBC-connector-in-NiFi/ta-p/246783].
> After having configured a DBConnectionPool service with the Teradata connection parameters we are able to execute our flow. The GenerateTableFetch processors generates flowfiles containing SQL Queries which are then executed by the ExecuteSQL processor.
> After having processed the first 15 flowfiles the ExecuteSQL processor yields the following error:
> {noformat}
> 2020-12-17T12:53:17+01:00 L921000109090A nifi-app.log: 2020-12-17 12:53:11,578 ERROR [Timer-Driven Process Thread-2] o.a.nifi.processors.standard.ExecuteSQL ExecuteSQL[id=afa23b0f-2e57-1fb6-d047-13646de03ebf] Unable to execute SQL select query call devezv_replworkedec.get_edec_meldung(561, 562, 2, 0); for StandardFlowFileRecord[uuid=ff7219a7-14e9-404e-a57a-28121653fed8,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1607610786368-4986, container=repo0, section=890], offset=701888, length=1077672],offset=32266,name=ff7219a7-14e9-404e-a57a-28121653fed8,size=58] due to java.sql.SQLException: [Teradata Database] [TeraJDBC 16.10.00.07] [Error 3130] [SQLState HY000] GET_EDEC_MELDUNG:Response limit exceeded.; routing to failure: java.sql.SQLException: [Teradata Database] [TeraJDBC 16.10.00.07] [Error 3130] [SQLState HY000] GET_EDEC_MELDUNG:Response limit exceeded.
> java.sql.SQLException: [Teradata Database] [TeraJDBC 16.10.00.07] [Error 3130] [SQLState HY000] GET_EDEC_MELDUNG:Response limit exceeded.
> at com.teradata.jdbc.jdbc_4.util.ErrorFactory.makeDatabaseSQLException(ErrorFactory.java:309)
> at com.teradata.jdbc.jdbc_4.statemachine.ReceiveInitSubState.action(ReceiveInitSubState.java:103)
> at com.teradata.jdbc.jdbc_4.statemachine.StatementReceiveState.subStateMachine(StatementReceiveState.java:311)
> at com.teradata.jdbc.jdbc_4.statemachine.StatementReceiveState.action(StatementReceiveState.java:200)
> at com.teradata.jdbc.jdbc_4.statemachine.StatementController.runBody(StatementController.java:137)
> at com.teradata.jdbc.jdbc_4.statemachine.PreparedStatementController.run(PreparedStatementController.java:46)
> at com.teradata.jdbc.jdbc_4.TDStatement.executeStatement(TDStatement.java:389)
> at com.teradata.jdbc.jdbc_4.TDStatement.executeStatement(TDStatement.java:331)
> at com.teradata.jdbc.jdbc_4.TDPreparedStatement.doPrepExecute(TDPreparedStatement.java:177)
> at com.teradata.jdbc.jdbc_4.TDPreparedStatement.execute(TDPreparedStatement.java:2778)
> at org.apache.commons.dbcp2.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:94)
> at org.apache.commons.dbcp2.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:94)
> at org.apache.nifi.processors.standard.AbstractExecuteSQL.onTrigger(AbstractExecuteSQL.java:266)
> at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176)
> at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)
> at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){noformat}
> After googling for a while it appears that the important part is
> *java.sql.SQLException: [Teradata Database] [TeraJDBC 16.10.00.07] [Error 3130] [SQLState HY000] GET_EDEC_MELDUNG:Response limit exceeded.*
> The [Teradata documentation for "Working with LOBs"|https://teradata-docs.s3.amazonaws.com/doc/connectivity/jdbc/reference/current/jdbcug_chapter_5.html#CHDGCHBB] contains information on how to properly free resources to avoid this limit.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)