[jira] [Created] (NIFI-6040) ExecuteSQL Batch Loading

["Artem Anokhin (JIRA)" <ji...@apache.org>]

Artem Anokhin created NIFI-6040:
-----------------------------------

             Summary: ExecuteSQL Batch Loading 
                 Key: NIFI-6040
                 URL: https://issues.apache.org/jira/browse/NIFI-6040
             Project: Apache NiFi
          Issue Type: Bug
          Components: Core Framework, Docker
    Affects Versions: 1.8.0
            Reporter: Artem Anokhin
             Fix For: 1.9.0
         Attachments: image-2019-02-15-18-13-39-190.png

Hello!
TC to reproduce:
1) Create GenerateFlowFile with the next Custom Text:
select 1 from dual
union all
select 2 from dual
union all
select 3 from dual
2) Create ExecuteSQL. Set "Output Batch Size" = 1.
3) Start

AR:
 !image-2019-02-15-18-13-39-190.png!

ER:
3 flowfiles in the "success" connection.

Root Cause:
{code:java}
if (outputBatchSize > 0 && resultSetFlowFiles.size() >= outputBatchSize) {
    session.transfer(resultSetFlowFiles, REL_SUCCESS);
    session.commit();
    resultSetFlowFiles.clear();
}
{code}

Commit method has the next requirement:
Commits the current session ensuring all operations against FlowFiles within this session are atomically persisted. *All FlowFiles operated on within this session must be accounted for by transfer or removal or the commit will fail*.

At the time of any commit, an input flowfile (fileToProcess) still exists and has not been accounted for by transfer or removal. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)