You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Artem Anokhin (JIRA)" <ji...@apache.org> on 2019/02/15 15:24:00 UTC
[jira] [Updated] (NIFI-6040) ExecuteSQL Batch Loading
[ https://issues.apache.org/jira/browse/NIFI-6040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Artem Anokhin updated NIFI-6040:
--------------------------------
Description:
Hello!
h2. TC to reproduce:
1) Create GenerateFlowFile with the next Custom Text:
select 1 from dual
union all
select 2 from dual
union all
select 3 from dual
2) Create ExecuteSQL. Set "Output Batch Size" = 1.
3) Start
h2. AR:
!image-2019-02-15-18-13-39-190.png!
h2. ER:
3 flowfiles in the "success" connection.
h2. Root Cause:
{code:java}
if (outputBatchSize > 0 && resultSetFlowFiles.size() >= outputBatchSize) {
session.transfer(resultSetFlowFiles, REL_SUCCESS);
session.commit();
resultSetFlowFiles.clear();
}
{code}
Commit method has the next requirement:
Commits the current session ensuring all operations against FlowFiles within this session are atomically persisted. *All FlowFiles operated on within this session must be accounted for by transfer or removal or the commit will fail*.
At the time of any commit, an input flowfile (fileToProcess) still exists and has not been accounted for by transfer or removal.
was:
Hello!
TC to reproduce:
1) Create GenerateFlowFile with the next Custom Text:
select 1 from dual
union all
select 2 from dual
union all
select 3 from dual
2) Create ExecuteSQL. Set "Output Batch Size" = 1.
3) Start
AR:
!image-2019-02-15-18-13-39-190.png!
ER:
3 flowfiles in the "success" connection.
Root Cause:
{code:java}
if (outputBatchSize > 0 && resultSetFlowFiles.size() >= outputBatchSize) {
session.transfer(resultSetFlowFiles, REL_SUCCESS);
session.commit();
resultSetFlowFiles.clear();
}
{code}
Commit method has the next requirement:
Commits the current session ensuring all operations against FlowFiles within this session are atomically persisted. *All FlowFiles operated on within this session must be accounted for by transfer or removal or the commit will fail*.
At the time of any commit, an input flowfile (fileToProcess) still exists and has not been accounted for by transfer or removal.
> ExecuteSQL Batch Loading
> -------------------------
>
> Key: NIFI-6040
> URL: https://issues.apache.org/jira/browse/NIFI-6040
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework, Docker
> Affects Versions: 1.8.0
> Reporter: Artem Anokhin
> Priority: Major
> Labels: easyfix, usability
> Fix For: 1.9.0
>
> Attachments: image-2019-02-15-18-13-39-190.png
>
>
> Hello!
> h2. TC to reproduce:
> 1) Create GenerateFlowFile with the next Custom Text:
> select 1 from dual
> union all
> select 2 from dual
> union all
> select 3 from dual
> 2) Create ExecuteSQL. Set "Output Batch Size" = 1.
> 3) Start
> h2. AR:
> !image-2019-02-15-18-13-39-190.png!
> h2. ER:
> 3 flowfiles in the "success" connection.
> h2. Root Cause:
> {code:java}
> if (outputBatchSize > 0 && resultSetFlowFiles.size() >= outputBatchSize) {
> session.transfer(resultSetFlowFiles, REL_SUCCESS);
> session.commit();
> resultSetFlowFiles.clear();
> }
> {code}
> Commit method has the next requirement:
> Commits the current session ensuring all operations against FlowFiles within this session are atomically persisted. *All FlowFiles operated on within this session must be accounted for by transfer or removal or the commit will fail*.
> At the time of any commit, an input flowfile (fileToProcess) still exists and has not been accounted for by transfer or removal.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)