You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "ChuaSweeChin (Jira)" <ji...@apache.org> on 2021/07/15 18:29:00 UTC
[jira] [Comment Edited] (NIFI-8733) PutSQL fragmented transaction
issue
[ https://issues.apache.org/jira/browse/NIFI-8733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17381527#comment-17381527 ]
ChuaSweeChin edited comment on NIFI-8733 at 7/15/21, 6:28 PM:
--------------------------------------------------------------
During the test, I have set the batch size to 10,000. I have attached my setting in the ticket.
was (Author: sampiochua):
During the test, I have set the batch size to 10,000.
> PutSQL fragmented transaction issue
> -----------------------------------
>
> Key: NIFI-8733
> URL: https://issues.apache.org/jira/browse/NIFI-8733
> Project: Apache NiFi
> Issue Type: Bug
> Components: NiFi Stateless
> Affects Versions: 1.12.1
> Environment: 3 Node Cluster
> Reporter: ChuaSweeChin
> Priority: Major
> Attachments: Screenshot 2021-06-22 at 1.40.37 AM.png, Screenshot 2021-06-22 at 1.48.13 AM.png
>
>
> I am hitting an issue with PutSQL processor which is pretty weird.
> What happen is, I have a 987 flow files sharing the same attributes [fragment.identifier], [fragment.count] and their respective [fragment.index] all stuck in upstream of PutSQL.
> In PutSQL, I have configured [Support Fragmented Transactions] to TRUE. If I understand correctly, when [Support Fragmented Transactions] is TRUE, PutSQL will wait till all the fragments arrive before executing the transaction, in which in my case, the fragment all arrived however the sql execution did not.
> Only when I restart PutSQL, then all the 987 flow files will get processed successfully right away.
> If I reconfigure [Transaction Timeout] to 5000ms, then the 987 flow files will flow to failure relationship after 5000ms. This I assume PutSQL is unable to detect that all the fragments has arrived upstream and push it to failure.
> The other thing I tried is that if I reconfigure [Support Fragmented Transactions] to FALSE, then the flow files will all get processed successfully right away.
> I am not entirely sure why PutSQL is behaving this way for fragmented transaction I was hoping someone could help me out on this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)