[jira] [Created] (NIFI-8130) PutDatabaseRecord after MergeRecord randomly hangs forcing to discard the whole queue

["Alessandro D'Armiento (Jira)" <ji...@apache.org>]

Alessandro D'Armiento created NIFI-8130:
-------------------------------------------

             Summary: PutDatabaseRecord after MergeRecord randomly hangs forcing to discard the whole queue
                 Key: NIFI-8130
                 URL: https://issues.apache.org/jira/browse/NIFI-8130
             Project: Apache NiFi
          Issue Type: Bug
            Reporter: Alessandro D'Armiento


This bug is hard to replicate as it happens randomly.

In the following (common) configuration, in which multiple records are merged and then sent to a PutDatabaseRecord, it happens sometimes that a specific FlowFile cause the PutDatabaseRecord to fail with `FlowFileHandlingException: FlowFile already marked for transfer`

!Screenshot 2021-01-11 at 11.38.45.png!

!Screenshot 2021-01-11 at 11.34.06.png!

In case of such an event, the processor remains stuck trying to process that specific FlowFile (i.e. it is not routed to the failure relationship). This forces the user to empty the whole queue in order to continue, which causes data loss. 

I noticed the following: 
 * The issue is bound with the FlowFile: the same FlowFile will make multiple processors to fail with the same error.
 * Creating a new FlowFile with the same content (i.e. publishing the FlowFile on a Kafka queue and consuming it right after) doesn't solve the issue, and the FlowFile will raise the error again once sent to the PutDatabaseRecord
 * This error happened to me only when using the PutDatabaseRecord after a MergeRecord (in order to batch multiple records in a single DB transaction). 
 * This issue was already raised in the Cloudera Community Forum, alas, without any answer. 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)