[jira] [Created] (NIFI-7220) Handle partial failure of bulk insert in PutMongoRecord

["Rupert Bignall (Jira)" <ji...@apache.org>]

Rupert Bignall created NIFI-7220:
------------------------------------

             Summary: Handle partial failure of bulk insert in PutMongoRecord
                 Key: NIFI-7220
                 URL: https://issues.apache.org/jira/browse/NIFI-7220
             Project: Apache NiFi
          Issue Type: Improvement
    Affects Versions: 1.11.0
            Reporter: Rupert Bignall


We need to be able to handle partial failures in PutMongoRecord batches, so we can ignore duplicates, retry ones that have failed for other reasons and not send failed records onto subsequent processing in the flow.

 

First, a new parameter is required to allow the 'ordered' option to be set to false if required - this means that all records in the batch will be attempted, instead of stopping at the first error.

 

Second, a new parameter is required to send the writeErrors array from mongo into an 'errors' relationship. If this parameter is used, only the successful rows will be sent to the success relationship and the writeErrors (which contain the error code/description as well as the failed record) to the errors relationship.

 

Implementation note: There is a similar approach in PutElasticSearchHttpRecord, but that does not allow you to check what the errors were, so the exact same approach cannot be used.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)