[jira] [Created] (NIFI-10066) Option in ValidateRecord to only send failed record to failure relationship

["Anders (Jira)" <ji...@apache.org>]

Anders created NIFI-10066:
-----------------------------

             Summary: Option in ValidateRecord to only send failed record to failure relationship
                 Key: NIFI-10066
                 URL: https://issues.apache.org/jira/browse/NIFI-10066
             Project: Apache NiFi
          Issue Type: Improvement
            Reporter: Anders


Currently, the ValidateRecord processor treats invalid and failure differently.

With invalid records (records passing parsing but not validaiton), they are sent to the invalid relationship, while valid records in the flowfile are sent to the valid relationship.

Whenever the processor encounters a junk records (random text, invalid synax, incomplete json, etc.), the _entire_ flowfile is sent to the failure relationship, not just the failed record.

I am sure there are use cases where this makes sense, but I currently have a use-case where I get large flowfiles with a large amount of json objects. Every now and then a junk (non-json) line gets into the (flow)files, and I want to filter them out.

It would be very usefull if ValidateRecord could _only_ send those junk records to the failure relationship, and not the entire flowfile. Currently, I have to split everything to the failure relationship in single line flowfiles, and run them by ValidateRecord again to identify only the junk records, and send the remaining valid records onward in my flow.

This results in needlessly complicated logic in the flow.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)