[jira] [Commented] (NIFI-7444) Add "retry" relation to the Convert Record Processor

["Mark Payne (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-7444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17106435#comment-17106435 ] 

Mark Payne commented on NIFI-7444:
----------------------------------

Thanks for bringing this up [~rsoni]. I would rather see a 'comms.failure' relationship than a 'retry'. Retry is a bit prescriptive, as it assumes that comms failures should be retried, and doesn't really explain why the data was routed that way. But otherwise, I think this is reasonable.

Now, the down side to this, is that it means that existing processors would be invalid because they would not have a connection configured with that relationship. Often, we can avoid this by auto-terminating a new relationship, but in this case we certainly wouldn't want to auto-terminate the relationship by default because that would result in data loss. May need to think a bit about the best way to address the backward compatibility angle. Worst case is we just note it on the Migration Guidance but it can be pretty painful, especially if we update several processors for the same reason.

> Add "retry" relation to the Convert Record Processor
> ----------------------------------------------------
>
>                 Key: NIFI-7444
>                 URL: https://issues.apache.org/jira/browse/NIFI-7444
>             Project: Apache NiFi
>          Issue Type: New Feature
>          Components: Configuration
>            Reporter: Rahul Soni
>            Priority: Minor
>
> Follows the case for adding a retry relationship and redirect the "failed" conversion to this relationship in certain scenarios with one of those discussed below.
> Today we have either success or failure relationship for ConvertRecord processor. It works well until the Record Reader/Writer is using Schema Registry and then we reach a point where Schema Registry is unavailable due to any reason. Now the records which are supposed to be converted from one format to another are routed to failure because “schema” is not found for those flow files.
> One can re-attempt those flow files in one way or another but the issue is it’s not effective to retry a flow file any number of times which doesn’t have a schema available, actually. On the flip side, we can have a counter that the flow file should be tried n number of times before being redirected to the failure relationship but then again we are on the mercy of Schema Registry to be up within the configured period of re-attempts before we potentially "lose" the flow file by redirecting it to the failed relationship.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)