[jira] [Commented] (NIFI-7371) Improve error handling of S3 processors

["Paul Grey (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-7371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413376#comment-17413376 ] 

Paul Grey commented on NIFI-7371:
---------------------------------

Hi [~EndzeitBegins].

I was looking over open tickets a few weeks ago and noticed this one. I'm fairly new to the NiFi project, have some prior experience working with AWS, and I thought this ticket might be a good opportunity to become more familiar with both. It occurred to me later that I should've reached out first to gauge your interest in working this ticket, or in
collaborating. Sorry about that!

Thanks for sharing your vision for the ticket scope, and for the concrete examples you offered. I think this would be a terrific enhancement. It seems like additional relationships and additional attribute(s) might both be useful. I plan to pursue this work in the next few days, but I'm open to any new thoughts you may have about the feature. If you have time to pitch in, that would be awesome too.

Looking forward to making this idea real, and improving the usability of the project.

 

> Improve error handling of S3 processors
> ---------------------------------------
>
>                 Key: NIFI-7371
>                 URL: https://issues.apache.org/jira/browse/NIFI-7371
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>    Affects Versions: 1.11.4
>            Reporter: endzeit
>            Assignee: Paul Grey
>            Priority: Major
>
> Currently the S3 processors, such as FetchS3Object, only expose the relationsships "success" and "failure". However the are a multitute of reasons why an interaction with an S3 storage might fail.
> As of now there is no easy way of knowing why a flow file was directed to the "failure" relationsship just by the flow file itself.
> A way of finding out the reason might be to search for a corresponing log / bulletin entry.
>  This seems rather complicated.
> A use case where having this information is useful could be when deciding whether
>  * the action should be retried, e.g. on a timeout,
>  * or the failure be handled, e.g. when the object for the specified key does not exists.
> I haven't looked much into the underlying AWS library yet as I wanted to discuss whether such an improvement is desired at all first?
>  If so, should the information be exposed via
>  * additional / other relationsships, such as in the FetchSFTP processor,
>  * or rather an attribute added to the flow file, such as in the ValidateXml processor?
> This might also apply to other AWS processors but we just have come across the S3 processors as we use them quite regulary.
> Any thoughts or comments would be highly appreciated!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)