[jira] [Commented] (NIFI-7371) Improve error handling of S3 processors

["Paul Grey (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NIFI-7371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17460948#comment-17460948 ] 

Paul Grey commented on NIFI-7371:
---------------------------------

Finally circled back to this.  PR is up.  Did a little testing for some failure conditions, and pulled this data out:

[INCORRECT BUCKET NAME]
s3.additionalDetails = \{BucketName=<bad bucket name>, Error=<base64 string>}
s3.errorCode = NoSuchBucket
s3.errorMessage = The specified bucket does not exist
s3.exception = com.amazonaws.services.s3.model.AmazonS3Exception
s3.statusCode = 404

[BAD FILENAME] (appended garbage to end of FetchS3Object field "Object Key")
s3.additionalDetails = \{Error=<base64 string>, Key=<bad path>}
s3.errorCode = NoSuchKey
s3.errorMessage = The specified key does not exist.
s3.exception = com.amazonaws.services.s3.model.AmazonS3Exception
s3.statusCode = 404

[NETWORK CONNECTIVITY FAILURE] (disabled wifi)
s3.exception = com.amazonaws.SdkClientException

[BAD CREDENTIALS] (incorrect AWS Secret Access Key)
s3.additionalDetails = \{CanonicalRequestBytes=<hex string>}
s3.errorCode = SignatureDoesNotMatch
s3.errorMessage = The request signature we calculated does not match the signature you provided. Check your key and signing method.
s3.exception = com.amazonaws.services.s3.model.AmazonS3Exception
s3.statusCode = 403

[BAD CREDENTIALS] (incorrect AWS Access Key ID)
s3.additionalDetails = \{AWSAccessKeyId=<bad key id>, Error=<base64 string>}
s3.errorCode = InvalidAccessKeyId
s3.errorMessage = The AWS Access Key Id you provided does not exist in our records.
s3.exception = com.amazonaws.services.s3.model.AmazonS3Exception
s3.statusCode = 403

 

> Improve error handling of S3 processors
> ---------------------------------------
>
>                 Key: NIFI-7371
>                 URL: https://issues.apache.org/jira/browse/NIFI-7371
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>    Affects Versions: 1.11.4
>            Reporter: endzeit
>            Assignee: Paul Grey
>            Priority: Major
>          Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Currently the S3 processors, such as FetchS3Object, only expose the relationsships "success" and "failure". However the are a multitute of reasons why an interaction with an S3 storage might fail.
> As of now there is no easy way of knowing why a flow file was directed to the "failure" relationsship just by the flow file itself.
> A way of finding out the reason might be to search for a corresponing log / bulletin entry.
>  This seems rather complicated.
> A use case where having this information is useful could be when deciding whether
>  * the action should be retried, e.g. on a timeout,
>  * or the failure be handled, e.g. when the object for the specified key does not exists.
> I haven't looked much into the underlying AWS library yet as I wanted to discuss whether such an improvement is desired at all first?
>  If so, should the information be exposed via
>  * additional / other relationsships, such as in the FetchSFTP processor,
>  * or rather an attribute added to the flow file, such as in the ValidateXml processor?
> This might also apply to other AWS processors but we just have come across the S3 processors as we use them quite regulary.
> Any thoughts or comments would be highly appreciated!



--
This message was sent by Atlassian Jira
(v8.20.1#820001)