[jira] [Resolved] (NIFI-7008) PutS3Object: Invalid V4 Authorization Header When Using Custom S3 Blobstore

["Pierre Villard (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/NIFI-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Pierre Villard resolved NIFI-7008.
----------------------------------
    Resolution: Fixed

When testing recent versions of NiFi with recent versions of MinIO, I cannot reproduce this problem and it properly uses Signature v4.

> PutS3Object: Invalid V4 Authorization Header When Using Custom S3 Blobstore
> ---------------------------------------------------------------------------
>
>                 Key: NIFI-7008
>                 URL: https://issues.apache.org/jira/browse/NIFI-7008
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>    Affects Versions: 1.10.0
>         Environment:  Nifi 1.10.0, connecting to MinIO 2019-12-19 S3-Compatible Blobstore
>            Reporter: Matt M
>            Priority: Minor
>
> Hello!
> Some background: I'm currently attempting to use a {{PutS3Object}} processor in Nifi {{1.10.0}} to upload an object to a [MinIO|https://min.io/] cluster. The MinIO cluster is configured to act as an S3-compatible blobstore in the {{us-east-1}} region. The MinIO cluster is running on an internal private network at my company at https://s3.mydomain.mycompany.com .
> The {{PutS3Object}} processor is configured thusly:
> - {{Bucket}}: {{mybucket}}
> - {{Region}}: {{US East (N. Virginia)}}
> - {{Endpoint Override URL}}: {{https://s3.mydomain.mycompany.com:9000}}
> - {{Signer Override}}: {{Signature v4}}
> All other options are left at their default values.
> What happens when I attempt to use the processor to put a file into MinIO is that the processor shows an error like the following: {{Status Code: 400, Error Code: AuthorizationHeaderMalformed}}.
> After some debugging, it looks like that the HTTP {{Authorization}} header being generated by Nifi isn't quite what I would expect. The {{Authorization}} header starts off like this:
> {noformat}
> Authorization: AWS4-HMAC-SHA256 Credential=AKIAIOSFODNN7EXAMPLE/20200111/mydomain/s3/aws4_request ...
> {noformat}
> Whereas what I would _expect_ is something more like this:
> {noformat}
> Authorization: AWS4-HMAC-SHA256 Credential=AKIAIOSFODNN7EXAMPLE/20200111/us-east-1/s3/aws4_request ...
> {noformat}
> The current behaviour seems to be: take part of the domain from the {{Endpoint Override URL}} and use that as the region inside of the {{Authorization}} header, instead of using the {{Region}} that was specified. 
> As a workaround for now we can use {{Signature v2}} instead, but how long MinIO will continue to support {{Signature v2}} at this time is unknown. 
> Would it be possible to fix the S3 family of processors so that they use the {{Region}} being specified instead of attempting to extract the region from the URL instead?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)