[jira] [Created] (NIFI-7008) PutS3Object: Invalid V4 Authorization Header When Using Custom S3 Blobstore

["Matt M (Jira)" <ji...@apache.org>]

Matt M created NIFI-7008:
----------------------------

             Summary: PutS3Object: Invalid V4 Authorization Header When Using Custom S3 Blobstore
                 Key: NIFI-7008
                 URL: https://issues.apache.org/jira/browse/NIFI-7008
             Project: Apache NiFi
          Issue Type: Bug
          Components: Extensions
    Affects Versions: 1.10.0
         Environment:  Nifi 1.10.0, connecting to MinIO 2019-12-19 S3-Compatible Blobstore
            Reporter: Matt M


Hello!

Some background: I'm currently attempting to use a {{PutS3Object}} processor in Nifi {{1.10.0}} to upload an object to a [MinIO|https://min.io/] cluster. The MinIO cluster is configured to act as an S3-compatible blobstore in the {{us-east-1}} region. The MinIO cluster is running on an internal private network at my company at https://s3.mydomain.mycompany.com .

The {{PutS3Object}} processor is configured thusly:
- {{Bucket}}: {{mybucket}}
- {{Region}}: {{US East (N. Virginia)}}
- {{Endpoint Override URL}}: {{https://s3.mydomain.mycompany.com:9000}}
- {{Signer Override}}: {{Signature v4}}

All other options are left at their default values.

What happens when I attempt to use the processor to put a file into MinIO is that the processor shows an error like the following: {{Status Code: 400, Error Code: AuthorizationHeaderMalformed}}.

After some debugging, it looks like that the HTTP {{Authorization}} header being generated by Nifi isn't quite what I would expect. The {{Authorization}} header starts off like this:

{noformat}
Authorization: AWS4-HMAC-SHA256 Credential=AKIAIOSFODNN7EXAMPLE/20200111/mydomain/s3/aws4_request ...
{noformat}
Whereas what I would _expect_ is something more like this:
{noformat}
Authorization: AWS4-HMAC-SHA256 Credential=AKIAIOSFODNN7EXAMPLE/20200111/us-east-1/s3/aws4_request ...
{noformat}

The current behaviour seems to be: take part of the domain from the {{Endpoint Override URL}} and use that as the region inside of the {{Authorization}} header, instead of using the {{Region}} that was specified. 

As a workaround for now we can use {{Signature v2}} instead, but how long MinIO will continue to support {{Signature v2}} at this time is unknown. 

Would it be possible to fix the S3 family of processors so that they use the {{Region}} being specified instead of attempting to extract the region from the URL instead?




--
This message was sent by Atlassian Jira
(v8.3.4#803005)