You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by "Joseph E. Gottman" <jg...@proteuseng.com> on 2016/02/19 18:56:35 UTC
Problem with PutS3Object processor
I am trying to use a PutS3Object processor with the "Endpoint Override URL" pointing to a custom endpoint. I keep failing with the error message "You must specify a date for this operation". I am using NiFi version 0.4.1 with Java 8 and Centos 6.7. I suspect this might have something to do with bug #1025, but according to your notes it was fixed for version 0.4.0.
[Proteus Technologies Outsmart Logo]<http://www.proteus-technologies.com/> Joe Gottman
Senior Member of Technical Staff
jgottman@proteuseng.com<ma...@proteuseng.com>
133 National Business Pkwy, Ste 150
Annapolis Junction, MD 20701
(Office) 301.377.7144
www.proteus-technologies.com<http://www.proteus-technologies.com/>
TheBlend.ProteusEng.com<http://theblend.proteuseng.com/> (Digital magazine)
This electronic message and any files transmitted with it contain information which may be privileged and/or proprietary. The information is intended for use solely by the intended recipient(s). If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of this information is prohibited. If you have received this electronic message in error, please advise the sender by reply email or by telephone (443.539.3400) and delete the message.
Re: Problem with PutS3Object processor
Posted by Joe Skora <js...@gmail.com>.
I spoke to Joseph off list, he is not using Amazon but an S3 compatible
back-end that authenticates request differently than Amazon. Adding
credentials to his request eliminated the error message.
The underlying AWS library assumes an Amazon back-end which does not
require the date header for anonymous requests and without credentials it
assumes it is an anonymous request. Though it doesn't require them, this
backend can use Amazon credentials and doing so fixes the problem.
I'll look into more testing with other non-Amazon S3 compatible platforms
and into finer grained control over the library behaviors. But, it seems
reasonable that the Amazon library might not accommodate deviation from the
anticipated AWS S3 behaviors.
On Fri, Feb 19, 2016 at 3:02 PM, Joseph E. Gottman <jg...@proteuseng.com>
wrote:
> I am using an SSLContextService for my credentials.
>
>
> [image: Proteus Technologies Outsmart Logo]
> <http://www.proteus-technologies.com/> *Joe Gottman*
> *Senior Member of Technical Staff*
> jgottman@proteuseng.com <jg...@proteuseng.com>
> 133 National Business Pkwy, Ste 150
> Annapolis Junction, MD 20701
> (Office) 301.377.7144
> *www.proteus-technologies.com* <http://www.proteus-technologies.com/>
> *TheBlend.ProteusEng.com* <http://theblend.proteuseng.com/>* (Digital
> magazine)* This electronic message and any files transmitted with it
> contain information which may be privileged and/or proprietary. The
> information is intended for use solely by the intended recipient(s). If you
> are not the intended recipient, be aware that any disclosure, copying,
> distribution or use of this information is prohibited. If you have received
> this electronic message in error, please advise the sender by reply email
> or by telephone (443.539.3400) and delete the message.
> ------------------------------
> *From:* Joe Skora <js...@gmail.com>
> *Sent:* Friday, February 19, 2016 2:56 PM
> *To:* users@nifi.apache.org
> *Subject:* Re: Problem with PutS3Object processor
>
> Joseph,
>
> I ran into this same problem last week when I forgot to provide
> credentials to an S3 compatible endpoint. AWS S3 requires the date header
> when an authorization header is provided, so the underlying Amazon library
> provides it automatically if authorization is used by the processor, but if
> it thinks the request is anonymous it leaves off the date header.
>
> Does your endpoint require authentication Amazon (or Amazon-like) Access
> Key and Secret Key credentials? If not, can you try providing credentials
> and see if that helps?
>
> Regards,
> Joe Skora
>
> On Fri, Feb 19, 2016 at 12:56 PM, Joseph E. Gottman <
> jgottman@proteuseng.com> wrote:
>
>> I am trying to use a PutS3Object processor with the "Endpoint Override
>> URL" pointing to a custom endpoint. I keep failing with the error message
>> "You must specify a date for this operation". I am using NiFi version
>> 0.4.1 with Java 8 and Centos 6.7. I suspect this might have something to
>> do with bug #1025, but according to your notes it was fixed for version
>> 0.4.0.
>>
>>
>>
>>
>> [image: Proteus Technologies Outsmart Logo]
>> <http://www.proteus-technologies.com/> *Joe Gottman*
>> *Senior Member of Technical Staff*
>> jgottman@proteuseng.com <jg...@proteuseng.com>
>> 133 National Business Pkwy, Ste 150
>> Annapolis Junction, MD 20701
>> (Office) 301.377.7144
>> *www.proteus-technologies.com* <http://www.proteus-technologies.com/>
>> *TheBlend.ProteusEng.com* <http://theblend.proteuseng.com/>* (Digital
>> magazine)* This electronic message and any files transmitted with it
>> contain information which may be privileged and/or proprietary. The
>> information is intended for use solely by the intended recipient(s). If you
>> are not the intended recipient, be aware that any disclosure, copying,
>> distribution or use of this information is prohibited. If you have received
>> this electronic message in error, please advise the sender by reply email
>> or by telephone (443.539.3400) and delete the message.
>>
>
>
Re: Problem with PutS3Object processor
Posted by "Joseph E. Gottman" <jg...@proteuseng.com>.
I am using an SSLContextService for my credentials.
[Proteus Technologies Outsmart Logo]<http://www.proteus-technologies.com/> Joe Gottman
Senior Member of Technical Staff
jgottman@proteuseng.com<ma...@proteuseng.com>
133 National Business Pkwy, Ste 150
Annapolis Junction, MD 20701
(Office) 301.377.7144
www.proteus-technologies.com<http://www.proteus-technologies.com/>
TheBlend.ProteusEng.com<http://theblend.proteuseng.com/> (Digital magazine)
This electronic message and any files transmitted with it contain information which may be privileged and/or proprietary. The information is intended for use solely by the intended recipient(s). If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of this information is prohibited. If you have received this electronic message in error, please advise the sender by reply email or by telephone (443.539.3400) and delete the message.
________________________________
From: Joe Skora <js...@gmail.com>
Sent: Friday, February 19, 2016 2:56 PM
To: users@nifi.apache.org
Subject: Re: Problem with PutS3Object processor
Joseph,
I ran into this same problem last week when I forgot to provide credentials to an S3 compatible endpoint. AWS S3 requires the date header when an authorization header is provided, so the underlying Amazon library provides it automatically if authorization is used by the processor, but if it thinks the request is anonymous it leaves off the date header.
Does your endpoint require authentication Amazon (or Amazon-like) Access Key and Secret Key credentials? If not, can you try providing credentials and see if that helps?
Regards,
Joe Skora
On Fri, Feb 19, 2016 at 12:56 PM, Joseph E. Gottman <jg...@proteuseng.com>> wrote:
I am trying to use a PutS3Object processor with the "Endpoint Override URL" pointing to a custom endpoint. I keep failing with the error message "You must specify a date for this operation". I am using NiFi version 0.4.1 with Java 8 and Centos 6.7. I suspect this might have something to do with bug #1025, but according to your notes it was fixed for version 0.4.0.
[Proteus Technologies Outsmart Logo]<http://www.proteus-technologies.com/> Joe Gottman
Senior Member of Technical Staff
jgottman@proteuseng.com<ma...@proteuseng.com>
133 National Business Pkwy, Ste 150
Annapolis Junction, MD 20701
(Office) 301.377.7144<tel:301.377.7144>
www.proteus-technologies.com<http://www.proteus-technologies.com/>
TheBlend.ProteusEng.com<http://theblend.proteuseng.com/> (Digital magazine)
This electronic message and any files transmitted with it contain information which may be privileged and/or proprietary. The information is intended for use solely by the intended recipient(s). If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of this information is prohibited. If you have received this electronic message in error, please advise the sender by reply email or by telephone (443.539.3400<tel:%28443.539.3400>) and delete the message.
Re: Problem with PutS3Object processor
Posted by Joe Skora <js...@gmail.com>.
Joseph,
I ran into this same problem last week when I forgot to provide credentials
to an S3 compatible endpoint. AWS S3 requires the date header when an
authorization header is provided, so the underlying Amazon library provides
it automatically if authorization is used by the processor, but if it
thinks the request is anonymous it leaves off the date header.
Does your endpoint require authentication Amazon (or Amazon-like) Access
Key and Secret Key credentials? If not, can you try providing credentials
and see if that helps?
Regards,
Joe Skora
On Fri, Feb 19, 2016 at 12:56 PM, Joseph E. Gottman <jgottman@proteuseng.com
> wrote:
> I am trying to use a PutS3Object processor with the "Endpoint Override
> URL" pointing to a custom endpoint. I keep failing with the error message
> "You must specify a date for this operation". I am using NiFi version
> 0.4.1 with Java 8 and Centos 6.7. I suspect this might have something to
> do with bug #1025, but according to your notes it was fixed for version
> 0.4.0.
>
>
>
>
> [image: Proteus Technologies Outsmart Logo]
> <http://www.proteus-technologies.com/> *Joe Gottman*
> *Senior Member of Technical Staff*
> jgottman@proteuseng.com <jg...@proteuseng.com>
> 133 National Business Pkwy, Ste 150
> Annapolis Junction, MD 20701
> (Office) 301.377.7144
> *www.proteus-technologies.com* <http://www.proteus-technologies.com/>
> *TheBlend.ProteusEng.com* <http://theblend.proteuseng.com/>* (Digital
> magazine)* This electronic message and any files transmitted with it
> contain information which may be privileged and/or proprietary. The
> information is intended for use solely by the intended recipient(s). If you
> are not the intended recipient, be aware that any disclosure, copying,
> distribution or use of this information is prohibited. If you have received
> this electronic message in error, please advise the sender by reply email
> or by telephone (443.539.3400) and delete the message.
>