You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Wido den Hollander <wi...@widodh.nl> on 2015/07/17 10:53:29 UTC

[BLOCKER][4.6] New AWS SDK in master breaks S3 uploads

Hi,

Issue: https://issues.apache.org/jira/browse/CLOUDSTACK-8640

I encountered this by using the AWS SDK under 4.5 (homebrew).

4.5 uses the 1.3.22 SDK, but master uses 1.9.8 and that breaks S3 uploads.

AWS SDK 1.3.22 is old, very old, but 1.9.8 is also rather old.

The new SDK throws different Event Types and we can't simply fix a
"else" statement since I tried and it failed.

The S3TemplateDownloader was written in 2013, but imho it is a poor
implementation which is rather buggy.

It will need some work to get it working with the new AWS SDK however.

Amazon in the meantime also started to switch to the new V4 Signature
for Authorization, but that isn't supported yet by projects like Ceph's
RADOS Gateway and probably by other S3 providers.

So this seems like a blocker for 4.6, since we would ship a broken S3
implementation we know about.

We can revert to the 1.3.22 AWS SDK, but I don't know the reasoning
behind the switch to the 1.9.8 SDK.

This was done in fac7bfc5d503aa25a82a684f7ec545197d255fb2 by Rohit, but
I'm not sure if it was properly tested.

The best route would be to fix the S3TemplateDownloader, I agree, but
that would take a lot more time.

Wido

Re: [BLOCKER][4.6] New AWS SDK in master breaks S3 uploads

Posted by Wido den Hollander <wi...@widodh.nl>.

On 17-07-15 16:50, sebgoa wrote:
> 
> On Jul 17, 2015, at 10:53 AM, Wido den Hollander <wi...@widodh.nl> wrote:
> 
>> Hi,
>>
>> Issue: https://issues.apache.org/jira/browse/CLOUDSTACK-8640
>>
>> I encountered this by using the AWS SDK under 4.5 (homebrew).
>>
>> 4.5 uses the 1.3.22 SDK, but master uses 1.9.8 and that breaks S3 uploads.
>>
>> AWS SDK 1.3.22 is old, very old, but 1.9.8 is also rather old.
>>
>> The new SDK throws different Event Types and we can't simply fix a
>> "else" statement since I tried and it failed.
>>
>> The S3TemplateDownloader was written in 2013, but imho it is a poor
>> implementation which is rather buggy.
>>
>> It will need some work to get it working with the new AWS SDK however.
>>
>> Amazon in the meantime also started to switch to the new V4 Signature
>> for Authorization, but that isn't supported yet by projects like Ceph's
>> RADOS Gateway and probably by other S3 providers.
>>
>> So this seems like a blocker for 4.6, since we would ship a broken S3
>> implementation we know about.
>>
>> We can revert to the 1.3.22 AWS SDK, but I don't know the reasoning
>> behind the switch to the 1.9.8 SDK.
>>
>> This was done in fac7bfc5d503aa25a82a684f7ec545197d255fb2 by Rohit, but
>> I'm not sure if it was properly tested.
>>
>> The best route would be to fix the S3TemplateDownloader, I agree, but
>> that would take a lot more time.
> 
> I we talked about it for the roadmap, and Francois Gaudreault is also interested to get this better supported.
> 
> If you can edit the roadmap with the bug ID that would help, at least to track it down.
> 
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Roadmap
> 

Done! Added it.

Wido

> thanks
> 
>>
>> Wido
>

Re: [BLOCKER][4.6] New AWS SDK in master breaks S3 uploads

Posted by sebgoa <ru...@gmail.com>.
On Jul 17, 2015, at 10:53 AM, Wido den Hollander <wi...@widodh.nl> wrote:

> Hi,
> 
> Issue: https://issues.apache.org/jira/browse/CLOUDSTACK-8640
> 
> I encountered this by using the AWS SDK under 4.5 (homebrew).
> 
> 4.5 uses the 1.3.22 SDK, but master uses 1.9.8 and that breaks S3 uploads.
> 
> AWS SDK 1.3.22 is old, very old, but 1.9.8 is also rather old.
> 
> The new SDK throws different Event Types and we can't simply fix a
> "else" statement since I tried and it failed.
> 
> The S3TemplateDownloader was written in 2013, but imho it is a poor
> implementation which is rather buggy.
> 
> It will need some work to get it working with the new AWS SDK however.
> 
> Amazon in the meantime also started to switch to the new V4 Signature
> for Authorization, but that isn't supported yet by projects like Ceph's
> RADOS Gateway and probably by other S3 providers.
> 
> So this seems like a blocker for 4.6, since we would ship a broken S3
> implementation we know about.
> 
> We can revert to the 1.3.22 AWS SDK, but I don't know the reasoning
> behind the switch to the 1.9.8 SDK.
> 
> This was done in fac7bfc5d503aa25a82a684f7ec545197d255fb2 by Rohit, but
> I'm not sure if it was properly tested.
> 
> The best route would be to fix the S3TemplateDownloader, I agree, but
> that would take a lot more time.

I we talked about it for the roadmap, and Francois Gaudreault is also interested to get this better supported.

If you can edit the roadmap with the bug ID that would help, at least to track it down.

https://cwiki.apache.org/confluence/display/CLOUDSTACK/Roadmap

thanks

> 
> Wido