S3 as Secondary Storage

[Will Stevens <sw...@apache.org>]

Hey All,
If anyone is using S3 as Secondary Storage in production (or even testing
at this point), please respond to this thread.

We have been using Swift in production for the last couple years and I
think we have worked out most of the kinks.  Not all of our fixes have been
pushed upstream yet because the Swift and S3 integrations are very
intertwined and we did not want to break the S3 implementation with our
Swift fixes.

We would like to get a sense of the stability of the S3 implementation to
see if it is also a good option as an object based secondary storage.

Thanks,

*Will Stevens*

Re: S3 as Secondary Storage

[Will Stevens <sw...@apache.org>]

Thanks Wido.  Yes, we have the same 'outdated' SDK being used issue for
Swift as well.  It is embedded in the code and it is ancient.  I have not
had a chance to get in there and update to a recent library.

We upgraded our setup from 4.4 -> 4.7 and we are in the process of
validating 4.10 as our next release base.

We have also found the 'upload to object store, then download again right
away' to be slow and very inefficient.  We have considered reimplementing
that logic, but we have not gotten to it yet.

I will be interested to hear how things go with 4.9.

Thanks for the feedback Wido.  :)

*Will Stevens*



On Mon, Mar 27, 2017 at 10:08 AM, Wido den Hollander <wi...@widodh.nl> wrote:

>
> > Op 27 maart 2017 om 14:09 schreef Will Stevens <sw...@apache.org>:
> >
> >
> > Hey All,
> > If anyone is using S3 as Secondary Storage in production (or even testing
> > at this point), please respond to this thread.
> >
>
> We are. Using Ceph's RADOS Gateway (Hammer, 0.94.11) as the gateway with
> CloudStack 4.5 (Yes... ;( ).
>
> > We have been using Swift in production for the last couple years and I
> > think we have worked out most of the kinks.  Not all of our fixes have
> been
> > pushed upstream yet because the Swift and S3 integrations are very
> > intertwined and we did not want to break the S3 implementation with our
> > Swift fixes.
> >
> > We would like to get a sense of the stability of the S3 implementation to
> > see if it is also a good option as an object based secondary storage.
> >
>
> It is, but our experiences are mainly with 4.5. It still uses a ancient
> version of the AWS S3 SDK and that causes most of our troubles.
>
> In general we however see the ACS copies a lot of Objects to the S3 store
> and downloads them again afterwards. Not always the best thing. I wouldn't
> use Amazon S3 directly as that might cause a lot of data transfer.
>
> Should be running 4.9 within 4 weeks, can report on it a lot better then.
>
> Wido
>
> > Thanks,
> >
> > *Will Stevens*
>

Re: S3 as Secondary Storage

[Wido den Hollander <wi...@widodh.nl>]

> Op 27 maart 2017 om 14:09 schreef Will Stevens <sw...@apache.org>:
> 
> 
> Hey All,
> If anyone is using S3 as Secondary Storage in production (or even testing
> at this point), please respond to this thread.
> 

We are. Using Ceph's RADOS Gateway (Hammer, 0.94.11) as the gateway with CloudStack 4.5 (Yes... ;( ).

> We have been using Swift in production for the last couple years and I
> think we have worked out most of the kinks.  Not all of our fixes have been
> pushed upstream yet because the Swift and S3 integrations are very
> intertwined and we did not want to break the S3 implementation with our
> Swift fixes.
> 
> We would like to get a sense of the stability of the S3 implementation to
> see if it is also a good option as an object based secondary storage.
> 

It is, but our experiences are mainly with 4.5. It still uses a ancient version of the AWS S3 SDK and that causes most of our troubles.

In general we however see the ACS copies a lot of Objects to the S3 store and downloads them again afterwards. Not always the best thing. I wouldn't use Amazon S3 directly as that might cause a lot of data transfer.

Should be running 4.9 within 4 weeks, can report on it a lot better then.

Wido

> Thanks,
> 
> *Will Stevens*