You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Sam Robertson <Sa...@citrix.com> on 2012/06/13 23:48:58 UTC
Discuss EC2/S3 Test Framework for CloudStack
I know this has come up some in recent weeks on this list and even in discussion locally. We are trying to extend CloudStack's existing Python Testing Framework (http://wiki.cloudstack.org/display/QA/Python+testing+framework) to test the new EC2 and S3 api's recently added (http://wiki.cloudstack.org/display/RelOps/EC2+API+support+in+CloudStack and http://wiki.cloudstack.org/display/RelOps/S3+API+in+CloudStack).
As an initial cut, we are exploring using boto (http://docs.pythonboto.org/en/latest/index.html) and it's corresponding test suite as a foundation (https://github.com/boto/boto/tree/develop/tests) for our own test scripts.
Boto does not support SOAP api and we need to test both REST and SOAP api's in CloudStack. My current thinking is to use the AWS tools provided by Amazon (http://aws.amazon.com/developertools/Amazon-EC2/351) and call them directly from our test scripts, which will satisfy our SOAP requirement, but then we have two test scripts, one that uses boto and one that uses SOAP, which will get messy/complex.
The goal here is to test these the new api's in a fashion that can be run without too much interaction by the tester and finally the entire framework needs to be flexible enough to work in a committers environment without too much 'setting up' and configuration. I'm reaching out to the community before moving to much further on this in the hopes that someone might have other ideas and/or even assist us building this out.
Sam
Re: Discuss EC2/S3 Test Framework for CloudStack
Posted by Sam Robertson <Sa...@citrix.com>.
On 6/13/12 3:40 PM, "Adrian Cole" <ad...@gmail.com> wrote:
>Boto is a cool python library that focuses on AWS specifically. Sounds
>like folks already built something so maybe let them have their fun? :)
At this stage, we need something running, and this is what we have at the
moment, so agreed.
>
>When you want something for the JVM, happy to help with a jclouds option.
>Seems straightforward to choose JVM tooling as you can integrate them
>directly in the CloudBridge tests, also written in java. Anyways, to each
>their own.
Did a bit of a deep dive on this the past couple of days, and I agree,
jclouds is definitely the way forward, in addition to our existing work
with boto. Multiple tool sets would be useful. Boto is so widely used
that it's an obvious checkbox, and jclouds supports both the ec2/s3 api's
as well as our native cloudstack api. Beyond just ec2/s3, it would be
useful to at a minimum do sanity tests with existing 3rd party suites like
jclouds!
>
>Like I mentioned before, jclouds already has code to deal with and test
>EC2
>dialects like Eucalyptus. Moreover, we are currently finding
>compatibility
>issues with CloudBridge. We will likely end up with a separate
>cloudbridge
>dialect osgi package/jar like we do for euca and nova.
Until this past weekend (and not in a released package yet) CloudBridge
and now our CloudStack EC2 api wouldn't get past authentication. Since
fixing that, I've discovered a variety of issues with the REST api, and
will be committing fixes for those in the next few days. Will post more
as that gets closer to finish.
>
>Will keep you posted on that end.
>-A
Looking forward to it!
Sam
>On Jun 13, 2012 6:27 PM, "Sam Robertson" <Sa...@citrix.com> wrote:
>
>> Thanks David!
>>
>> On 6/13/12 2:55 PM, "David Nalley" <da...@gnsa.us> wrote:
>>
>> >On Wed, Jun 13, 2012 at 5:48 PM, Sam Robertson
>><Sa...@citrix.com>
>> >wrote:
>> >> I know this has come up some in recent weeks on this list and even in
>> >>discussion locally. We are trying to extend CloudStack's existing
>> >>Python Testing Framework
>> >>(http://wiki.cloudstack.org/display/QA/Python+testing+framework) to
>>test
>> >>the new EC2 and S3 api's recently added
>>
>>>>(http://wiki.cloudstack.org/display/RelOps/EC2+API+support+in+CloudStac
>>>>k
>> >>and http://wiki.cloudstack.org/display/RelOps/S3+API+in+CloudStack).
>> >>
>> >> As an initial cut, we are exploring using boto
>> >>(http://docs.pythonboto.org/en/latest/index.html) and it's
>>corresponding
>> >>test suite as a foundation
>> >>(https://github.com/boto/boto/tree/develop/tests) for our own test
>> >>scripts.
>> >
>> >Interesting - what about boto is more appealing than eutester or
>>jclouds?
>>
>> Eutester appears to be a 'beginning' but not as far along as the boto
>>test
>> suite. It also uses boto, however it might be the better solution for
>> testing since it's purpose is a test suite, but still doesn't solve our
>> SOAP problem.
>>
>>
>> >
>> >>
>> >> Boto does not support SOAP api and we need to test both REST and SOAP
>> >>api's in CloudStack. My current thinking is to use the AWS tools
>> >>provided by Amazon
>>(http://aws.amazon.com/developertools/Amazon-EC2/351)
>> >>and call them directly from our test scripts, which will satisfy our
>> >>SOAP requirement, but then we have two test scripts, one that uses
>>boto
>> >>and one that uses SOAP, which will get messy/complex.
>> >
>> >The AWS tools have several problems in my mind.
>> >1. They aren't open source
>> >2. Even more troubling their license actually actively limits you to
>> >only using it for/with Amazon's service offering.
>> >
>> >Could we use euca2ools in it's place?
>>
>> Euca2ools sits on top of boto, but again might be a better approach
>> overall.
>>
>> The licensing issue with the AWS tools is a big problem. Most customers
>> that use CloudBridge and the EC2 support in CloudStack use the AWS tools
>> and are violating the license of those tools. One could argue that we
>> shouldn't even expose the SOAP API.
>>
>> In general, boto appears to be the magic sauce that most tools are being
>> built on top of, besides amazon's own tools.
>>
>>
>> >
>> >--David
>>
>> Sam
>>
>>
Re: Discuss EC2/S3 Test Framework for CloudStack
Posted by Adrian Cole <ad...@gmail.com>.
Boto is a cool python library that focuses on AWS specifically. Sounds
like folks already built something so maybe let them have their fun? :)
When you want something for the JVM, happy to help with a jclouds option.
Seems straightforward to choose JVM tooling as you can integrate them
directly in the CloudBridge tests, also written in java. Anyways, to each
their own.
Like I mentioned before, jclouds already has code to deal with and test EC2
dialects like Eucalyptus. Moreover, we are currently finding compatibility
issues with CloudBridge. We will likely end up with a separate cloudbridge
dialect osgi package/jar like we do for euca and nova.
Will keep you posted on that end.
-A
On Jun 13, 2012 6:27 PM, "Sam Robertson" <Sa...@citrix.com> wrote:
> Thanks David!
>
> On 6/13/12 2:55 PM, "David Nalley" <da...@gnsa.us> wrote:
>
> >On Wed, Jun 13, 2012 at 5:48 PM, Sam Robertson <Sa...@citrix.com>
> >wrote:
> >> I know this has come up some in recent weeks on this list and even in
> >>discussion locally. We are trying to extend CloudStack's existing
> >>Python Testing Framework
> >>(http://wiki.cloudstack.org/display/QA/Python+testing+framework) to test
> >>the new EC2 and S3 api's recently added
> >>(http://wiki.cloudstack.org/display/RelOps/EC2+API+support+in+CloudStack
> >>and http://wiki.cloudstack.org/display/RelOps/S3+API+in+CloudStack).
> >>
> >> As an initial cut, we are exploring using boto
> >>(http://docs.pythonboto.org/en/latest/index.html) and it's corresponding
> >>test suite as a foundation
> >>(https://github.com/boto/boto/tree/develop/tests) for our own test
> >>scripts.
> >
> >Interesting - what about boto is more appealing than eutester or jclouds?
>
> Eutester appears to be a 'beginning' but not as far along as the boto test
> suite. It also uses boto, however it might be the better solution for
> testing since it's purpose is a test suite, but still doesn't solve our
> SOAP problem.
>
>
> >
> >>
> >> Boto does not support SOAP api and we need to test both REST and SOAP
> >>api's in CloudStack. My current thinking is to use the AWS tools
> >>provided by Amazon (http://aws.amazon.com/developertools/Amazon-EC2/351)
> >>and call them directly from our test scripts, which will satisfy our
> >>SOAP requirement, but then we have two test scripts, one that uses boto
> >>and one that uses SOAP, which will get messy/complex.
> >
> >The AWS tools have several problems in my mind.
> >1. They aren't open source
> >2. Even more troubling their license actually actively limits you to
> >only using it for/with Amazon's service offering.
> >
> >Could we use euca2ools in it's place?
>
> Euca2ools sits on top of boto, but again might be a better approach
> overall.
>
> The licensing issue with the AWS tools is a big problem. Most customers
> that use CloudBridge and the EC2 support in CloudStack use the AWS tools
> and are violating the license of those tools. One could argue that we
> shouldn't even expose the SOAP API.
>
> In general, boto appears to be the magic sauce that most tools are being
> built on top of, besides amazon's own tools.
>
>
> >
> >--David
>
> Sam
>
>
Re: Discuss EC2/S3 Test Framework for CloudStack
Posted by Sam Robertson <Sa...@citrix.com>.
On 6/13/12 3:52 PM, "David Nalley" <da...@gnsa.us> wrote:
>On Wed, Jun 13, 2012 at 6:26 PM, Sam Robertson <Sa...@citrix.com>
>wrote:
>
>> The licensing issue with the AWS tools is a big problem. Most customers
>> that use CloudBridge and the EC2 support in CloudStack use the AWS tools
>> and are violating the license of those tools. One could argue that we
>> shouldn't even expose the SOAP API.
>
>That's a good point - perhaps we shouldn't. Isn't the SOAP API
>considered to be deprecated - or at least out of favor with AWS?
>
>AWS seems to be pushing the query interface pretty heavily as well FWIU.
Not sure if it's deprecated, especially considering the CLI tools are
actually built against the SOAP API.
Re: Discuss EC2/S3 Test Framework for CloudStack
Posted by David Nalley <da...@gnsa.us>.
On Wed, Jun 13, 2012 at 6:26 PM, Sam Robertson <Sa...@citrix.com> wrote:
> The licensing issue with the AWS tools is a big problem. Most customers
> that use CloudBridge and the EC2 support in CloudStack use the AWS tools
> and are violating the license of those tools. One could argue that we
> shouldn't even expose the SOAP API.
That's a good point - perhaps we shouldn't. Isn't the SOAP API
considered to be deprecated - or at least out of favor with AWS?
AWS seems to be pushing the query interface pretty heavily as well FWIU.
Re: Discuss EC2/S3 Test Framework for CloudStack
Posted by Sam Robertson <Sa...@citrix.com>.
Thanks David!
On 6/13/12 2:55 PM, "David Nalley" <da...@gnsa.us> wrote:
>On Wed, Jun 13, 2012 at 5:48 PM, Sam Robertson <Sa...@citrix.com>
>wrote:
>> I know this has come up some in recent weeks on this list and even in
>>discussion locally. We are trying to extend CloudStack's existing
>>Python Testing Framework
>>(http://wiki.cloudstack.org/display/QA/Python+testing+framework) to test
>>the new EC2 and S3 api's recently added
>>(http://wiki.cloudstack.org/display/RelOps/EC2+API+support+in+CloudStack
>>and http://wiki.cloudstack.org/display/RelOps/S3+API+in+CloudStack).
>>
>> As an initial cut, we are exploring using boto
>>(http://docs.pythonboto.org/en/latest/index.html) and it's corresponding
>>test suite as a foundation
>>(https://github.com/boto/boto/tree/develop/tests) for our own test
>>scripts.
>
>Interesting - what about boto is more appealing than eutester or jclouds?
Eutester appears to be a 'beginning' but not as far along as the boto test
suite. It also uses boto, however it might be the better solution for
testing since it's purpose is a test suite, but still doesn't solve our
SOAP problem.
>
>>
>> Boto does not support SOAP api and we need to test both REST and SOAP
>>api's in CloudStack. My current thinking is to use the AWS tools
>>provided by Amazon (http://aws.amazon.com/developertools/Amazon-EC2/351)
>>and call them directly from our test scripts, which will satisfy our
>>SOAP requirement, but then we have two test scripts, one that uses boto
>>and one that uses SOAP, which will get messy/complex.
>
>The AWS tools have several problems in my mind.
>1. They aren't open source
>2. Even more troubling their license actually actively limits you to
>only using it for/with Amazon's service offering.
>
>Could we use euca2ools in it's place?
Euca2ools sits on top of boto, but again might be a better approach
overall.
The licensing issue with the AWS tools is a big problem. Most customers
that use CloudBridge and the EC2 support in CloudStack use the AWS tools
and are violating the license of those tools. One could argue that we
shouldn't even expose the SOAP API.
In general, boto appears to be the magic sauce that most tools are being
built on top of, besides amazon's own tools.
>
>--David
Sam
Re: Discuss EC2/S3 Test Framework for CloudStack
Posted by David Nalley <da...@gnsa.us>.
On Wed, Jun 13, 2012 at 5:48 PM, Sam Robertson <Sa...@citrix.com> wrote:
> I know this has come up some in recent weeks on this list and even in discussion locally. We are trying to extend CloudStack's existing Python Testing Framework (http://wiki.cloudstack.org/display/QA/Python+testing+framework) to test the new EC2 and S3 api's recently added (http://wiki.cloudstack.org/display/RelOps/EC2+API+support+in+CloudStack and http://wiki.cloudstack.org/display/RelOps/S3+API+in+CloudStack).
>
> As an initial cut, we are exploring using boto (http://docs.pythonboto.org/en/latest/index.html) and it's corresponding test suite as a foundation (https://github.com/boto/boto/tree/develop/tests) for our own test scripts.
Interesting - what about boto is more appealing than eutester or jclouds?
>
> Boto does not support SOAP api and we need to test both REST and SOAP api's in CloudStack. My current thinking is to use the AWS tools provided by Amazon (http://aws.amazon.com/developertools/Amazon-EC2/351) and call them directly from our test scripts, which will satisfy our SOAP requirement, but then we have two test scripts, one that uses boto and one that uses SOAP, which will get messy/complex.
The AWS tools have several problems in my mind.
1. They aren't open source
2. Even more troubling their license actually actively limits you to
only using it for/with Amazon's service offering.
Could we use euca2ools in it's place?
--David