You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Rohit Yadav <ro...@shapeblue.com> on 2014/07/31 14:12:25 UTC
How to speed up testing using BVT/smoke tests with Simulator
Hi,
Santosh put together a good wiki page on how to validate local changes using our Python/marvin based build verification tests (path: test/integration/smoke): https://cwiki.apache.org/confluence/display/CLOUDSTACK/Validating+check-ins+for+your+local+changes%2C+using+Simulator
I’ve a mini PC for this and using CloudStack 4.4.0 to build/test CloudStack 4.4/master branch on it in a VM. Some of us are also exploring free/cheap CI services such as Travis, CloudBees etc. which can be used by developers to test their check-ins. If anyone of you have tried something like this please share.
This is how I build CloudStack for validating with simulator:
mvn -Pdeveloper -Dsimulator clean install
mvn -Pdeveloper -pl developer -Ddeploydb
mvn -Pdeveloper -pl developer -Ddeploydb-simulator
mvn -pl client jetty:run -Dsimulator
And finally run smoke tests (BVT):
nosetests --with-marvin --marvin-config=setup/dev/advanced.cfg --with-xunit --xunit-file=/tmp/bvt_selfservice_cases.xml -a tags=advanced,required_hardware=false -w test/integration/smoke --hypervisor=simulator
It currently took 50 mins on my setup. How can we speed it up, say by reducing global variable timeout settings etc? Should we reduce timeouts etc. in deploydb-simulator specific sql files?
Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.yadav@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab
Find out more about ShapeBlue and our range of CloudStack related services
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: How to speed up testing using BVT/smoke tests with Simulator
Posted by Rohit Yadav <ro...@shapeblue.com>.
Hi Ian,
Thanks for catching that, I’ve fixed it on master and 4.4 now using insert on duplicate update syntax.
b157104563d2d30ede7bfabe2df8cf42d3dadb1d master
9315321dacc877e67279b3b59b6b2e08d125ddbe 4.4
On 04-Aug-2014, at 9:31 pm, Ian Duffy <ia...@ianduffy.ie> wrote:
> Rohit,
>
> Those changes cause conflicts when attempting to deploy the devcloud.sql
> file as they specify the same primary keys.
>
>
> On 1 August 2014 15:56, Rohit Yadav <ro...@shapeblue.com> wrote:
>
>> I’ve added config in developer/developer-prefill.sql to have three new
>> config params commonly used by devs and to speed up account cleanup (60s
>> now).
>>
>> 137655d4e11cb490391d0f4982c52262bbb7427c master
>> a68d8986829c462d655bf59a0cbcc0d4e8b1cc3e 4.4
>>
>> Regards.
>>
>> On 01-Aug-2014, at 7:27 am, Santhosh Edukulla <
>> santhosh.edukulla@citrix.com> wrote:
>>
>>> 1. yes, test cases currently run in parallel in ci environment, using
>> parallel nose commands.
>>>
>>> 2. There were few hard codings for sleep, where we removed at some
>> places, still there could be few\many out there.
>>>
>>> 3. 1 sec sleep and poll check is too heavy i believe, for async. So,
>> worst a test case can add time of execution by 5 seconds more, compared to
>> succesful operation time.
>>>
>>> Santhosh
>>> ________________________________________
>>> From: Rohit Yadav [rohit.yadav@shapeblue.com]
>>> Sent: Thursday, July 31, 2014 7:15 PM
>>> To: dev@cloudstack.apache.org
>>> Subject: Re: How to speed up testing using BVT/smoke tests with Simulator
>>>
>>> Hi Edison,
>>>
>>> Thanks for the pointers! I’ll try them out and see if there is way to do
>> it on Travis/CloudBees as well and I hope other people will religiously
>> start using simulator/bvt (at least the basic ones) for their check-ins.
>>>
>>> Regards.
>>>
>>> On 01-Aug-2014, at 12:15 am, Edison Su <Ed...@citrix.com> wrote:
>>>
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: Rohit Yadav [mailto:rohit.yadav@shapeblue.com]
>>>>> Sent: Thursday, July 31, 2014 5:12 AM
>>>>> To: dev@cloudstack.apache.org
>>>>> Subject: How to speed up testing using BVT/smoke tests with Simulator
>>>>>
>>>>> Hi,
>>>>>
>>>>> Santosh put together a good wiki page on how to validate local changes
>> using
>>>>> our Python/marvin based build verification tests (path:
>>>>> test/integration/smoke):
>>>>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Validating+check
>>>>> -ins+for+your+local+changes%2C+using+Simulator
>>>>>
>>>>> I've a mini PC for this and using CloudStack 4.4.0 to build/test
>> CloudStack
>>>>> 4.4/master branch on it in a VM. Some of us are also exploring
>> free/cheap CI
>>>>> services such as Travis, CloudBees etc. which can be used by
>> developers to
>>>>> test their check-ins. If anyone of you have tried something like this
>> please
>>>>> share.
>>>> Today, I tried to build and test CloudStack on a super powerful machine
>> provided by Azure. Imaging, build & test on a 16 Cores, 120G machine, it
>> should be awesome, and most importantly, it's FREE. You can get a free MSDN
>> subscription fromhttps://
>> svn.apache.org/repos/private/committers/donated-licenses/msdn-subscription.html,
>> after that, you will get $150 credit monthly in Azure. For build &test
>> only, $150 is good enough.
>>>>
>>>>>
>>>>> This is how I build CloudStack for validating with simulator:
>>>>> mvn -Pdeveloper -Dsimulator clean install
>>>>> mvn -Pdeveloper -pl developer -Ddeploydb
>>>>> mvn -Pdeveloper -pl developer -Ddeploydb-simulator
>>>>> mvn -pl client jetty:run -Dsimulator
>>>>>
>>>>> And finally run smoke tests (BVT):
>>>>> nosetests --with-marvin --marvin-config=setup/dev/advanced.cfg --with-
>>>>> xunit --xunit-file=/tmp/bvt_selfservice_cases.xml -a
>>>>> tags=advanced,required_hardware=false -w test/integration/smoke --
>>>>> hypervisor=simulator
>>>>>
>>>>> It currently took 50 mins on my setup. How can we speed it up, say by
>>>>> reducing global variable timeout settings etc? Should we reduce
>> timeouts etc.
>>>>> in deploydb-simulator specific sql files?
>>>>
>>>> There several places we can improve the marvin test:
>>>> 1. queryAsyncJob waits 5 second for each call, can change to 1s.
>>>> 2. There are hardcoded sleep in test code, such base.py, search
>> time.sleep
>>>> 3. global configuration: account.cleanup.interval sets to 600s, so the
>> test suite will stop for 10 minutes after running for a while.
>>>> 3. most importantly, if we can run the test cases in parallel, then
>> speedup should be great. Does anybody try to run it in parallel before?
>>>>
>>>>>
>>>>> Regards,
>>>>> Rohit Yadav
>>>>> Software Architect, ShapeBlue
>>>>> M. +41 779015219 | rohit.yadav@shapeblue.com
>>>>> Blog: bhaisaab.org | Twitter: @_bhaisaab
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Find out more about ShapeBlue and our range of CloudStack related
>> services
>>>>>
>>>>> IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-
>>>>> build//>
>>>>> CSForge - rapid IaaS deployment
>>>>> framework<http://shapeblue.com/csforge/>
>>>>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>>>>> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-
>>>>> infrastructure-support/>
>>>>> CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-
>>>>> training/>
>>>>>
>>>>> This email and any attachments to it may be confidential and are
>> intended
>>>>> solely for the use of the individual to whom it is addressed. Any
>> views or
>>>>> opinions expressed are solely those of the author and do not
>> necessarily
>>>>> represent those of Shape Blue Ltd or related companies. If you are not
>> the
>>>>> intended recipient of this email, you must neither take any action
>> based
>>>>> upon its contents, nor copy or show it to anyone. Please contact the
>> sender if
>>>>> you believe you have received this email in error. Shape Blue Ltd is a
>>>>> company incorporated in England & Wales. ShapeBlue Services India LLP
>> is a
>>>>> company incorporated in India and is operated under license from Shape
>>>>> Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated
>> in
>>>>> Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA
>> Pty
>>>>> Ltd is a company registered by The Republic of South Africa and is
>> traded
>>>>> under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>>>
>>> Regards,
>>> Rohit Yadav
>>> Software Architect, ShapeBlue
>>> M. +41 779015219 | rohit.yadav@shapeblue.com
>>> Blog: bhaisaab.org | Twitter: @_bhaisaab
>>>
>>>
>>>
>>>
>>> Find out more about ShapeBlue and our range of CloudStack related
>> services
>>>
>>> IaaS Cloud Design & Build<
>> http://shapeblue.com/iaas-cloud-design-and-build//>
>>> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
>>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>>> CloudStack Infrastructure Support<
>> http://shapeblue.com/cloudstack-infrastructure-support/>
>>> CloudStack Bootcamp Training Courses<
>> http://shapeblue.com/cloudstack-training/>
>>>
>>> This email and any attachments to it may be confidential and are
>> intended solely for the use of the individual to whom it is addressed. Any
>> views or opinions expressed are solely those of the author and do not
>> necessarily represent those of Shape Blue Ltd or related companies. If you
>> are not the intended recipient of this email, you must neither take any
>> action based upon its contents, nor copy or show it to anyone. Please
>> contact the sender if you believe you have received this email in error.
>> Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue
>> Services India LLP is a company incorporated in India and is operated under
>> license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a
>> company incorporated in Brasil and is operated under license from Shape
>> Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of
>> South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is
>> a registered trademark.
>>
>> Regards,
>> Rohit Yadav
>> Software Architect, ShapeBlue
>> M. +41 779015219 | rohit.yadav@shapeblue.com
>> Blog: bhaisaab.org | Twitter: @_bhaisaab
>>
>>
>>
>>
>> Find out more about ShapeBlue and our range of CloudStack related services
>>
>> IaaS Cloud Design & Build<
>> http://shapeblue.com/iaas-cloud-design-and-build//>
>> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>> CloudStack Infrastructure Support<
>> http://shapeblue.com/cloudstack-infrastructure-support/>
>> CloudStack Bootcamp Training Courses<
>> http://shapeblue.com/cloudstack-training/>
>>
>> This email and any attachments to it may be confidential and are intended
>> solely for the use of the individual to whom it is addressed. Any views or
>> opinions expressed are solely those of the author and do not necessarily
>> represent those of Shape Blue Ltd or related companies. If you are not the
>> intended recipient of this email, you must neither take any action based
>> upon its contents, nor copy or show it to anyone. Please contact the sender
>> if you believe you have received this email in error. Shape Blue Ltd is a
>> company incorporated in England & Wales. ShapeBlue Services India LLP is a
>> company incorporated in India and is operated under license from Shape Blue
>> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
>> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is
>> a company registered by The Republic of South Africa and is traded under
>> license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>>
Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.yadav@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab
Find out more about ShapeBlue and our range of CloudStack related services
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: How to speed up testing using BVT/smoke tests with Simulator
Posted by Ian Duffy <ia...@ianduffy.ie>.
Rohit,
Those changes cause conflicts when attempting to deploy the devcloud.sql
file as they specify the same primary keys.
On 1 August 2014 15:56, Rohit Yadav <ro...@shapeblue.com> wrote:
> I’ve added config in developer/developer-prefill.sql to have three new
> config params commonly used by devs and to speed up account cleanup (60s
> now).
>
> 137655d4e11cb490391d0f4982c52262bbb7427c master
> a68d8986829c462d655bf59a0cbcc0d4e8b1cc3e 4.4
>
> Regards.
>
> On 01-Aug-2014, at 7:27 am, Santhosh Edukulla <
> santhosh.edukulla@citrix.com> wrote:
>
> > 1. yes, test cases currently run in parallel in ci environment, using
> parallel nose commands.
> >
> > 2. There were few hard codings for sleep, where we removed at some
> places, still there could be few\many out there.
> >
> > 3. 1 sec sleep and poll check is too heavy i believe, for async. So,
> worst a test case can add time of execution by 5 seconds more, compared to
> succesful operation time.
> >
> > Santhosh
> > ________________________________________
> > From: Rohit Yadav [rohit.yadav@shapeblue.com]
> > Sent: Thursday, July 31, 2014 7:15 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: How to speed up testing using BVT/smoke tests with Simulator
> >
> > Hi Edison,
> >
> > Thanks for the pointers! I’ll try them out and see if there is way to do
> it on Travis/CloudBees as well and I hope other people will religiously
> start using simulator/bvt (at least the basic ones) for their check-ins.
> >
> > Regards.
> >
> > On 01-Aug-2014, at 12:15 am, Edison Su <Ed...@citrix.com> wrote:
> >
> >>
> >>
> >>> -----Original Message-----
> >>> From: Rohit Yadav [mailto:rohit.yadav@shapeblue.com]
> >>> Sent: Thursday, July 31, 2014 5:12 AM
> >>> To: dev@cloudstack.apache.org
> >>> Subject: How to speed up testing using BVT/smoke tests with Simulator
> >>>
> >>> Hi,
> >>>
> >>> Santosh put together a good wiki page on how to validate local changes
> using
> >>> our Python/marvin based build verification tests (path:
> >>> test/integration/smoke):
> >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Validating+check
> >>> -ins+for+your+local+changes%2C+using+Simulator
> >>>
> >>> I've a mini PC for this and using CloudStack 4.4.0 to build/test
> CloudStack
> >>> 4.4/master branch on it in a VM. Some of us are also exploring
> free/cheap CI
> >>> services such as Travis, CloudBees etc. which can be used by
> developers to
> >>> test their check-ins. If anyone of you have tried something like this
> please
> >>> share.
> >> Today, I tried to build and test CloudStack on a super powerful machine
> provided by Azure. Imaging, build & test on a 16 Cores, 120G machine, it
> should be awesome, and most importantly, it's FREE. You can get a free MSDN
> subscription fromhttps://
> svn.apache.org/repos/private/committers/donated-licenses/msdn-subscription.html,
> after that, you will get $150 credit monthly in Azure. For build &test
> only, $150 is good enough.
> >>
> >>>
> >>> This is how I build CloudStack for validating with simulator:
> >>> mvn -Pdeveloper -Dsimulator clean install
> >>> mvn -Pdeveloper -pl developer -Ddeploydb
> >>> mvn -Pdeveloper -pl developer -Ddeploydb-simulator
> >>> mvn -pl client jetty:run -Dsimulator
> >>>
> >>> And finally run smoke tests (BVT):
> >>> nosetests --with-marvin --marvin-config=setup/dev/advanced.cfg --with-
> >>> xunit --xunit-file=/tmp/bvt_selfservice_cases.xml -a
> >>> tags=advanced,required_hardware=false -w test/integration/smoke --
> >>> hypervisor=simulator
> >>>
> >>> It currently took 50 mins on my setup. How can we speed it up, say by
> >>> reducing global variable timeout settings etc? Should we reduce
> timeouts etc.
> >>> in deploydb-simulator specific sql files?
> >>
> >> There several places we can improve the marvin test:
> >> 1. queryAsyncJob waits 5 second for each call, can change to 1s.
> >> 2. There are hardcoded sleep in test code, such base.py, search
> time.sleep
> >> 3. global configuration: account.cleanup.interval sets to 600s, so the
> test suite will stop for 10 minutes after running for a while.
> >> 3. most importantly, if we can run the test cases in parallel, then
> speedup should be great. Does anybody try to run it in parallel before?
> >>
> >>>
> >>> Regards,
> >>> Rohit Yadav
> >>> Software Architect, ShapeBlue
> >>> M. +41 779015219 | rohit.yadav@shapeblue.com
> >>> Blog: bhaisaab.org | Twitter: @_bhaisaab
> >>>
> >>>
> >>>
> >>>
> >>> Find out more about ShapeBlue and our range of CloudStack related
> services
> >>>
> >>> IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-
> >>> build//>
> >>> CSForge - rapid IaaS deployment
> >>> framework<http://shapeblue.com/csforge/>
> >>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> >>> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-
> >>> infrastructure-support/>
> >>> CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-
> >>> training/>
> >>>
> >>> This email and any attachments to it may be confidential and are
> intended
> >>> solely for the use of the individual to whom it is addressed. Any
> views or
> >>> opinions expressed are solely those of the author and do not
> necessarily
> >>> represent those of Shape Blue Ltd or related companies. If you are not
> the
> >>> intended recipient of this email, you must neither take any action
> based
> >>> upon its contents, nor copy or show it to anyone. Please contact the
> sender if
> >>> you believe you have received this email in error. Shape Blue Ltd is a
> >>> company incorporated in England & Wales. ShapeBlue Services India LLP
> is a
> >>> company incorporated in India and is operated under license from Shape
> >>> Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated
> in
> >>> Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA
> Pty
> >>> Ltd is a company registered by The Republic of South Africa and is
> traded
> >>> under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
> >
> > Regards,
> > Rohit Yadav
> > Software Architect, ShapeBlue
> > M. +41 779015219 | rohit.yadav@shapeblue.com
> > Blog: bhaisaab.org | Twitter: @_bhaisaab
> >
> >
> >
> >
> > Find out more about ShapeBlue and our range of CloudStack related
> services
> >
> > IaaS Cloud Design & Build<
> http://shapeblue.com/iaas-cloud-design-and-build//>
> > CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
> > CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> > CloudStack Infrastructure Support<
> http://shapeblue.com/cloudstack-infrastructure-support/>
> > CloudStack Bootcamp Training Courses<
> http://shapeblue.com/cloudstack-training/>
> >
> > This email and any attachments to it may be confidential and are
> intended solely for the use of the individual to whom it is addressed. Any
> views or opinions expressed are solely those of the author and do not
> necessarily represent those of Shape Blue Ltd or related companies. If you
> are not the intended recipient of this email, you must neither take any
> action based upon its contents, nor copy or show it to anyone. Please
> contact the sender if you believe you have received this email in error.
> Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue
> Services India LLP is a company incorporated in India and is operated under
> license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a
> company incorporated in Brasil and is operated under license from Shape
> Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of
> South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is
> a registered trademark.
>
> Regards,
> Rohit Yadav
> Software Architect, ShapeBlue
> M. +41 779015219 | rohit.yadav@shapeblue.com
> Blog: bhaisaab.org | Twitter: @_bhaisaab
>
>
>
>
> Find out more about ShapeBlue and our range of CloudStack related services
>
> IaaS Cloud Design & Build<
> http://shapeblue.com/iaas-cloud-design-and-build//>
> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> CloudStack Infrastructure Support<
> http://shapeblue.com/cloudstack-infrastructure-support/>
> CloudStack Bootcamp Training Courses<
> http://shapeblue.com/cloudstack-training/>
>
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is
> a company registered by The Republic of South Africa and is traded under
> license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>
Re: How to speed up testing using BVT/smoke tests with Simulator
Posted by Rohit Yadav <ro...@shapeblue.com>.
I’ve added config in developer/developer-prefill.sql to have three new config params commonly used by devs and to speed up account cleanup (60s now).
137655d4e11cb490391d0f4982c52262bbb7427c master
a68d8986829c462d655bf59a0cbcc0d4e8b1cc3e 4.4
Regards.
On 01-Aug-2014, at 7:27 am, Santhosh Edukulla <sa...@citrix.com> wrote:
> 1. yes, test cases currently run in parallel in ci environment, using parallel nose commands.
>
> 2. There were few hard codings for sleep, where we removed at some places, still there could be few\many out there.
>
> 3. 1 sec sleep and poll check is too heavy i believe, for async. So, worst a test case can add time of execution by 5 seconds more, compared to succesful operation time.
>
> Santhosh
> ________________________________________
> From: Rohit Yadav [rohit.yadav@shapeblue.com]
> Sent: Thursday, July 31, 2014 7:15 PM
> To: dev@cloudstack.apache.org
> Subject: Re: How to speed up testing using BVT/smoke tests with Simulator
>
> Hi Edison,
>
> Thanks for the pointers! I’ll try them out and see if there is way to do it on Travis/CloudBees as well and I hope other people will religiously start using simulator/bvt (at least the basic ones) for their check-ins.
>
> Regards.
>
> On 01-Aug-2014, at 12:15 am, Edison Su <Ed...@citrix.com> wrote:
>
>>
>>
>>> -----Original Message-----
>>> From: Rohit Yadav [mailto:rohit.yadav@shapeblue.com]
>>> Sent: Thursday, July 31, 2014 5:12 AM
>>> To: dev@cloudstack.apache.org
>>> Subject: How to speed up testing using BVT/smoke tests with Simulator
>>>
>>> Hi,
>>>
>>> Santosh put together a good wiki page on how to validate local changes using
>>> our Python/marvin based build verification tests (path:
>>> test/integration/smoke):
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Validating+check
>>> -ins+for+your+local+changes%2C+using+Simulator
>>>
>>> I've a mini PC for this and using CloudStack 4.4.0 to build/test CloudStack
>>> 4.4/master branch on it in a VM. Some of us are also exploring free/cheap CI
>>> services such as Travis, CloudBees etc. which can be used by developers to
>>> test their check-ins. If anyone of you have tried something like this please
>>> share.
>> Today, I tried to build and test CloudStack on a super powerful machine provided by Azure. Imaging, build & test on a 16 Cores, 120G machine, it should be awesome, and most importantly, it's FREE. You can get a free MSDN subscription fromhttps://svn.apache.org/repos/private/committers/donated-licenses/msdn-subscription.html, after that, you will get $150 credit monthly in Azure. For build &test only, $150 is good enough.
>>
>>>
>>> This is how I build CloudStack for validating with simulator:
>>> mvn -Pdeveloper -Dsimulator clean install
>>> mvn -Pdeveloper -pl developer -Ddeploydb
>>> mvn -Pdeveloper -pl developer -Ddeploydb-simulator
>>> mvn -pl client jetty:run -Dsimulator
>>>
>>> And finally run smoke tests (BVT):
>>> nosetests --with-marvin --marvin-config=setup/dev/advanced.cfg --with-
>>> xunit --xunit-file=/tmp/bvt_selfservice_cases.xml -a
>>> tags=advanced,required_hardware=false -w test/integration/smoke --
>>> hypervisor=simulator
>>>
>>> It currently took 50 mins on my setup. How can we speed it up, say by
>>> reducing global variable timeout settings etc? Should we reduce timeouts etc.
>>> in deploydb-simulator specific sql files?
>>
>> There several places we can improve the marvin test:
>> 1. queryAsyncJob waits 5 second for each call, can change to 1s.
>> 2. There are hardcoded sleep in test code, such base.py, search time.sleep
>> 3. global configuration: account.cleanup.interval sets to 600s, so the test suite will stop for 10 minutes after running for a while.
>> 3. most importantly, if we can run the test cases in parallel, then speedup should be great. Does anybody try to run it in parallel before?
>>
>>>
>>> Regards,
>>> Rohit Yadav
>>> Software Architect, ShapeBlue
>>> M. +41 779015219 | rohit.yadav@shapeblue.com
>>> Blog: bhaisaab.org | Twitter: @_bhaisaab
>>>
>>>
>>>
>>>
>>> Find out more about ShapeBlue and our range of CloudStack related services
>>>
>>> IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-
>>> build//>
>>> CSForge - rapid IaaS deployment
>>> framework<http://shapeblue.com/csforge/>
>>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>>> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-
>>> infrastructure-support/>
>>> CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-
>>> training/>
>>>
>>> This email and any attachments to it may be confidential and are intended
>>> solely for the use of the individual to whom it is addressed. Any views or
>>> opinions expressed are solely those of the author and do not necessarily
>>> represent those of Shape Blue Ltd or related companies. If you are not the
>>> intended recipient of this email, you must neither take any action based
>>> upon its contents, nor copy or show it to anyone. Please contact the sender if
>>> you believe you have received this email in error. Shape Blue Ltd is a
>>> company incorporated in England & Wales. ShapeBlue Services India LLP is a
>>> company incorporated in India and is operated under license from Shape
>>> Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
>>> Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty
>>> Ltd is a company registered by The Republic of South Africa and is traded
>>> under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>
> Regards,
> Rohit Yadav
> Software Architect, ShapeBlue
> M. +41 779015219 | rohit.yadav@shapeblue.com
> Blog: bhaisaab.org | Twitter: @_bhaisaab
>
>
>
>
> Find out more about ShapeBlue and our range of CloudStack related services
>
> IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
> CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
>
> This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.yadav@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab
Find out more about ShapeBlue and our range of CloudStack related services
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
RE: How to speed up testing using BVT/smoke tests with Simulator
Posted by Santhosh Edukulla <sa...@citrix.com>.
1. yes, test cases currently run in parallel in ci environment, using parallel nose commands.
2. There were few hard codings for sleep, where we removed at some places, still there could be few\many out there.
3. 1 sec sleep and poll check is too heavy i believe, for async. So, worst a test case can add time of execution by 5 seconds more, compared to succesful operation time.
Santhosh
________________________________________
From: Rohit Yadav [rohit.yadav@shapeblue.com]
Sent: Thursday, July 31, 2014 7:15 PM
To: dev@cloudstack.apache.org
Subject: Re: How to speed up testing using BVT/smoke tests with Simulator
Hi Edison,
Thanks for the pointers! I’ll try them out and see if there is way to do it on Travis/CloudBees as well and I hope other people will religiously start using simulator/bvt (at least the basic ones) for their check-ins.
Regards.
On 01-Aug-2014, at 12:15 am, Edison Su <Ed...@citrix.com> wrote:
>
>
>> -----Original Message-----
>> From: Rohit Yadav [mailto:rohit.yadav@shapeblue.com]
>> Sent: Thursday, July 31, 2014 5:12 AM
>> To: dev@cloudstack.apache.org
>> Subject: How to speed up testing using BVT/smoke tests with Simulator
>>
>> Hi,
>>
>> Santosh put together a good wiki page on how to validate local changes using
>> our Python/marvin based build verification tests (path:
>> test/integration/smoke):
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Validating+check
>> -ins+for+your+local+changes%2C+using+Simulator
>>
>> I've a mini PC for this and using CloudStack 4.4.0 to build/test CloudStack
>> 4.4/master branch on it in a VM. Some of us are also exploring free/cheap CI
>> services such as Travis, CloudBees etc. which can be used by developers to
>> test their check-ins. If anyone of you have tried something like this please
>> share.
> Today, I tried to build and test CloudStack on a super powerful machine provided by Azure. Imaging, build & test on a 16 Cores, 120G machine, it should be awesome, and most importantly, it's FREE. You can get a free MSDN subscription fromhttps://svn.apache.org/repos/private/committers/donated-licenses/msdn-subscription.html, after that, you will get $150 credit monthly in Azure. For build &test only, $150 is good enough.
>
>>
>> This is how I build CloudStack for validating with simulator:
>> mvn -Pdeveloper -Dsimulator clean install
>> mvn -Pdeveloper -pl developer -Ddeploydb
>> mvn -Pdeveloper -pl developer -Ddeploydb-simulator
>> mvn -pl client jetty:run -Dsimulator
>>
>> And finally run smoke tests (BVT):
>> nosetests --with-marvin --marvin-config=setup/dev/advanced.cfg --with-
>> xunit --xunit-file=/tmp/bvt_selfservice_cases.xml -a
>> tags=advanced,required_hardware=false -w test/integration/smoke --
>> hypervisor=simulator
>>
>> It currently took 50 mins on my setup. How can we speed it up, say by
>> reducing global variable timeout settings etc? Should we reduce timeouts etc.
>> in deploydb-simulator specific sql files?
>
> There several places we can improve the marvin test:
> 1. queryAsyncJob waits 5 second for each call, can change to 1s.
> 2. There are hardcoded sleep in test code, such base.py, search time.sleep
> 3. global configuration: account.cleanup.interval sets to 600s, so the test suite will stop for 10 minutes after running for a while.
> 3. most importantly, if we can run the test cases in parallel, then speedup should be great. Does anybody try to run it in parallel before?
>
>>
>> Regards,
>> Rohit Yadav
>> Software Architect, ShapeBlue
>> M. +41 779015219 | rohit.yadav@shapeblue.com
>> Blog: bhaisaab.org | Twitter: @_bhaisaab
>>
>>
>>
>>
>> Find out more about ShapeBlue and our range of CloudStack related services
>>
>> IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-
>> build//>
>> CSForge - rapid IaaS deployment
>> framework<http://shapeblue.com/csforge/>
>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-
>> infrastructure-support/>
>> CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-
>> training/>
>>
>> This email and any attachments to it may be confidential and are intended
>> solely for the use of the individual to whom it is addressed. Any views or
>> opinions expressed are solely those of the author and do not necessarily
>> represent those of Shape Blue Ltd or related companies. If you are not the
>> intended recipient of this email, you must neither take any action based
>> upon its contents, nor copy or show it to anyone. Please contact the sender if
>> you believe you have received this email in error. Shape Blue Ltd is a
>> company incorporated in England & Wales. ShapeBlue Services India LLP is a
>> company incorporated in India and is operated under license from Shape
>> Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
>> Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty
>> Ltd is a company registered by The Republic of South Africa and is traded
>> under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.yadav@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab
Find out more about ShapeBlue and our range of CloudStack related services
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Re: How to speed up testing using BVT/smoke tests with Simulator
Posted by Rohit Yadav <ro...@shapeblue.com>.
Hi Edison,
Thanks for the pointers! I’ll try them out and see if there is way to do it on Travis/CloudBees as well and I hope other people will religiously start using simulator/bvt (at least the basic ones) for their check-ins.
Regards.
On 01-Aug-2014, at 12:15 am, Edison Su <Ed...@citrix.com> wrote:
>
>
>> -----Original Message-----
>> From: Rohit Yadav [mailto:rohit.yadav@shapeblue.com]
>> Sent: Thursday, July 31, 2014 5:12 AM
>> To: dev@cloudstack.apache.org
>> Subject: How to speed up testing using BVT/smoke tests with Simulator
>>
>> Hi,
>>
>> Santosh put together a good wiki page on how to validate local changes using
>> our Python/marvin based build verification tests (path:
>> test/integration/smoke):
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Validating+check
>> -ins+for+your+local+changes%2C+using+Simulator
>>
>> I've a mini PC for this and using CloudStack 4.4.0 to build/test CloudStack
>> 4.4/master branch on it in a VM. Some of us are also exploring free/cheap CI
>> services such as Travis, CloudBees etc. which can be used by developers to
>> test their check-ins. If anyone of you have tried something like this please
>> share.
> Today, I tried to build and test CloudStack on a super powerful machine provided by Azure. Imaging, build & test on a 16 Cores, 120G machine, it should be awesome, and most importantly, it's FREE. You can get a free MSDN subscription fromhttps://svn.apache.org/repos/private/committers/donated-licenses/msdn-subscription.html, after that, you will get $150 credit monthly in Azure. For build &test only, $150 is good enough.
>
>>
>> This is how I build CloudStack for validating with simulator:
>> mvn -Pdeveloper -Dsimulator clean install
>> mvn -Pdeveloper -pl developer -Ddeploydb
>> mvn -Pdeveloper -pl developer -Ddeploydb-simulator
>> mvn -pl client jetty:run -Dsimulator
>>
>> And finally run smoke tests (BVT):
>> nosetests --with-marvin --marvin-config=setup/dev/advanced.cfg --with-
>> xunit --xunit-file=/tmp/bvt_selfservice_cases.xml -a
>> tags=advanced,required_hardware=false -w test/integration/smoke --
>> hypervisor=simulator
>>
>> It currently took 50 mins on my setup. How can we speed it up, say by
>> reducing global variable timeout settings etc? Should we reduce timeouts etc.
>> in deploydb-simulator specific sql files?
>
> There several places we can improve the marvin test:
> 1. queryAsyncJob waits 5 second for each call, can change to 1s.
> 2. There are hardcoded sleep in test code, such base.py, search time.sleep
> 3. global configuration: account.cleanup.interval sets to 600s, so the test suite will stop for 10 minutes after running for a while.
> 3. most importantly, if we can run the test cases in parallel, then speedup should be great. Does anybody try to run it in parallel before?
>
>>
>> Regards,
>> Rohit Yadav
>> Software Architect, ShapeBlue
>> M. +41 779015219 | rohit.yadav@shapeblue.com
>> Blog: bhaisaab.org | Twitter: @_bhaisaab
>>
>>
>>
>>
>> Find out more about ShapeBlue and our range of CloudStack related services
>>
>> IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-
>> build//>
>> CSForge - rapid IaaS deployment
>> framework<http://shapeblue.com/csforge/>
>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-
>> infrastructure-support/>
>> CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-
>> training/>
>>
>> This email and any attachments to it may be confidential and are intended
>> solely for the use of the individual to whom it is addressed. Any views or
>> opinions expressed are solely those of the author and do not necessarily
>> represent those of Shape Blue Ltd or related companies. If you are not the
>> intended recipient of this email, you must neither take any action based
>> upon its contents, nor copy or show it to anyone. Please contact the sender if
>> you believe you have received this email in error. Shape Blue Ltd is a
>> company incorporated in England & Wales. ShapeBlue Services India LLP is a
>> company incorporated in India and is operated under license from Shape
>> Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
>> Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty
>> Ltd is a company registered by The Republic of South Africa and is traded
>> under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.yadav@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab
Find out more about ShapeBlue and our range of CloudStack related services
IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
RE: How to speed up testing using BVT/smoke tests with Simulator
Posted by Edison Su <Ed...@citrix.com>.
> -----Original Message-----
> From: Rohit Yadav [mailto:rohit.yadav@shapeblue.com]
> Sent: Thursday, July 31, 2014 5:12 AM
> To: dev@cloudstack.apache.org
> Subject: How to speed up testing using BVT/smoke tests with Simulator
>
> Hi,
>
> Santosh put together a good wiki page on how to validate local changes using
> our Python/marvin based build verification tests (path:
> test/integration/smoke):
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Validating+check
> -ins+for+your+local+changes%2C+using+Simulator
>
> I've a mini PC for this and using CloudStack 4.4.0 to build/test CloudStack
> 4.4/master branch on it in a VM. Some of us are also exploring free/cheap CI
> services such as Travis, CloudBees etc. which can be used by developers to
> test their check-ins. If anyone of you have tried something like this please
> share.
Today, I tried to build and test CloudStack on a super powerful machine provided by Azure. Imaging, build & test on a 16 Cores, 120G machine, it should be awesome, and most importantly, it's FREE. You can get a free MSDN subscription from https://svn.apache.org/repos/private/committers/donated-licenses/msdn-subscription.html, after that, you will get $150 credit monthly in Azure. For build &test only, $150 is good enough.
>
> This is how I build CloudStack for validating with simulator:
> mvn -Pdeveloper -Dsimulator clean install
> mvn -Pdeveloper -pl developer -Ddeploydb
> mvn -Pdeveloper -pl developer -Ddeploydb-simulator
> mvn -pl client jetty:run -Dsimulator
>
> And finally run smoke tests (BVT):
> nosetests --with-marvin --marvin-config=setup/dev/advanced.cfg --with-
> xunit --xunit-file=/tmp/bvt_selfservice_cases.xml -a
> tags=advanced,required_hardware=false -w test/integration/smoke --
> hypervisor=simulator
>
> It currently took 50 mins on my setup. How can we speed it up, say by
> reducing global variable timeout settings etc? Should we reduce timeouts etc.
> in deploydb-simulator specific sql files?
There several places we can improve the marvin test:
1. queryAsyncJob waits 5 second for each call, can change to 1s.
2. There are hardcoded sleep in test code, such base.py, search time.sleep
3. global configuration: account.cleanup.interval sets to 600s, so the test suite will stop for 10 minutes after running for a while.
3. most importantly, if we can run the test cases in parallel, then speedup should be great. Does anybody try to run it in parallel before?
>
> Regards,
> Rohit Yadav
> Software Architect, ShapeBlue
> M. +41 779015219 | rohit.yadav@shapeblue.com
> Blog: bhaisaab.org | Twitter: @_bhaisaab
>
>
>
>
> Find out more about ShapeBlue and our range of CloudStack related services
>
> IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-
> build//>
> CSForge - rapid IaaS deployment
> framework<http://shapeblue.com/csforge/>
> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-
> infrastructure-support/>
> CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-
> training/>
>
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender if
> you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape
> Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
> Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty
> Ltd is a company registered by The Republic of South Africa and is traded
> under license from Shape Blue Ltd. ShapeBlue is a registered trademark.