You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by da...@zv.fraunhofer.de on 2018/03/20 09:38:25 UTC
Automated testing of our environment
Hi all,
We’d like to automatically run some tests again our live environment on regular basis. Something like:
“Deploy a VM with a specific template, assign an IP address, enable static NAT, and check if the SSH password and keys work”
While researching, I came across the Marvin tests described here: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+-+Testing+with+Python . I guess this would allow us to greatly reduce the amount of required work to setup test cases. What makes me curious however:
The Framework does require access to the CS database. Is it actually writing to the DB, or is it only using the DB to verify the API operations, such that a read-only user account would be enough?
Regards
Daniel
--
Daniel Herrmann
Network Engineer – Fraunhofer Private Cloud
CCIE #55056 (Routing and Switching)
Re: Automated testing of our environment
Posted by Boris Stoyanov <bo...@shapeblue.com>.
Hi Daniel,
Marvin is the framework that carries out the integration tests of CloudStack. We use it extensively in the community when testing new changes. I can recommend you to use the all the tests in ‘smoke' directory since they cover most of the functionality.
Since 4.10 we’ve invested a big effort into fixing most of the issues in the smoke tests and now they are mostly passing, with some small exceptions.
On your question, it’s not recommended to run marvin tests agains your prod environment since it’ll be creating new vms, networks and etc. This can cause downtime and unavailability of some of your resources. Marvin DOES NOT write anything in the data base, with some ignorable exceptions (changing global settings and then reverting them), so you should not be worried about marvin interactions with the DB. I would recommend you to run tests only against a test environment.
Boris Stoyanov
boris.stoyanov@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London WC2N 4HSUK
@shapeblue
On 20 Mar 2018, at 11:38, daniel.herrmann@zv.fraunhofer.de<ma...@zv.fraunhofer.de> wrote:
Hi all,
We’d like to automatically run some tests again our live environment on regular basis. Something like:
“Deploy a VM with a specific template, assign an IP address, enable static NAT, and check if the SSH password and keys work”
While researching, I came across the Marvin tests described here: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+-+Testing+with+Python . I guess this would allow us to greatly reduce the amount of required work to setup test cases. What makes me curious however:
The Framework does require access to the CS database. Is it actually writing to the DB, or is it only using the DB to verify the API operations, such that a read-only user account would be enough?
Regards
Daniel
--
Daniel Herrmann
Network Engineer – Fraunhofer Private Cloud
CCIE #55056 (Routing and Switching)
Re: Automated testing of our environment
Posted by Rene Moser <ma...@renemoser.net>.
Hi Daniel again
On 03/21/2018 09:33 AM, daniel.herrmann@zv.fraunhofer.de wrote:
> Hi Rene, Hi Boris,
>
> Thanks @Boris for the information about Marvin. I guess in our case (running automated test against our prod environment) that would not be the ideal choice.
>
>> This would be very easy to do with ansible.
>>
>> It would even allow to log in to the deployed VM and run tasks on the
>> VM, e.g. install packages like a webserver and then check if the
>> webserver is accessable from the CI system.
>>
>> Or in case of VPC integration tests: verify if a configured site2site
>> VPN works, etc. All with ansible's battery included...
>>
>> https://github.com/resmo/ansible-cloudstack-example may give you some
>> ideas how to setup.
>
> The ansible approach however looks promising, thank you. Going to look into that and find out how to write "tests" using ansible. I'm much more addicted to programming in Python than I am to ansible, and from the automation point of view we are using puppet, so let's see what happens. :-)
If you like, you could even write ansible playbooks in python. Ansible
is in Python... But I don't see a real benefit
However, probably one more quickstart hint, I already wrote playbooks to
test the ansible modules.
These are located in
https://github.com/ansible/ansible/blob/devel/test/integration/targets/
and prefixed with cs_
e.g
https://github.com/ansible/ansible/blob/devel/test/integration/targets/cs_domain/tasks/main.yml
The nice thing is the interaction with the "assert" module which allows
to compare the return values from the cloudstack modules to be in the
expected state.
Have fun! ;)
Re: Automated testing of our environment
Posted by da...@zv.fraunhofer.de.
Hi Rene, Hi Boris,
Thanks @Boris for the information about Marvin. I guess in our case (running automated test against our prod environment) that would not be the ideal choice.
> This would be very easy to do with ansible.
>
> It would even allow to log in to the deployed VM and run tasks on the
> VM, e.g. install packages like a webserver and then check if the
> webserver is accessable from the CI system.
>
> Or in case of VPC integration tests: verify if a configured site2site
> VPN works, etc. All with ansible's battery included...
>
> https://github.com/resmo/ansible-cloudstack-example may give you some
> ideas how to setup.
The ansible approach however looks promising, thank you. Going to look into that and find out how to write "tests" using ansible. I'm much more addicted to programming in Python than I am to ansible, and from the automation point of view we are using puppet, so let's see what happens. :-)
Thanks and regards
Daniel
--
Daniel Herrmann
Network Engineer – Fraunhofer Private Cloud
CCIE #55056 (Routing and Switching)
Cisco CCDP, CCIP; Fluke CCTT
Re: Automated testing of our environment
Posted by Rene Moser <ma...@renemoser.net>.
Hi Daniel
On 03/20/2018 10:38 AM, daniel.herrmann@zv.fraunhofer.de wrote:
> We’d like to automatically run some tests again our live environment on
> regular basis. Something like:
>
> “Deploy a VM with a specific template, assign an IP address, enable
> static NAT, and check if the SSH password and keys work”
This would be very easy to do with ansible.
It would even allow to log in to the deployed VM and run tasks on the
VM, e.g. install packages like a webserver and then check if the
webserver is accessable from the CI system.
Or in case of VPC integration tests: verify if a configured site2site
VPN works, etc. All with ansible's battery included...
https://github.com/resmo/ansible-cloudstack-example may give you some
ideas how to setup.
Yours
René