You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Yichi Lu <yi...@sungard.com> on 2012/09/17 23:56:10 UTC
unit tests for api/src/com/cloud/agent/api
I'd start to write unit tests for classes in api/src/com/cloud/agent/api in
alphabetical order.
Let me know if you have any suggestions.
Yichi Lu
RE: unit tests for api/src/com/cloud/agent/api
Posted by Sudha Ponnaganti <su...@citrix.com>.
+1
Awesome indeed!! Looking forward for it
-----Original Message-----
From: David Nalley [mailto:david@gnsa.us]
Sent: Monday, September 17, 2012 3:08 PM
To: cloudstack-dev@incubator.apache.org
Subject: Re: unit tests for api/src/com/cloud/agent/api
On Mon, Sep 17, 2012 at 5:56 PM, Yichi Lu <yi...@sungard.com> wrote:
> I'd start to write unit tests for classes in
> api/src/com/cloud/agent/api in alphabetical order.
>
> Let me know if you have any suggestions.
>
> Yichi Lu
AWESOME - terribly excited to see this!
--David
Re: unit tests for api/src/com/cloud/agent/api
Posted by David Nalley <da...@gnsa.us>.
On Mon, Sep 17, 2012 at 5:56 PM, Yichi Lu <yi...@sungard.com> wrote:
> I'd start to write unit tests for classes in api/src/com/cloud/agent/api in
> alphabetical order.
>
> Let me know if you have any suggestions.
>
> Yichi Lu
AWESOME - terribly excited to see this!
--David
Re: unit tests for api/src/com/cloud/agent/api
Posted by Yichi Lu <yi...@sungard.com>.
Prasanna:
It is my fault. I installed the wrong marvin package.
Yichi
On Tue, Sep 18, 2012 at 11:55 AM, prasanna <ts...@apache.org> wrote:
> Sorry jclouds has a cloudstack suite for advanced zone as well. Not
> just ec2. cloudstack-ec2 is to test the ec2 shim of cloudstack.
>
> On 18 September 2012 22:23, prasanna <ts...@apache.org> wrote:
> > On 18 September 2012 18:20, David Nalley <da...@gnsa.us> wrote:
> >> On Mon, Sep 17, 2012 at 7:24 PM, Edison Su <Ed...@citrix.com>
> wrote:
> >>> First, thanks for working on test stuff, we really need more automate
> test cases.
> >>> If you plan to work on the API test, we have a python test framework:
> https://cwiki.apache.org/CLOUDSTACK/testing-with-python.html. The most
> compelling feature is that it supports code completion for cloudstack API,
> very easy to write test case in Eclipse environment. And we are setting up
> a test infrastructure, which will execute all the test cases written for
> this framework.
> >>> We already have around 500+ test cases, or maybe more, but apparently
> it's not enough. If you like to work on it, that will be great!
> >>>
> >>
> >> While I like the functional/integration tests, as you note we have
> >> 500+ currently. Those tests also take many hours to run at present.
> >> Unit tests should be something fast, a sanity check, and we only have
> >> 25 right now for the entire code base, so while I like
> >> func/integration tests, I'd really like for our unit tests to be
> >> runnable on every commit, and provide some decent level of coverage so
> >> that we have a bit of confidence that code works well, and provide
> >> good feedback to folks on whether their commit passes the sniff test.
> >>
> >> --David
> >
> > We can speed up the tests. Just never got to have marvin drive
> > everything asynchronously. It's been on my TODO list forever now as
> > have the API tests. I for one would be extremely happy to see some.
> >
> > About the tool though, here's my opinion:
> >
> > An API test would be a great addition to the marvin suite. Marvin
> > makes it easier to write them too. All you will need is compare a
> > canned response from say listVirtualMachines with what is returned by
> > the CS endpoint. Writing these as python's doctests would make it
> > cleaner and easier to maintain too. Having one tool do all functional
> > and sanity tests is a bonus as well.
> >
> > If you do choose to go the jUnit/java way I suggest you look at
> > jClouds as well. It has a cloudstack-ec2 suite that does API tests
> > already. They will need to be extended and brought to match with the
> > changes done for 4.0.
> >
> > Yichi - I think if you begin testing with all the listXxx API calls
> > and put them down as unit-tests the createXxx, addXxx, deleteXxx will
> > fall in place automatically.
> >
> >
> > HTH
>
>
Re: unit tests for api/src/com/cloud/agent/api
Posted by prasanna <ts...@apache.org>.
Sorry jclouds has a cloudstack suite for advanced zone as well. Not
just ec2. cloudstack-ec2 is to test the ec2 shim of cloudstack.
On 18 September 2012 22:23, prasanna <ts...@apache.org> wrote:
> On 18 September 2012 18:20, David Nalley <da...@gnsa.us> wrote:
>> On Mon, Sep 17, 2012 at 7:24 PM, Edison Su <Ed...@citrix.com> wrote:
>>> First, thanks for working on test stuff, we really need more automate test cases.
>>> If you plan to work on the API test, we have a python test framework: https://cwiki.apache.org/CLOUDSTACK/testing-with-python.html. The most compelling feature is that it supports code completion for cloudstack API, very easy to write test case in Eclipse environment. And we are setting up a test infrastructure, which will execute all the test cases written for this framework.
>>> We already have around 500+ test cases, or maybe more, but apparently it's not enough. If you like to work on it, that will be great!
>>>
>>
>> While I like the functional/integration tests, as you note we have
>> 500+ currently. Those tests also take many hours to run at present.
>> Unit tests should be something fast, a sanity check, and we only have
>> 25 right now for the entire code base, so while I like
>> func/integration tests, I'd really like for our unit tests to be
>> runnable on every commit, and provide some decent level of coverage so
>> that we have a bit of confidence that code works well, and provide
>> good feedback to folks on whether their commit passes the sniff test.
>>
>> --David
>
> We can speed up the tests. Just never got to have marvin drive
> everything asynchronously. It's been on my TODO list forever now as
> have the API tests. I for one would be extremely happy to see some.
>
> About the tool though, here's my opinion:
>
> An API test would be a great addition to the marvin suite. Marvin
> makes it easier to write them too. All you will need is compare a
> canned response from say listVirtualMachines with what is returned by
> the CS endpoint. Writing these as python's doctests would make it
> cleaner and easier to maintain too. Having one tool do all functional
> and sanity tests is a bonus as well.
>
> If you do choose to go the jUnit/java way I suggest you look at
> jClouds as well. It has a cloudstack-ec2 suite that does API tests
> already. They will need to be extended and brought to match with the
> changes done for 4.0.
>
> Yichi - I think if you begin testing with all the listXxx API calls
> and put them down as unit-tests the createXxx, addXxx, deleteXxx will
> fall in place automatically.
>
>
> HTH
Re: unit tests for api/src/com/cloud/agent/api
Posted by Prasanna Santhanam <Pr...@citrix.com>.
> > We can speed up the tests. Just never got to have marvin drive
> > everything asynchronously.It's been on my TODO list forever now as
> > have the API tests. I for one would be extremely happy to see some.
>
> I am gonna look into the speed up. Where you thinking of threading
> it or using nose which seems to support multiprocess out of the box
> ?
>
Thanks Sebastien -
The threading model should split each functional test ie
cloudstackTestCase to go in a different thread. We need to ensure that
the API requests inside each test itself goes synchronously because
they depend on the respnonses from the prev. request.
nosetests with the multiprocess plugin I thought would fix this. For
some reason urllib fails when run through this plugin. Guess urllib
itself seems to have issues with reentrant code like this.
We can replace urllib with something threadsafe and retry the
multiprocess plugin too. You could look at asyncJobMgr.py also where
some threading implementation exist which could probably be salvaged.
--
Prasanna.,
Re: unit tests for api/src/com/cloud/agent/api
Posted by Sebastien Goasguen <ru...@gmail.com>.
On Sep 18, 2012, at 6:53 PM, prasanna <ts...@apache.org> wrote:
> On 18 September 2012 18:20, David Nalley <da...@gnsa.us> wrote:
>> On Mon, Sep 17, 2012 at 7:24 PM, Edison Su <Ed...@citrix.com> wrote:
>>> First, thanks for working on test stuff, we really need more automate test cases.
>>> If you plan to work on the API test, we have a python test framework: https://cwiki.apache.org/CLOUDSTACK/testing-with-python.html. The most compelling feature is that it supports code completion for cloudstack API, very easy to write test case in Eclipse environment. And we are setting up a test infrastructure, which will execute all the test cases written for this framework.
>>> We already have around 500+ test cases, or maybe more, but apparently it's not enough. If you like to work on it, that will be great!
>>>
>>
>> While I like the functional/integration tests, as you note we have
>> 500+ currently. Those tests also take many hours to run at present.
>> Unit tests should be something fast, a sanity check, and we only have
>> 25 right now for the entire code base, so while I like
>> func/integration tests, I'd really like for our unit tests to be
>> runnable on every commit, and provide some decent level of coverage so
>> that we have a bit of confidence that code works well, and provide
>> good feedback to folks on whether their commit passes the sniff test.
>>
>> --David
>
> We can speed up the tests. Just never got to have marvin drive
> everything asynchronously.It's been on my TODO list forever now as
> have the API tests. I for one would be extremely happy to see some.
I am gonna look into the speed up. Where you thinking of threading it or using nose which seems to support multiprocess out of the box ?
> About the tool though, here's my opinion:
>
> An API test would be a great addition to the marvin suite. Marvin
> makes it easier to write them too. All you will need is compare a
> canned response from say listVirtualMachines with what is returned by
> the CS endpoint. Writing these as python's doctests would make it
> cleaner and easier to maintain too. Having one tool do all functional
> and sanity tests is a bonus as well.
>
> If you do choose to go the jUnit/java way I suggest you look at
> jClouds as well. It has a cloudstack-ec2 suite that does API tests
> already. They will need to be extended and brought to match with the
> changes done for 4.0.
>
> Yichi - I think if you begin testing with all the listXxx API calls
> and put them down as unit-tests the createXxx, addXxx, deleteXxx will
> fall in place automatically.
>
>
> HTH
Re: unit tests for api/src/com/cloud/agent/api
Posted by prasanna <ts...@apache.org>.
On 18 September 2012 18:20, David Nalley <da...@gnsa.us> wrote:
> On Mon, Sep 17, 2012 at 7:24 PM, Edison Su <Ed...@citrix.com> wrote:
>> First, thanks for working on test stuff, we really need more automate test cases.
>> If you plan to work on the API test, we have a python test framework: https://cwiki.apache.org/CLOUDSTACK/testing-with-python.html. The most compelling feature is that it supports code completion for cloudstack API, very easy to write test case in Eclipse environment. And we are setting up a test infrastructure, which will execute all the test cases written for this framework.
>> We already have around 500+ test cases, or maybe more, but apparently it's not enough. If you like to work on it, that will be great!
>>
>
> While I like the functional/integration tests, as you note we have
> 500+ currently. Those tests also take many hours to run at present.
> Unit tests should be something fast, a sanity check, and we only have
> 25 right now for the entire code base, so while I like
> func/integration tests, I'd really like for our unit tests to be
> runnable on every commit, and provide some decent level of coverage so
> that we have a bit of confidence that code works well, and provide
> good feedback to folks on whether their commit passes the sniff test.
>
> --David
We can speed up the tests. Just never got to have marvin drive
everything asynchronously. It's been on my TODO list forever now as
have the API tests. I for one would be extremely happy to see some.
About the tool though, here's my opinion:
An API test would be a great addition to the marvin suite. Marvin
makes it easier to write them too. All you will need is compare a
canned response from say listVirtualMachines with what is returned by
the CS endpoint. Writing these as python's doctests would make it
cleaner and easier to maintain too. Having one tool do all functional
and sanity tests is a bonus as well.
If you do choose to go the jUnit/java way I suggest you look at
jClouds as well. It has a cloudstack-ec2 suite that does API tests
already. They will need to be extended and brought to match with the
changes done for 4.0.
Yichi - I think if you begin testing with all the listXxx API calls
and put them down as unit-tests the createXxx, addXxx, deleteXxx will
fall in place automatically.
HTH
Re: unit tests for api/src/com/cloud/agent/api
Posted by Yichi Lu <yi...@sungard.com>.
I think so. JUnit probably is a better way at this point.
Yichi
On Tue, Sep 18, 2012 at 11:32 AM, Chip Childers
<ch...@sungard.com>wrote:
> Yichi,
>
> Would more traditional unit tests (using JUnit and moc'ed
> dependencies) perhaps be a better way to get to unit test coverage?
> (As opposed to other automated test types)
>
> -chip
>
> On Tue, Sep 18, 2012 at 12:27 PM, Yichi Lu <yi...@sungard.com> wrote:
> > So I folllowed the doc:
> > https://cwiki.apache.org/CLOUDSTACK/testing-with-python.html, and tried
> to
> > install marvin to my 64 bit MacBook, OS: Lion. I installed marvin.0.6.2,
> > and also biopython-1.59
> > because:
> >>>> import marvin
> > Traceback (most recent call last):
> > File "<stdin>", line 1, in <module>
> > File "/usr/local/bin/marvin.py", line 2, in <module>
> > import Marvin.app.main
> > File "/Library/Python/2.7/site-packages/Marvin/__init__.py", line 8, in
> > <module>
> > from Marvin.primer import PrimerDesigner
> > File "/Library/Python/2.7/site-packages/Marvin/primer.py", line 3, in
> > <module>
> > from Bio.Seq import Seq
> > ImportError: No module named Bio.Seq
> >
> > and wxPython2.8-osx-unicode-py2.7
> > because:
> >>>> import marvin
> > Traceback (most recent call last):
> > File "<stdin>", line 1, in <module>
> > File "/usr/local/bin/marvin.py", line 2, in <module>
> > import Marvin.app.main
> > File "/Library/Python/2.7/site-packages/Marvin/app/main.py", line 2, in
> > <module>
> > import wx
> > ImportError: No module named wx
> >
> > Next I got:
> > ImportError:
> >
> /usr/local/lib/wxPython-unicode-2.8.12.1/lib/python2.7/site-packages/wx-2.8-mac-unicode/wx/_core_.so:
> > no appropriate 64-bit architecture (see "man python" for running in
> 32-bit
> > mode)
> >
> > After googling, I was told that wxPython on Mac is only available in
> 32-bit
> > mode, and this link (
> >
> http://stackoverflow.com/questions/2565201/wxpython-incompatible-with-snow-leopard
> )
> > suggests to run python in 32 bit mode.
> >
> > My question to you guys is: am I on the right path so far? Should I run
> > python marvin in 32 bit python mode? Any suggestions?
> >
> > Thanks.
> >
> > Yichi
> >
> >
> > On Tue, Sep 18, 2012 at 7:50 AM, David Nalley <da...@gnsa.us> wrote:
> >
> >> On Mon, Sep 17, 2012 at 7:24 PM, Edison Su <Ed...@citrix.com>
> wrote:
> >> > First, thanks for working on test stuff, we really need more automate
> >> test cases.
> >> > If you plan to work on the API test, we have a python test framework:
> >> https://cwiki.apache.org/CLOUDSTACK/testing-with-python.html. The most
> >> compelling feature is that it supports code completion for cloudstack
> API,
> >> very easy to write test case in Eclipse environment. And we are setting
> up
> >> a test infrastructure, which will execute all the test cases written for
> >> this framework.
> >> > We already have around 500+ test cases, or maybe more, but apparently
> >> it's not enough. If you like to work on it, that will be great!
> >> >
> >>
> >> While I like the functional/integration tests, as you note we have
> >> 500+ currently. Those tests also take many hours to run at present.
> >> Unit tests should be something fast, a sanity check, and we only have
> >> 25 right now for the entire code base, so while I like
> >> func/integration tests, I'd really like for our unit tests to be
> >> runnable on every commit, and provide some decent level of coverage so
> >> that we have a bit of confidence that code works well, and provide
> >> good feedback to folks on whether their commit passes the sniff test.
> >>
> >> --David
> >>
> >>
>
>
Re: unit tests for api/src/com/cloud/agent/api
Posted by Chip Childers <ch...@sungard.com>.
Yichi,
Would more traditional unit tests (using JUnit and moc'ed
dependencies) perhaps be a better way to get to unit test coverage?
(As opposed to other automated test types)
-chip
On Tue, Sep 18, 2012 at 12:27 PM, Yichi Lu <yi...@sungard.com> wrote:
> So I folllowed the doc:
> https://cwiki.apache.org/CLOUDSTACK/testing-with-python.html, and tried to
> install marvin to my 64 bit MacBook, OS: Lion. I installed marvin.0.6.2,
> and also biopython-1.59
> because:
>>>> import marvin
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "/usr/local/bin/marvin.py", line 2, in <module>
> import Marvin.app.main
> File "/Library/Python/2.7/site-packages/Marvin/__init__.py", line 8, in
> <module>
> from Marvin.primer import PrimerDesigner
> File "/Library/Python/2.7/site-packages/Marvin/primer.py", line 3, in
> <module>
> from Bio.Seq import Seq
> ImportError: No module named Bio.Seq
>
> and wxPython2.8-osx-unicode-py2.7
> because:
>>>> import marvin
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "/usr/local/bin/marvin.py", line 2, in <module>
> import Marvin.app.main
> File "/Library/Python/2.7/site-packages/Marvin/app/main.py", line 2, in
> <module>
> import wx
> ImportError: No module named wx
>
> Next I got:
> ImportError:
> /usr/local/lib/wxPython-unicode-2.8.12.1/lib/python2.7/site-packages/wx-2.8-mac-unicode/wx/_core_.so:
> no appropriate 64-bit architecture (see "man python" for running in 32-bit
> mode)
>
> After googling, I was told that wxPython on Mac is only available in 32-bit
> mode, and this link (
> http://stackoverflow.com/questions/2565201/wxpython-incompatible-with-snow-leopard)
> suggests to run python in 32 bit mode.
>
> My question to you guys is: am I on the right path so far? Should I run
> python marvin in 32 bit python mode? Any suggestions?
>
> Thanks.
>
> Yichi
>
>
> On Tue, Sep 18, 2012 at 7:50 AM, David Nalley <da...@gnsa.us> wrote:
>
>> On Mon, Sep 17, 2012 at 7:24 PM, Edison Su <Ed...@citrix.com> wrote:
>> > First, thanks for working on test stuff, we really need more automate
>> test cases.
>> > If you plan to work on the API test, we have a python test framework:
>> https://cwiki.apache.org/CLOUDSTACK/testing-with-python.html. The most
>> compelling feature is that it supports code completion for cloudstack API,
>> very easy to write test case in Eclipse environment. And we are setting up
>> a test infrastructure, which will execute all the test cases written for
>> this framework.
>> > We already have around 500+ test cases, or maybe more, but apparently
>> it's not enough. If you like to work on it, that will be great!
>> >
>>
>> While I like the functional/integration tests, as you note we have
>> 500+ currently. Those tests also take many hours to run at present.
>> Unit tests should be something fast, a sanity check, and we only have
>> 25 right now for the entire code base, so while I like
>> func/integration tests, I'd really like for our unit tests to be
>> runnable on every commit, and provide some decent level of coverage so
>> that we have a bit of confidence that code works well, and provide
>> good feedback to folks on whether their commit passes the sniff test.
>>
>> --David
>>
>>
Re: unit tests for api/src/com/cloud/agent/api
Posted by prasanna <ts...@apache.org>.
Yichi - I think you've got a completely different marvin from the
cheeseshop here: http://pypi.python.org/pypi/marvin/0.6.2
What you need is this: http://jenkins.cloudstack.org/job/build-marvin-master/
DId the documentation mislead you somehwere?
On 18 September 2012 21:57, Yichi Lu <yi...@sungard.com> wrote:
> So I folllowed the doc:
> https://cwiki.apache.org/CLOUDSTACK/testing-with-python.html, and tried to
> install marvin to my 64 bit MacBook, OS: Lion. I installed marvin.0.6.2,
> and also biopython-1.59
> because:
>>>> import marvin
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "/usr/local/bin/marvin.py", line 2, in <module>
> import Marvin.app.main
> File "/Library/Python/2.7/site-packages/Marvin/__init__.py", line 8, in
> <module>
> from Marvin.primer import PrimerDesigner
> File "/Library/Python/2.7/site-packages/Marvin/primer.py", line 3, in
> <module>
> from Bio.Seq import Seq
> ImportError: No module named Bio.Seq
>
> and wxPython2.8-osx-unicode-py2.7
> because:
>>>> import marvin
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "/usr/local/bin/marvin.py", line 2, in <module>
> import Marvin.app.main
> File "/Library/Python/2.7/site-packages/Marvin/app/main.py", line 2, in
> <module>
> import wx
> ImportError: No module named wx
>
> Next I got:
> ImportError:
> /usr/local/lib/wxPython-unicode-2.8.12.1/lib/python2.7/site-packages/wx-2.8-mac-unicode/wx/_core_.so:
> no appropriate 64-bit architecture (see "man python" for running in 32-bit
> mode)
>
> After googling, I was told that wxPython on Mac is only available in 32-bit
> mode, and this link (
> http://stackoverflow.com/questions/2565201/wxpython-incompatible-with-snow-leopard)
> suggests to run python in 32 bit mode.
>
> My question to you guys is: am I on the right path so far? Should I run
> python marvin in 32 bit python mode? Any suggestions?
>
> Thanks.
>
> Yichi
>
>
> On Tue, Sep 18, 2012 at 7:50 AM, David Nalley <da...@gnsa.us> wrote:
>
>> On Mon, Sep 17, 2012 at 7:24 PM, Edison Su <Ed...@citrix.com> wrote:
>> > First, thanks for working on test stuff, we really need more automate
>> test cases.
>> > If you plan to work on the API test, we have a python test framework:
>> https://cwiki.apache.org/CLOUDSTACK/testing-with-python.html. The most
>> compelling feature is that it supports code completion for cloudstack API,
>> very easy to write test case in Eclipse environment. And we are setting up
>> a test infrastructure, which will execute all the test cases written for
>> this framework.
>> > We already have around 500+ test cases, or maybe more, but apparently
>> it's not enough. If you like to work on it, that will be great!
>> >
>>
>> While I like the functional/integration tests, as you note we have
>> 500+ currently. Those tests also take many hours to run at present.
>> Unit tests should be something fast, a sanity check, and we only have
>> 25 right now for the entire code base, so while I like
>> func/integration tests, I'd really like for our unit tests to be
>> runnable on every commit, and provide some decent level of coverage so
>> that we have a bit of confidence that code works well, and provide
>> good feedback to folks on whether their commit passes the sniff test.
>>
>> --David
>>
>>
Re: unit tests for api/src/com/cloud/agent/api
Posted by Yichi Lu <yi...@sungard.com>.
So I folllowed the doc:
https://cwiki.apache.org/CLOUDSTACK/testing-with-python.html, and tried to
install marvin to my 64 bit MacBook, OS: Lion. I installed marvin.0.6.2,
and also biopython-1.59
because:
>>> import marvin
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/bin/marvin.py", line 2, in <module>
import Marvin.app.main
File "/Library/Python/2.7/site-packages/Marvin/__init__.py", line 8, in
<module>
from Marvin.primer import PrimerDesigner
File "/Library/Python/2.7/site-packages/Marvin/primer.py", line 3, in
<module>
from Bio.Seq import Seq
ImportError: No module named Bio.Seq
and wxPython2.8-osx-unicode-py2.7
because:
>>> import marvin
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/bin/marvin.py", line 2, in <module>
import Marvin.app.main
File "/Library/Python/2.7/site-packages/Marvin/app/main.py", line 2, in
<module>
import wx
ImportError: No module named wx
Next I got:
ImportError:
/usr/local/lib/wxPython-unicode-2.8.12.1/lib/python2.7/site-packages/wx-2.8-mac-unicode/wx/_core_.so:
no appropriate 64-bit architecture (see "man python" for running in 32-bit
mode)
After googling, I was told that wxPython on Mac is only available in 32-bit
mode, and this link (
http://stackoverflow.com/questions/2565201/wxpython-incompatible-with-snow-leopard)
suggests to run python in 32 bit mode.
My question to you guys is: am I on the right path so far? Should I run
python marvin in 32 bit python mode? Any suggestions?
Thanks.
Yichi
On Tue, Sep 18, 2012 at 7:50 AM, David Nalley <da...@gnsa.us> wrote:
> On Mon, Sep 17, 2012 at 7:24 PM, Edison Su <Ed...@citrix.com> wrote:
> > First, thanks for working on test stuff, we really need more automate
> test cases.
> > If you plan to work on the API test, we have a python test framework:
> https://cwiki.apache.org/CLOUDSTACK/testing-with-python.html. The most
> compelling feature is that it supports code completion for cloudstack API,
> very easy to write test case in Eclipse environment. And we are setting up
> a test infrastructure, which will execute all the test cases written for
> this framework.
> > We already have around 500+ test cases, or maybe more, but apparently
> it's not enough. If you like to work on it, that will be great!
> >
>
> While I like the functional/integration tests, as you note we have
> 500+ currently. Those tests also take many hours to run at present.
> Unit tests should be something fast, a sanity check, and we only have
> 25 right now for the entire code base, so while I like
> func/integration tests, I'd really like for our unit tests to be
> runnable on every commit, and provide some decent level of coverage so
> that we have a bit of confidence that code works well, and provide
> good feedback to folks on whether their commit passes the sniff test.
>
> --David
>
>
Re: unit tests for api/src/com/cloud/agent/api
Posted by David Nalley <da...@gnsa.us>.
On Mon, Sep 17, 2012 at 7:24 PM, Edison Su <Ed...@citrix.com> wrote:
> First, thanks for working on test stuff, we really need more automate test cases.
> If you plan to work on the API test, we have a python test framework: https://cwiki.apache.org/CLOUDSTACK/testing-with-python.html. The most compelling feature is that it supports code completion for cloudstack API, very easy to write test case in Eclipse environment. And we are setting up a test infrastructure, which will execute all the test cases written for this framework.
> We already have around 500+ test cases, or maybe more, but apparently it's not enough. If you like to work on it, that will be great!
>
While I like the functional/integration tests, as you note we have
500+ currently. Those tests also take many hours to run at present.
Unit tests should be something fast, a sanity check, and we only have
25 right now for the entire code base, so while I like
func/integration tests, I'd really like for our unit tests to be
runnable on every commit, and provide some decent level of coverage so
that we have a bit of confidence that code works well, and provide
good feedback to folks on whether their commit passes the sniff test.
--David
RE: unit tests for api/src/com/cloud/agent/api
Posted by Edison Su <Ed...@citrix.com>.
First, thanks for working on test stuff, we really need more automate test cases.
If you plan to work on the API test, we have a python test framework: https://cwiki.apache.org/CLOUDSTACK/testing-with-python.html. The most compelling feature is that it supports code completion for cloudstack API, very easy to write test case in Eclipse environment. And we are setting up a test infrastructure, which will execute all the test cases written for this framework.
We already have around 500+ test cases, or maybe more, but apparently it's not enough. If you like to work on it, that will be great!
> -----Original Message-----
> From: Yichi Lu [mailto:yichi.lu@sungard.com]
> Sent: Monday, September 17, 2012 2:56 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: unit tests for api/src/com/cloud/agent/api
>
> I'd start to write unit tests for classes in
> api/src/com/cloud/agent/api in
> alphabetical order.
>
> Let me know if you have any suggestions.
>
> Yichi Lu