You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Daan Hoogland <DH...@schubergphilis.com> on 2013/05/27 07:49:08 UTC
master build breaks at com.cloud.vpc.NetworkACLServiceTest
LS,
Lately I've been getting, "com.cloud.exception.InvalidParameterValueException: ACL item with number 1 already exists in ACL: null", while building cloudstack. I first thought it was my own hack so I don't know when it was introduced (it is in a clean master that it happens). The full surfire report follows:
-------------------------------------------------------------------------------
Test set: com.cloud.vpc.NetworkACLServiceTest
-------------------------------------------------------------------------------
Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.219 sec <<< FAILURE!
testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest) Time elapsed: 0.016 sec <<< ERROR!
com.cloud.exception.InvalidParameterValueException: ACL item with number 1 already exists in ACL: null
at com.cloud.network.vpc.NetworkACLServiceImpl.createNetworkACLItem(NetworkACLServiceImpl.java:270)
at com.cloud.vpc.NetworkACLServiceTest.testCreateACLItem(NetworkACLServiceTest.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:72)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:231)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:71)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:174)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)
RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Posted by Pranav Saxena <pr...@citrix.com>.
I deployed the latest master today , infact moments back , but build was a success !
-----Original Message-----
From: Chip Childers [mailto:chip.childers@sungard.com]
Sent: Tuesday, May 28, 2013 10:09 PM
To: dev@cloudstack.apache.org
Subject: Re: master build breaks at com.cloud.vpc.NetworkACLServiceTest
On Mon, May 27, 2013 at 04:23:20PM +0000, Daan Hoogland wrote:
> And if not how do I manipulate the order of tests?
IMO, tests should *never* require that they execute in a specific order.
That's what the setup and teardown phases of test execution are for.
If you find this is the issue, let's fix it.
Re: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Posted by Chip Childers <ch...@sungard.com>.
On Mon, May 27, 2013 at 04:23:20PM +0000, Daan Hoogland wrote:
> And if not how do I manipulate the order of tests?
IMO, tests should *never* require that they execute in a specific order.
That's what the setup and teardown phases of test execution are for.
If you find this is the issue, let's fix it.
RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Posted by Daan Hoogland <DH...@schubergphilis.com>.
Kishan,
Master build successfully, thanks
Regards,
Daan Hoogland
-----Original Message-----
From: Kishan Kavala [mailto:Kishan.Kavala@citrix.com]
Sent: woensdag 29 mei 2013 13:48
To: dev@cloudstack.apache.org
Cc: Alex Huang
Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Daan,
I mocked the method causing the failure (I couldn't replicate the error you are seeing though). Please check with latest master and let us know if you still see issues.
~kishan
________________________________________
From: Daan Hoogland [DHoogland@schubergphilis.com]
Sent: Wednesday, May 29, 2013 1:49 PM
To: dev@cloudstack.apache.org
Cc: Alex Huang
Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
H,
I hope this is what Alex means the test programmer needs.
The order that is reported is as follows (I included the server test run only):
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.cloud.alert.AlertControlsUnitTest
log4j:WARN No appenders could be found for logger (com.cloud.alert.AlertControlsUnitTest).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.546 sec Running com.cloud.capacity.CapacityManagerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11 sec Running com.cloud.configuration.ConfigurationManagerTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.499 sec Running com.cloud.configuration.ValidateIpRangeTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 sec Running com.cloud.event.EventControlsUnitTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec Running com.cloud.keystore.KeystoreTest org.apache.cloudstack.api.response.UserVmResponse/null/{"id":"3","securitygroup":[],"nic":[],"tags":[],"affinitygroup":[]}
org.apache.cloudstack.api.response.AlertResponse/null/{"id":"100","description":"Hello"}
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.109 sec Running com.cloud.metadata.ResourceMetaDataManagerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.047 sec Running com.cloud.network.CreatePrivateNetworkTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec Running com.cloud.network.dao.NetworkDaoTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec Running com.cloud.network.DedicateGuestVlanRangesTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 sec Running com.cloud.network.firewall.FirewallManagerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec Running com.cloud.network.NetworkManagerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec Running com.cloud.network.NetworkModelTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec Running com.cloud.network.security.SecurityGroupQueueTest
Num Vms= 50 Queue size = 50
Num Vms= 20000 Queue size = 20000 time=827 ms Num Vms= 5000 Queue size = 5000 time=2544 ms Num Vms= 1 Queue size = 1 time=0 ms Num Vms= 1000000 Queue size = 1000000 time=1171 ms Num Vms= 1 Queue size = 1 time=1140 ms Total jobs dequeued = 10, num queued=1010 queue current size=1000 Total jobs dequeued = 10, num queued=1008 queue current size=998 Total jobs dequeued = 1, num queued=1001 queue current size=1000 Total jobs dequeued = 10, num queued=1000 queue current size=990 Total jobs dequeued = 10, num queued=10 queue current size=0 Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.745 sec Running com.cloud.network.UpdatePhysicalNetworkTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.312 sec Running com.cloud.resourcelimit.ResourceLimitManagerImplTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec Running com.cloud.vm.DeploymentPlanningManagerImplTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.713 sec Running com.cloud.vm.snapshot.VMSnapshotManagerTest
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.203 sec Running com.cloud.vm.UserVmManagerTest Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.64 sec Running com.cloud.vm.VirtualMachineManagerImplTest
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.297 sec Running com.cloud.vpc.NetworkACLManagerTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.406 sec Running com.cloud.vpc.NetworkACLServiceTest
Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.203 sec <<< FAILURE!
Running com.cloud.vpc.Site2SiteVpnTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.491 sec Running com.cloud.vpc.VpcTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.235 sec Running org.apache.cloudstack.affinity.AffinityApiUnitTest
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.188 sec Running org.apache.cloudstack.lb.ApplicationLoadBalancerTest
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.249 sec Running org.apache.cloudstack.networkoffering.CreateNetworkOfferingTest
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.817 sec Running org.apache.cloudstack.privategw.AclOnPrivateGwTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11 sec Running org.apache.cloudstack.region.gslb.GlobalLoadBalancingRulesServiceImplTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.032 sec Running org.apache.cloudstack.region.RegionManagerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
Results :
Tests in error:
testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest): ACL item with number 1 already exists in ACL: null
-----Original Message-----
From: Pranav Saxena [mailto:pranav.saxena@citrix.com]
Sent: woensdag 29 mei 2013 5:58
To: dev@cloudstack.apache.org
Cc: Alex Huang
Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Thanks Alex for throwing some light on this . I have myself seen this when one of the intenalLb tests were failing on my machine and few others but compiling successfully on Alena's machine but that time I also didn't know how this could have happened and I ended up reporting this to her and then after probably some amendments , unit tests compiled on all the machines.
-----Original Message-----
From: Alex Huang [mailto:Alex.Huang@citrix.com]
Sent: Wednesday, May 29, 2013 12:05 AM
To: dev@cloudstack.apache.org
Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Unit tests are ran in random order because each unit test should be self-sufficient. I'm saying I've seen cases where unit tests aren't written to be self-sufficient and that's why an unit test fails sometimes and passes sometimes.
In those cases, we should fix the unit test and not the order. A common problem is sharing the mocked classes. For example, unit test A tests the set of a mock object and unit test B tests if the field is null but they shared the mock instance. In this case, unit test B passes if it is ran before unit test A but fails if it was ran after.
But often times the randomness of how it's ran is actually pseudo-random so it works consistently on someone's laptop but as soon as it gets ran on another machine, the order changes and it fails. So when you see the problem such as this, you should always let the unit test writer know the order in which the unit tests were executed so that the unit test writer can try it.
--Alex
> -----Original Message-----
> From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> Sent: Monday, May 27, 2013 9:23 AM
> To: dev@cloudstack.apache.org
> Subject: RE: master build breaks at
> com.cloud.vpc.NetworkACLServiceTest
>
> And if not how do I manipulate the order of tests?
>
> -----Original Message-----
> From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> Sent: maandag 27 mei 2013 17:02
> To: dev@cloudstack.apache.org
> Subject: RE: master build breaks at
> com.cloud.vpc.NetworkACLServiceTest
>
> Alex,
>
> Can this change out of the box? I had the problem on a clean checkout
> of master.
>
> -----Original Message-----
> From: Alex Huang [mailto:Alex.Huang@citrix.com]
> Sent: maandag 27 mei 2013 15:55
> To: dev@cloudstack.apache.org
> Subject: RE: master build breaks at
> com.cloud.vpc.NetworkACLServiceTest
>
> Daan,
>
> The other possibility is that the test is affected by another test so
> that it breaks when the tests are in a certain order. Have you
> noticed what's the order of the tests ran when it breaks?
>
> --Alex
>
> > -----Original Message-----
> > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > Sent: Monday, May 27, 2013 2:37 AM
> > To: dev@cloudstack.apache.org
> > Subject: RE: master build breaks at
> > com.cloud.vpc.NetworkACLServiceTest
> >
> > Being in the habit of replying on my own mails as I seems to be on
> > this list;
> >
> > This seems to be related to eclipse refresh/build code. Does anybody
> > have a solution to it? The problem is not consistently present but
> > all the more irritating.
> >
> > Regards,
> >
> > -----Original Message-----
> > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > Sent: maandag 27 mei 2013 7:49
> > To: dev@cloudstack.apache.org
> > Subject: master build breaks at com.cloud.vpc.NetworkACLServiceTest
> >
> > LS,
> >
> > Lately I've been getting,
> > "com.cloud.exception.InvalidParameterValueException: ACL item with
> > number 1 already exists in ACL: null", while building cloudstack. I
> > first thought it was my own hack so I don't know when it was
> > introduced (it is in a clean master that it happens). The full
> > surfire report
> follows:
> >
> > --------------------------------------------------------------------
> > --
> > --------- Test set: com.cloud.vpc.NetworkACLServiceTest
> > --------------------------------------------------------------------
> > --
> > --------- Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time
> > elapsed: 0.219 sec <<< FAILURE!
> > testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest) Time elapsed:
> > 0.016 sec <<< ERROR!
> > com.cloud.exception.InvalidParameterValueException: ACL item with
> > number 1 already exists in ACL: null
> > at
> >
> com.cloud.network.vpc.NetworkACLServiceImpl.createNetworkACLItem(Ne
> > tworkACLServiceImpl.java:270)
> > at
> >
> com.cloud.vpc.NetworkACLServiceTest.testCreateACLItem(NetworkACLServ
> > iceTest.java:141)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > Source)
> > at java.lang.reflect.Method.invoke(Unknown Source)
> > at
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(Framework
> > Method.java:45)
> > at
> > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCa
> > ll
> > able.jav
> > a:15)
> > at
> > org.junit.runners.model.FrameworkMethod.invokeExplosively(Framework
> > Method.java:42)
> > at
> >
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMeth
> > od.java:20)
> > at
> > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores
> > .j
> > ava
> > :28)
> > at
> >
> org.springframework.test.context.junit4.statements.RunBeforeTestMethod
> > Callbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
> > at
> >
> org.springframework.test.context.junit4.statements.RunAfterTestMethodC
> > allbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
> > at
> > org.springframework.test.context.junit4.statements.SpringRepeat.eval
> > ua
> > te(
> > SpringRepeat.java:72)
> > at
> > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runC
> > hi
> > ld(S
> > pringJUnit4ClassRunner.java:231)
> > at
> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRu
> > nn
> > er.j
> > ava:47)
> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> > at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> > at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> > at
> > org.springframework.test.context.junit4.statements.RunBeforeTestClas
> > sC
> > all
> > backs.evaluate(RunBeforeTestClassCallbacks.java:61)
> > at
> > org.springframework.test.context.junit4.statements.RunAfterTestClass
> > Ca
> > llb
> > acks.evaluate(RunAfterTestClassCallbacks.java:71)
> > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> > at
> > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(
> > Sp
> > ring
> > JUnit4ClassRunner.java:174)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provid
> > er
> > .jav
> > a:236)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit
> > 4P
> > rov
> > ider.java:134)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.
> > java
> > :113)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > Source)
> > at java.lang.reflect.Method.invoke(Unknown Source)
> > at
> > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray
> > (R
> > efl
> > ectionUtils.java:189)
> > at
> > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invok
> > e(
> > Pr
> > oviderFactory.java:165)
> > at
> > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(Prov
> > id
> > er
> > Factory.java:85)
> > at
> >
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(Forke
> > d
> > Booter.java:103)
> > at
> >
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:7
> > 4)
Re: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Posted by Prashanth Reddy Mandadi <pr...@citrix.com>.
Sent from my HTC
----- Reply message -----
From: "Alex Huang" <Al...@citrix.com>
To: "dev@cloudstack.apache.org" <de...@cloudstack.apache.org>
Subject: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Date: Wed, May 29, 2013 21:50
Daan,
I'm new to mockito as well and I haven't had time to really look at what the right patterns are. In general, in my dealings with other mocking frameworks, I have never liked having the mocking framework returning a default value. I've always like the mocking framework to throw an exception if the method being access was not explicitly set. Not sure if that can be a setting in mockito.
Thanks.
--Alex
> -----Original Message-----
> From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> Sent: Wednesday, May 29, 2013 8:58 AM
> To: 'dev@cloudstack.apache.org'
> Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
>
> Alex,
>
> Thanks for the mockito lesson. I think a lot of us are now going to improve
> their tests.
> Can one make mockito forget the when-conditions? I would intuitively say
> MockitoAnnotations.initMocks(this); would do this.
>
> Alternative to the answer to that last question would be to follow the
> javadoc for when:
> //you can set different behavior for consecutive method calls.
> //Last stubbing (e.g: thenReturn("foo")) determines the behavior of further
> consecutive calls.
> when(mock.someMethod("some arg"))
> .thenThrow(new RuntimeException())
> .thenReturn("foo");
>
> //Alternative, shorter version for consecutive stubbing:
> when(mock.someMethod("some arg"))
> .thenReturn("one", "two");
> //is the same as:
> when(mock.someMethod("some arg"))
> .thenReturn("one")
> .thenReturn("two");
>
> Regards,
> Daan (not new to unit-tests sec but to mockito though) Hoogland
>
> -----Original Message-----
> From: Alex Huang [mailto:Alex.Huang@citrix.com]
> Sent: woensdag 29 mei 2013 16:34
> To: Kishan Kavala; dev@cloudstack.apache.org
> Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
>
> Kishan,
>
> The problem is caused by these two methods. I hope everyone reads this
> because I know some of us are new to writing unit tests. The problem is
> basically as I described in a previous email. In here, both tests makes use of
> and shares of the _networkACLItemDao and the mock is shared because the
> DAO is a singleton and is injected. Now, note that testCreateACLItem()
> invokes the method findByAclAndNumber() via the _networkAclMgr but it
> didn't specifically mock the return value of the method in the test. If this test
> is ran before testCreateACLItemDuplicateNumber(), then it will be successful
> because by default mocks return null. However,
> testCreateACLItemDuplicateNumber() mocks findByAclAndNumber() to
> return a value, so if testCreateACLItem() is ran after
> testCreateACLItemDuplicateNumber(), the test will fail because a value is
> returned and the condition inside _networkAclMgr will fire just as Daan
> found it.
>
> I was hoping maven prints out the order of the tests, but from Daan's log, it
> only printed out the order of the bigger test suite so it's not helpful.
> However, it's not too hard to find. Just go to the code with the error, check
> the method that fails and go back to the test and see if your test mocked that
> method and if it didn't, was it mocked by any other tests.
>
> @Test
> public void testCreateACLItem() throws Exception {
> Mockito.when(_vpcMgr.getVpc(Mockito.anyLong())).thenReturn(new
> VpcVO());
>
> Mockito.when(_networkAclMgr.getNetworkACL(Mockito.anyLong())).thenR
> eturn(acl);
>
> Mockito.when(_networkAclMgr.createNetworkACLItem(Mockito.anyInt(),
> Mockito.anyInt(), Mockito.anyString(), Mockito.anyList(), Mockito.anyInt(),
> Mockito.anyInt(),
> Mockito.any(NetworkACLItem.TrafficType.class), Mockito.anyLong(),
> Mockito.anyString(), Mockito.anyInt())).thenReturn(new
> NetworkACLItemVO());
> assertNotNull(_aclService.createNetworkACLItem(createACLItemCmd));
> }
>
> @Test(expected = InvalidParameterValueException.class)
> public void testCreateACLItemDuplicateNumber() throws Exception {
> Mockito.when(_vpcMgr.getVpc(Mockito.anyLong())).thenReturn(new
> VpcVO());
>
> Mockito.when(_networkAclMgr.getNetworkACL(Mockito.anyLong())).thenR
> eturn(acl);
>
> Mockito.when(_networkACLItemDao.findByAclAndNumber(Mockito.anyLon
> g(), Mockito.anyInt())).thenReturn(new NetworkACLItemVO());
> _aclService.createNetworkACLItem(createACLItemCmd);
> }
>
> > -----Original Message-----
> > From: Kishan Kavala
> > Sent: Wednesday, May 29, 2013 4:48 AM
> > To: dev@cloudstack.apache.org
> > Cc: Alex Huang
> > Subject: RE: master build breaks at
> > com.cloud.vpc.NetworkACLServiceTest
> >
> > Daan,
> > I mocked the method causing the failure (I couldn't replicate the
> > error you are seeing though). Please check with latest master and let
> > us know if you still see issues.
> >
> > ~kishan
> > ________________________________________
> > From: Daan Hoogland [DHoogland@schubergphilis.com]
> > Sent: Wednesday, May 29, 2013 1:49 PM
> > To: dev@cloudstack.apache.org
> > Cc: Alex Huang
> > Subject: RE: master build breaks at
> > com.cloud.vpc.NetworkACLServiceTest
> >
> > H,
> >
> > I hope this is what Alex means the test programmer needs.
> >
> > The order that is reported is as follows (I included the server test run only):
> > -------------------------------------------------------
> > T E S T S
> > -------------------------------------------------------
> > Running com.cloud.alert.AlertControlsUnitTest
> > log4j:WARN No appenders could be found for logger
> > (com.cloud.alert.AlertControlsUnitTest).
> > log4j:WARN Please initialize the log4j system properly.
> > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
> > for more info.
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.546
> > sec Running com.cloud.capacity.CapacityManagerTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11
> > sec Running com.cloud.configuration.ConfigurationManagerTest
> > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.499
> > sec Running com.cloud.configuration.ValidateIpRangeTest
> > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031
> > sec Running com.cloud.event.EventControlsUnitTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
> > Running com.cloud.keystore.KeystoreTest
> > org.apache.cloudstack.api.response.UserVmResponse/null/{"id":"3","secu
> > rit ygroup":[],"nic":[],"tags":[],"affinitygroup":[]}
> > org.apache.cloudstack.api.response.AlertResponse/null/{"id":"100","des
> > crip
> > tion":"Hello"}
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.109
> > sec Running com.cloud.metadata.ResourceMetaDataManagerTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.047
> > sec Running com.cloud.network.CreatePrivateNetworkTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec
> > Running com.cloud.network.dao.NetworkDaoTest
> > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
> > Running com.cloud.network.DedicateGuestVlanRangesTest
> > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031
> > sec Running com.cloud.network.firewall.FirewallManagerTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec
> > Running com.cloud.network.NetworkManagerTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec
> > Running com.cloud.network.NetworkModelTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
> > Running com.cloud.network.security.SecurityGroupQueueTest
> > Num Vms= 50 Queue size = 50
> > Num Vms= 20000 Queue size = 20000 time=827 ms Num Vms= 5000 Queue
> size
> > = 5000 time=2544 ms Num Vms= 1 Queue size = 1 time=0 ms Num Vms=
> > 1000000 Queue size = 1000000 time=1171 ms Num Vms= 1 Queue size = 1
> > time=1140 ms Total jobs dequeued = 10, num queued=1010 queue current
> > size=1000 Total jobs dequeued = 10, num queued=1008 queue current
> > size=998 Total jobs dequeued = 1, num queued=1001 queue current
> > size=1000 Total jobs dequeued = 10, num queued=1000 queue current
> > size=990 Total jobs dequeued = 10, num queued=10 queue current size=0
> > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.745
> > sec Running com.cloud.network.UpdatePhysicalNetworkTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.312
> > sec Running com.cloud.resourcelimit.ResourceLimitManagerImplTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
> > Running com.cloud.vm.DeploymentPlanningManagerImplTest
> > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.713
> > sec Running com.cloud.vm.snapshot.VMSnapshotManagerTest
> > Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.203
> > sec Running com.cloud.vm.UserVmManagerTest Tests run: 10, Failures: 0,
> > Errors: 0,
> > Skipped: 0, Time elapsed: 0.64 sec Running
> > com.cloud.vm.VirtualMachineManagerImplTest
> > Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.297
> > sec Running com.cloud.vpc.NetworkACLManagerTest
> > Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.406
> > sec Running com.cloud.vpc.NetworkACLServiceTest
> > Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.203
> > sec <<< FAILURE!
> > Running com.cloud.vpc.Site2SiteVpnTest Tests run: 1, Failures: 0,
> > Errors: 0, Skipped: 0, Time elapsed: 5.491 sec Running
> > com.cloud.vpc.VpcTest Tests run: 1, Failures: 0, Errors: 0, Skipped:
> > 0, Time
> > elapsed: 0.235 sec Running
> > org.apache.cloudstack.affinity.AffinityApiUnitTest
> > Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.188
> > sec Running org.apache.cloudstack.lb.ApplicationLoadBalancerTest
> > Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.249
> > sec Running
> > org.apache.cloudstack.networkoffering.CreateNetworkOfferingTest
> > Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.817
> > sec Running org.apache.cloudstack.privategw.AclOnPrivateGwTest
> > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11
> > sec Running
> > org.apache.cloudstack.region.gslb.GlobalLoadBalancingRulesServiceImplT
> > est Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > 0.032 sec Running org.apache.cloudstack.region.RegionManagerTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
> >
> > Results :
> >
> > Tests in error:
> > testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest): ACL item
> > with number 1 already exists in ACL: null
> >
> > -----Original Message-----
> > From: Pranav Saxena [mailto:pranav.saxena@citrix.com]
> > Sent: woensdag 29 mei 2013 5:58
> > To: dev@cloudstack.apache.org
> > Cc: Alex Huang
> > Subject: RE: master build breaks at
> > com.cloud.vpc.NetworkACLServiceTest
> >
> > Thanks Alex for throwing some light on this . I have myself seen this
> > when one of the intenalLb tests were failing on my machine and few
> > others but compiling successfully on Alena's machine but that time I
> > also didn't know how this could have happened and I ended up reporting
> > this to her and then after probably some amendments , unit tests compiled
> on all the machines.
> >
> > -----Original Message-----
> > From: Alex Huang [mailto:Alex.Huang@citrix.com]
> > Sent: Wednesday, May 29, 2013 12:05 AM
> > To: dev@cloudstack.apache.org
> > Subject: RE: master build breaks at
> > com.cloud.vpc.NetworkACLServiceTest
> >
> > Unit tests are ran in random order because each unit test should be
> > self- sufficient. I'm saying I've seen cases where unit tests aren't
> > written to be self-sufficient and that's why an unit test fails
> > sometimes and passes sometimes.
> >
> > In those cases, we should fix the unit test and not the order. A
> > common problem is sharing the mocked classes. For example, unit test
> > A tests the set of a mock object and unit test B tests if the field is
> > null but they shared the mock instance. In this case, unit test B
> > passes if it is ran before unit test A but fails if it was ran after.
> >
> > But often times the randomness of how it's ran is actually
> > pseudo-random so it works consistently on someone's laptop but as soon
> > as it gets ran on another machine, the order changes and it fails. So
> > when you see the problem such as this, you should always let the unit
> > test writer know the order in which the unit tests were executed so
> > that the unit test writer can try it.
> >
> > --Alex
> >
> > > -----Original Message-----
> > > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > > Sent: Monday, May 27, 2013 9:23 AM
> > > To: dev@cloudstack.apache.org
> > > Subject: RE: master build breaks at
> > > com.cloud.vpc.NetworkACLServiceTest
> > >
> > > And if not how do I manipulate the order of tests?
> > >
> > > -----Original Message-----
> > > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > > Sent: maandag 27 mei 2013 17:02
> > > To: dev@cloudstack.apache.org
> > > Subject: RE: master build breaks at
> > > com.cloud.vpc.NetworkACLServiceTest
> > >
> > > Alex,
> > >
> > > Can this change out of the box? I had the problem on a clean
> > > checkout of master.
> > >
> > > -----Original Message-----
> > > From: Alex Huang [mailto:Alex.Huang@citrix.com]
> > > Sent: maandag 27 mei 2013 15:55
> > > To: dev@cloudstack.apache.org
> > > Subject: RE: master build breaks at
> > > com.cloud.vpc.NetworkACLServiceTest
> > >
> > > Daan,
> > >
> > > The other possibility is that the test is affected by another test
> > > so that it breaks when the tests are in a certain order. Have you
> > > noticed what's the order of the tests ran when it breaks?
> > >
> > > --Alex
> > >
> > > > -----Original Message-----
> > > > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > > > Sent: Monday, May 27, 2013 2:37 AM
> > > > To: dev@cloudstack.apache.org
> > > > Subject: RE: master build breaks at
> > > > com.cloud.vpc.NetworkACLServiceTest
> > > >
> > > > Being in the habit of replying on my own mails as I seems to be on
> > > > this list;
> > > >
> > > > This seems to be related to eclipse refresh/build code. Does
> > > > anybody have a solution to it? The problem is not consistently
> > > > present but all the more irritating.
> > > >
> > > > Regards,
> > > >
> > > > -----Original Message-----
> > > > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > > > Sent: maandag 27 mei 2013 7:49
> > > > To: dev@cloudstack.apache.org
> > > > Subject: master build breaks at
> > > > com.cloud.vpc.NetworkACLServiceTest
> > > >
> > > > LS,
> > > >
> > > > Lately I've been getting,
> > > > "com.cloud.exception.InvalidParameterValueException: ACL item with
> > > > number 1 already exists in ACL: null", while building cloudstack.
> > > > I first thought it was my own hack so I don't know when it was
> > > > introduced (it is in a clean master that it happens). The full
> > > > surfire report
> > > follows:
> > > >
> > > > ------------------------------------------------------------------
> > > > --
> > > > --
> > > > --------- Test set: com.cloud.vpc.NetworkACLServiceTest
> > > > ------------------------------------------------------------------
> > > > --
> > > > --
> > > > --------- Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time
> > > > elapsed: 0.219 sec <<< FAILURE!
> > > > testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest) Time
> > elapsed:
> > > > 0.016 sec <<< ERROR!
> > > > com.cloud.exception.InvalidParameterValueException: ACL item with
> > > > number 1 already exists in ACL: null
> > > > at
> > > >
> > >
> >
> com.cloud.network.vpc.NetworkACLServiceImpl.createNetworkACLItem(Ne
> > > > tworkACLServiceImpl.java:270)
> > > > at
> > > >
> > >
> >
> com.cloud.vpc.NetworkACLServiceTest.testCreateACLItem(NetworkACLServ
> > > > iceTest.java:141)
> > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> > > > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown
> Source)
> > > > at
> > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > > > Source)
> > > > at java.lang.reflect.Method.invoke(Unknown Source)
> > > > at
> > > >
> > >
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(Framework
> > > > Method.java:45)
> > > > at
> > > > org.junit.internal.runners.model.ReflectiveCallable.run(Reflective
> > > > Ca
> > > > ll
> > > > able.jav
> > > > a:15)
> > > > at
> > > >
> > org.junit.runners.model.FrameworkMethod.invokeExplosively(Framework
> > > > Method.java:42)
> > > > at
> > > >
> > >
> >
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMeth
> > > > od.java:20)
> > > > at
> > > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefor
> > > > es
> > > > .j
> > > > ava
> > > > :28)
> > > > at
> > > >
> > >
> >
> org.springframework.test.context.junit4.statements.RunBeforeTestMethod
> > > > Callbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
> > > > at
> > > >
> > >
> >
> org.springframework.test.context.junit4.statements.RunAfterTestMethodC
> > > > allbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
> > > > at
> > > > org.springframework.test.context.junit4.statements.SpringRepeat.ev
> > > > al
> > > > ua
> > > > te(
> > > > SpringRepeat.java:72)
> > > > at
> > > > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.ru
> > > > nC
> > > > hi
> > > > ld(S
> > > > pringJUnit4ClassRunner.java:231)
> > > > at
> > > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4Class
> > > > Ru
> > > > nn
> > > > er.j
> > > > ava:47)
> > > > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> > > > at
> > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> > > > at
> > > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> > > > at
> > org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> > > > at
> > > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> > > > at
> > > > org.springframework.test.context.junit4.statements.RunBeforeTestCl
> > > > as
> > > > sC
> > > > all
> > > > backs.evaluate(RunBeforeTestClassCallbacks.java:61)
> > > > at
> > > > org.springframework.test.context.junit4.statements.RunAfterTestCla
> > > > ss
> > > > Ca
> > > > llb
> > > > acks.evaluate(RunAfterTestClassCallbacks.java:71)
> > > > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> > > > at
> > > > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.ru
> > > > n(
> > > > Sp
> > > > ring
> > > > JUnit4ClassRunner.java:174)
> > > > at
> > > > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Prov
> > > > id
> > > > er
> > > > .jav
> > > > a:236)
> > > > at
> > > > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUn
> > > > it
> > > > 4P
> > > > rov
> > > > ider.java:134)
> > > > at
> > > >
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.
> > > > java
> > > > :113)
> > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> > > > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown
> Source)
> > > > at
> > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > > > Source)
> > > > at java.lang.reflect.Method.invoke(Unknown Source)
> > > > at
> > > > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArr
> > > > ay
> > > > (R
> > > > efl
> > > > ectionUtils.java:189)
> > > > at
> > > > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.inv
> > > > ok
> > > > e(
> > > > Pr
> > > > oviderFactory.java:165)
> > > > at
> > > > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(Pr
> > > > ov
> > > > id
> > > > er
> > > > Factory.java:85)
> > > > at
> > > >
> > >
> >
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(Forke
> > > > d
> > > > Booter.java:103)
> > > > at
> > > >
> > >
> >
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:7
> > > > 4)
RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Posted by Alex Huang <Al...@citrix.com>.
Daan,
I'm new to mockito as well and I haven't had time to really look at what the right patterns are. In general, in my dealings with other mocking frameworks, I have never liked having the mocking framework returning a default value. I've always like the mocking framework to throw an exception if the method being access was not explicitly set. Not sure if that can be a setting in mockito.
Thanks.
--Alex
> -----Original Message-----
> From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> Sent: Wednesday, May 29, 2013 8:58 AM
> To: 'dev@cloudstack.apache.org'
> Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
>
> Alex,
>
> Thanks for the mockito lesson. I think a lot of us are now going to improve
> their tests.
> Can one make mockito forget the when-conditions? I would intuitively say
> MockitoAnnotations.initMocks(this); would do this.
>
> Alternative to the answer to that last question would be to follow the
> javadoc for when:
> //you can set different behavior for consecutive method calls.
> //Last stubbing (e.g: thenReturn("foo")) determines the behavior of further
> consecutive calls.
> when(mock.someMethod("some arg"))
> .thenThrow(new RuntimeException())
> .thenReturn("foo");
>
> //Alternative, shorter version for consecutive stubbing:
> when(mock.someMethod("some arg"))
> .thenReturn("one", "two");
> //is the same as:
> when(mock.someMethod("some arg"))
> .thenReturn("one")
> .thenReturn("two");
>
> Regards,
> Daan (not new to unit-tests sec but to mockito though) Hoogland
>
> -----Original Message-----
> From: Alex Huang [mailto:Alex.Huang@citrix.com]
> Sent: woensdag 29 mei 2013 16:34
> To: Kishan Kavala; dev@cloudstack.apache.org
> Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
>
> Kishan,
>
> The problem is caused by these two methods. I hope everyone reads this
> because I know some of us are new to writing unit tests. The problem is
> basically as I described in a previous email. In here, both tests makes use of
> and shares of the _networkACLItemDao and the mock is shared because the
> DAO is a singleton and is injected. Now, note that testCreateACLItem()
> invokes the method findByAclAndNumber() via the _networkAclMgr but it
> didn't specifically mock the return value of the method in the test. If this test
> is ran before testCreateACLItemDuplicateNumber(), then it will be successful
> because by default mocks return null. However,
> testCreateACLItemDuplicateNumber() mocks findByAclAndNumber() to
> return a value, so if testCreateACLItem() is ran after
> testCreateACLItemDuplicateNumber(), the test will fail because a value is
> returned and the condition inside _networkAclMgr will fire just as Daan
> found it.
>
> I was hoping maven prints out the order of the tests, but from Daan's log, it
> only printed out the order of the bigger test suite so it's not helpful.
> However, it's not too hard to find. Just go to the code with the error, check
> the method that fails and go back to the test and see if your test mocked that
> method and if it didn't, was it mocked by any other tests.
>
> @Test
> public void testCreateACLItem() throws Exception {
> Mockito.when(_vpcMgr.getVpc(Mockito.anyLong())).thenReturn(new
> VpcVO());
>
> Mockito.when(_networkAclMgr.getNetworkACL(Mockito.anyLong())).thenR
> eturn(acl);
>
> Mockito.when(_networkAclMgr.createNetworkACLItem(Mockito.anyInt(),
> Mockito.anyInt(), Mockito.anyString(), Mockito.anyList(), Mockito.anyInt(),
> Mockito.anyInt(),
> Mockito.any(NetworkACLItem.TrafficType.class), Mockito.anyLong(),
> Mockito.anyString(), Mockito.anyInt())).thenReturn(new
> NetworkACLItemVO());
> assertNotNull(_aclService.createNetworkACLItem(createACLItemCmd));
> }
>
> @Test(expected = InvalidParameterValueException.class)
> public void testCreateACLItemDuplicateNumber() throws Exception {
> Mockito.when(_vpcMgr.getVpc(Mockito.anyLong())).thenReturn(new
> VpcVO());
>
> Mockito.when(_networkAclMgr.getNetworkACL(Mockito.anyLong())).thenR
> eturn(acl);
>
> Mockito.when(_networkACLItemDao.findByAclAndNumber(Mockito.anyLon
> g(), Mockito.anyInt())).thenReturn(new NetworkACLItemVO());
> _aclService.createNetworkACLItem(createACLItemCmd);
> }
>
> > -----Original Message-----
> > From: Kishan Kavala
> > Sent: Wednesday, May 29, 2013 4:48 AM
> > To: dev@cloudstack.apache.org
> > Cc: Alex Huang
> > Subject: RE: master build breaks at
> > com.cloud.vpc.NetworkACLServiceTest
> >
> > Daan,
> > I mocked the method causing the failure (I couldn't replicate the
> > error you are seeing though). Please check with latest master and let
> > us know if you still see issues.
> >
> > ~kishan
> > ________________________________________
> > From: Daan Hoogland [DHoogland@schubergphilis.com]
> > Sent: Wednesday, May 29, 2013 1:49 PM
> > To: dev@cloudstack.apache.org
> > Cc: Alex Huang
> > Subject: RE: master build breaks at
> > com.cloud.vpc.NetworkACLServiceTest
> >
> > H,
> >
> > I hope this is what Alex means the test programmer needs.
> >
> > The order that is reported is as follows (I included the server test run only):
> > -------------------------------------------------------
> > T E S T S
> > -------------------------------------------------------
> > Running com.cloud.alert.AlertControlsUnitTest
> > log4j:WARN No appenders could be found for logger
> > (com.cloud.alert.AlertControlsUnitTest).
> > log4j:WARN Please initialize the log4j system properly.
> > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
> > for more info.
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.546
> > sec Running com.cloud.capacity.CapacityManagerTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11
> > sec Running com.cloud.configuration.ConfigurationManagerTest
> > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.499
> > sec Running com.cloud.configuration.ValidateIpRangeTest
> > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031
> > sec Running com.cloud.event.EventControlsUnitTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
> > Running com.cloud.keystore.KeystoreTest
> > org.apache.cloudstack.api.response.UserVmResponse/null/{"id":"3","secu
> > rit ygroup":[],"nic":[],"tags":[],"affinitygroup":[]}
> > org.apache.cloudstack.api.response.AlertResponse/null/{"id":"100","des
> > crip
> > tion":"Hello"}
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.109
> > sec Running com.cloud.metadata.ResourceMetaDataManagerTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.047
> > sec Running com.cloud.network.CreatePrivateNetworkTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec
> > Running com.cloud.network.dao.NetworkDaoTest
> > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
> > Running com.cloud.network.DedicateGuestVlanRangesTest
> > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031
> > sec Running com.cloud.network.firewall.FirewallManagerTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec
> > Running com.cloud.network.NetworkManagerTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec
> > Running com.cloud.network.NetworkModelTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
> > Running com.cloud.network.security.SecurityGroupQueueTest
> > Num Vms= 50 Queue size = 50
> > Num Vms= 20000 Queue size = 20000 time=827 ms Num Vms= 5000 Queue
> size
> > = 5000 time=2544 ms Num Vms= 1 Queue size = 1 time=0 ms Num Vms=
> > 1000000 Queue size = 1000000 time=1171 ms Num Vms= 1 Queue size = 1
> > time=1140 ms Total jobs dequeued = 10, num queued=1010 queue current
> > size=1000 Total jobs dequeued = 10, num queued=1008 queue current
> > size=998 Total jobs dequeued = 1, num queued=1001 queue current
> > size=1000 Total jobs dequeued = 10, num queued=1000 queue current
> > size=990 Total jobs dequeued = 10, num queued=10 queue current size=0
> > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.745
> > sec Running com.cloud.network.UpdatePhysicalNetworkTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.312
> > sec Running com.cloud.resourcelimit.ResourceLimitManagerImplTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
> > Running com.cloud.vm.DeploymentPlanningManagerImplTest
> > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.713
> > sec Running com.cloud.vm.snapshot.VMSnapshotManagerTest
> > Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.203
> > sec Running com.cloud.vm.UserVmManagerTest Tests run: 10, Failures: 0,
> > Errors: 0,
> > Skipped: 0, Time elapsed: 0.64 sec Running
> > com.cloud.vm.VirtualMachineManagerImplTest
> > Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.297
> > sec Running com.cloud.vpc.NetworkACLManagerTest
> > Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.406
> > sec Running com.cloud.vpc.NetworkACLServiceTest
> > Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.203
> > sec <<< FAILURE!
> > Running com.cloud.vpc.Site2SiteVpnTest Tests run: 1, Failures: 0,
> > Errors: 0, Skipped: 0, Time elapsed: 5.491 sec Running
> > com.cloud.vpc.VpcTest Tests run: 1, Failures: 0, Errors: 0, Skipped:
> > 0, Time
> > elapsed: 0.235 sec Running
> > org.apache.cloudstack.affinity.AffinityApiUnitTest
> > Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.188
> > sec Running org.apache.cloudstack.lb.ApplicationLoadBalancerTest
> > Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.249
> > sec Running
> > org.apache.cloudstack.networkoffering.CreateNetworkOfferingTest
> > Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.817
> > sec Running org.apache.cloudstack.privategw.AclOnPrivateGwTest
> > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11
> > sec Running
> > org.apache.cloudstack.region.gslb.GlobalLoadBalancingRulesServiceImplT
> > est Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > 0.032 sec Running org.apache.cloudstack.region.RegionManagerTest
> > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
> >
> > Results :
> >
> > Tests in error:
> > testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest): ACL item
> > with number 1 already exists in ACL: null
> >
> > -----Original Message-----
> > From: Pranav Saxena [mailto:pranav.saxena@citrix.com]
> > Sent: woensdag 29 mei 2013 5:58
> > To: dev@cloudstack.apache.org
> > Cc: Alex Huang
> > Subject: RE: master build breaks at
> > com.cloud.vpc.NetworkACLServiceTest
> >
> > Thanks Alex for throwing some light on this . I have myself seen this
> > when one of the intenalLb tests were failing on my machine and few
> > others but compiling successfully on Alena's machine but that time I
> > also didn't know how this could have happened and I ended up reporting
> > this to her and then after probably some amendments , unit tests compiled
> on all the machines.
> >
> > -----Original Message-----
> > From: Alex Huang [mailto:Alex.Huang@citrix.com]
> > Sent: Wednesday, May 29, 2013 12:05 AM
> > To: dev@cloudstack.apache.org
> > Subject: RE: master build breaks at
> > com.cloud.vpc.NetworkACLServiceTest
> >
> > Unit tests are ran in random order because each unit test should be
> > self- sufficient. I'm saying I've seen cases where unit tests aren't
> > written to be self-sufficient and that's why an unit test fails
> > sometimes and passes sometimes.
> >
> > In those cases, we should fix the unit test and not the order. A
> > common problem is sharing the mocked classes. For example, unit test
> > A tests the set of a mock object and unit test B tests if the field is
> > null but they shared the mock instance. In this case, unit test B
> > passes if it is ran before unit test A but fails if it was ran after.
> >
> > But often times the randomness of how it's ran is actually
> > pseudo-random so it works consistently on someone's laptop but as soon
> > as it gets ran on another machine, the order changes and it fails. So
> > when you see the problem such as this, you should always let the unit
> > test writer know the order in which the unit tests were executed so
> > that the unit test writer can try it.
> >
> > --Alex
> >
> > > -----Original Message-----
> > > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > > Sent: Monday, May 27, 2013 9:23 AM
> > > To: dev@cloudstack.apache.org
> > > Subject: RE: master build breaks at
> > > com.cloud.vpc.NetworkACLServiceTest
> > >
> > > And if not how do I manipulate the order of tests?
> > >
> > > -----Original Message-----
> > > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > > Sent: maandag 27 mei 2013 17:02
> > > To: dev@cloudstack.apache.org
> > > Subject: RE: master build breaks at
> > > com.cloud.vpc.NetworkACLServiceTest
> > >
> > > Alex,
> > >
> > > Can this change out of the box? I had the problem on a clean
> > > checkout of master.
> > >
> > > -----Original Message-----
> > > From: Alex Huang [mailto:Alex.Huang@citrix.com]
> > > Sent: maandag 27 mei 2013 15:55
> > > To: dev@cloudstack.apache.org
> > > Subject: RE: master build breaks at
> > > com.cloud.vpc.NetworkACLServiceTest
> > >
> > > Daan,
> > >
> > > The other possibility is that the test is affected by another test
> > > so that it breaks when the tests are in a certain order. Have you
> > > noticed what's the order of the tests ran when it breaks?
> > >
> > > --Alex
> > >
> > > > -----Original Message-----
> > > > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > > > Sent: Monday, May 27, 2013 2:37 AM
> > > > To: dev@cloudstack.apache.org
> > > > Subject: RE: master build breaks at
> > > > com.cloud.vpc.NetworkACLServiceTest
> > > >
> > > > Being in the habit of replying on my own mails as I seems to be on
> > > > this list;
> > > >
> > > > This seems to be related to eclipse refresh/build code. Does
> > > > anybody have a solution to it? The problem is not consistently
> > > > present but all the more irritating.
> > > >
> > > > Regards,
> > > >
> > > > -----Original Message-----
> > > > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > > > Sent: maandag 27 mei 2013 7:49
> > > > To: dev@cloudstack.apache.org
> > > > Subject: master build breaks at
> > > > com.cloud.vpc.NetworkACLServiceTest
> > > >
> > > > LS,
> > > >
> > > > Lately I've been getting,
> > > > "com.cloud.exception.InvalidParameterValueException: ACL item with
> > > > number 1 already exists in ACL: null", while building cloudstack.
> > > > I first thought it was my own hack so I don't know when it was
> > > > introduced (it is in a clean master that it happens). The full
> > > > surfire report
> > > follows:
> > > >
> > > > ------------------------------------------------------------------
> > > > --
> > > > --
> > > > --------- Test set: com.cloud.vpc.NetworkACLServiceTest
> > > > ------------------------------------------------------------------
> > > > --
> > > > --
> > > > --------- Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time
> > > > elapsed: 0.219 sec <<< FAILURE!
> > > > testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest) Time
> > elapsed:
> > > > 0.016 sec <<< ERROR!
> > > > com.cloud.exception.InvalidParameterValueException: ACL item with
> > > > number 1 already exists in ACL: null
> > > > at
> > > >
> > >
> >
> com.cloud.network.vpc.NetworkACLServiceImpl.createNetworkACLItem(Ne
> > > > tworkACLServiceImpl.java:270)
> > > > at
> > > >
> > >
> >
> com.cloud.vpc.NetworkACLServiceTest.testCreateACLItem(NetworkACLServ
> > > > iceTest.java:141)
> > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> > > > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown
> Source)
> > > > at
> > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > > > Source)
> > > > at java.lang.reflect.Method.invoke(Unknown Source)
> > > > at
> > > >
> > >
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(Framework
> > > > Method.java:45)
> > > > at
> > > > org.junit.internal.runners.model.ReflectiveCallable.run(Reflective
> > > > Ca
> > > > ll
> > > > able.jav
> > > > a:15)
> > > > at
> > > >
> > org.junit.runners.model.FrameworkMethod.invokeExplosively(Framework
> > > > Method.java:42)
> > > > at
> > > >
> > >
> >
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMeth
> > > > od.java:20)
> > > > at
> > > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefor
> > > > es
> > > > .j
> > > > ava
> > > > :28)
> > > > at
> > > >
> > >
> >
> org.springframework.test.context.junit4.statements.RunBeforeTestMethod
> > > > Callbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
> > > > at
> > > >
> > >
> >
> org.springframework.test.context.junit4.statements.RunAfterTestMethodC
> > > > allbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
> > > > at
> > > > org.springframework.test.context.junit4.statements.SpringRepeat.ev
> > > > al
> > > > ua
> > > > te(
> > > > SpringRepeat.java:72)
> > > > at
> > > > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.ru
> > > > nC
> > > > hi
> > > > ld(S
> > > > pringJUnit4ClassRunner.java:231)
> > > > at
> > > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4Class
> > > > Ru
> > > > nn
> > > > er.j
> > > > ava:47)
> > > > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> > > > at
> > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> > > > at
> > > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> > > > at
> > org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> > > > at
> > > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> > > > at
> > > > org.springframework.test.context.junit4.statements.RunBeforeTestCl
> > > > as
> > > > sC
> > > > all
> > > > backs.evaluate(RunBeforeTestClassCallbacks.java:61)
> > > > at
> > > > org.springframework.test.context.junit4.statements.RunAfterTestCla
> > > > ss
> > > > Ca
> > > > llb
> > > > acks.evaluate(RunAfterTestClassCallbacks.java:71)
> > > > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> > > > at
> > > > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.ru
> > > > n(
> > > > Sp
> > > > ring
> > > > JUnit4ClassRunner.java:174)
> > > > at
> > > > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Prov
> > > > id
> > > > er
> > > > .jav
> > > > a:236)
> > > > at
> > > > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUn
> > > > it
> > > > 4P
> > > > rov
> > > > ider.java:134)
> > > > at
> > > >
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.
> > > > java
> > > > :113)
> > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> > > > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown
> Source)
> > > > at
> > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > > > Source)
> > > > at java.lang.reflect.Method.invoke(Unknown Source)
> > > > at
> > > > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArr
> > > > ay
> > > > (R
> > > > efl
> > > > ectionUtils.java:189)
> > > > at
> > > > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.inv
> > > > ok
> > > > e(
> > > > Pr
> > > > oviderFactory.java:165)
> > > > at
> > > > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(Pr
> > > > ov
> > > > id
> > > > er
> > > > Factory.java:85)
> > > > at
> > > >
> > >
> >
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(Forke
> > > > d
> > > > Booter.java:103)
> > > > at
> > > >
> > >
> >
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:7
> > > > 4)
RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Posted by Daan Hoogland <DH...@schubergphilis.com>.
Alex,
Thanks for the mockito lesson. I think a lot of us are now going to improve their tests.
Can one make mockito forget the when-conditions? I would intuitively say MockitoAnnotations.initMocks(this); would do this.
Alternative to the answer to that last question would be to follow the javadoc for when:
//you can set different behavior for consecutive method calls.
//Last stubbing (e.g: thenReturn("foo")) determines the behavior of further consecutive calls.
when(mock.someMethod("some arg"))
.thenThrow(new RuntimeException())
.thenReturn("foo");
//Alternative, shorter version for consecutive stubbing:
when(mock.someMethod("some arg"))
.thenReturn("one", "two");
//is the same as:
when(mock.someMethod("some arg"))
.thenReturn("one")
.thenReturn("two");
Regards,
Daan (not new to unit-tests sec but to mockito though) Hoogland
-----Original Message-----
From: Alex Huang [mailto:Alex.Huang@citrix.com]
Sent: woensdag 29 mei 2013 16:34
To: Kishan Kavala; dev@cloudstack.apache.org
Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Kishan,
The problem is caused by these two methods. I hope everyone reads this because I know some of us are new to writing unit tests. The problem is basically as I described in a previous email. In here, both tests makes use of and shares of the _networkACLItemDao and the mock is shared because the DAO is a singleton and is injected. Now, note that testCreateACLItem() invokes the method findByAclAndNumber() via the _networkAclMgr but it didn't specifically mock the return value of the method in the test. If this test is ran before testCreateACLItemDuplicateNumber(), then it will be successful because by default mocks return null. However, testCreateACLItemDuplicateNumber() mocks findByAclAndNumber() to return a value, so if testCreateACLItem() is ran after testCreateACLItemDuplicateNumber(), the test will fail because a value is returned and the condition inside _networkAclMgr will fire just as Daan found it.
I was hoping maven prints out the order of the tests, but from Daan's log, it only printed out the order of the bigger test suite so it's not helpful. However, it's not too hard to find. Just go to the code with the error, check the method that fails and go back to the test and see if your test mocked that method and if it didn't, was it mocked by any other tests.
@Test
public void testCreateACLItem() throws Exception {
Mockito.when(_vpcMgr.getVpc(Mockito.anyLong())).thenReturn(new VpcVO());
Mockito.when(_networkAclMgr.getNetworkACL(Mockito.anyLong())).thenReturn(acl);
Mockito.when(_networkAclMgr.createNetworkACLItem(Mockito.anyInt(), Mockito.anyInt(), Mockito.anyString(), Mockito.anyList(), Mockito.anyInt(), Mockito.anyInt(),
Mockito.any(NetworkACLItem.TrafficType.class), Mockito.anyLong(), Mockito.anyString(), Mockito.anyInt())).thenReturn(new NetworkACLItemVO());
assertNotNull(_aclService.createNetworkACLItem(createACLItemCmd));
}
@Test(expected = InvalidParameterValueException.class)
public void testCreateACLItemDuplicateNumber() throws Exception {
Mockito.when(_vpcMgr.getVpc(Mockito.anyLong())).thenReturn(new VpcVO());
Mockito.when(_networkAclMgr.getNetworkACL(Mockito.anyLong())).thenReturn(acl);
Mockito.when(_networkACLItemDao.findByAclAndNumber(Mockito.anyLong(), Mockito.anyInt())).thenReturn(new NetworkACLItemVO());
_aclService.createNetworkACLItem(createACLItemCmd);
}
> -----Original Message-----
> From: Kishan Kavala
> Sent: Wednesday, May 29, 2013 4:48 AM
> To: dev@cloudstack.apache.org
> Cc: Alex Huang
> Subject: RE: master build breaks at
> com.cloud.vpc.NetworkACLServiceTest
>
> Daan,
> I mocked the method causing the failure (I couldn't replicate the
> error you are seeing though). Please check with latest master and let
> us know if you still see issues.
>
> ~kishan
> ________________________________________
> From: Daan Hoogland [DHoogland@schubergphilis.com]
> Sent: Wednesday, May 29, 2013 1:49 PM
> To: dev@cloudstack.apache.org
> Cc: Alex Huang
> Subject: RE: master build breaks at
> com.cloud.vpc.NetworkACLServiceTest
>
> H,
>
> I hope this is what Alex means the test programmer needs.
>
> The order that is reported is as follows (I included the server test run only):
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Running com.cloud.alert.AlertControlsUnitTest
> log4j:WARN No appenders could be found for logger
> (com.cloud.alert.AlertControlsUnitTest).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
> for more info.
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.546
> sec Running com.cloud.capacity.CapacityManagerTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11
> sec Running com.cloud.configuration.ConfigurationManagerTest
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.499
> sec Running com.cloud.configuration.ValidateIpRangeTest
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031
> sec Running com.cloud.event.EventControlsUnitTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
> Running com.cloud.keystore.KeystoreTest
> org.apache.cloudstack.api.response.UserVmResponse/null/{"id":"3","secu
> rit ygroup":[],"nic":[],"tags":[],"affinitygroup":[]}
> org.apache.cloudstack.api.response.AlertResponse/null/{"id":"100","des
> crip
> tion":"Hello"}
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.109
> sec Running com.cloud.metadata.ResourceMetaDataManagerTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.047
> sec Running com.cloud.network.CreatePrivateNetworkTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec
> Running com.cloud.network.dao.NetworkDaoTest
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
> Running com.cloud.network.DedicateGuestVlanRangesTest
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031
> sec Running com.cloud.network.firewall.FirewallManagerTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec
> Running com.cloud.network.NetworkManagerTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec
> Running com.cloud.network.NetworkModelTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
> Running com.cloud.network.security.SecurityGroupQueueTest
> Num Vms= 50 Queue size = 50
> Num Vms= 20000 Queue size = 20000 time=827 ms Num Vms= 5000 Queue size
> = 5000 time=2544 ms Num Vms= 1 Queue size = 1 time=0 ms Num Vms=
> 1000000 Queue size = 1000000 time=1171 ms Num Vms= 1 Queue size = 1
> time=1140 ms Total jobs dequeued = 10, num queued=1010 queue current
> size=1000 Total jobs dequeued = 10, num queued=1008 queue current
> size=998 Total jobs dequeued = 1, num queued=1001 queue current
> size=1000 Total jobs dequeued = 10, num queued=1000 queue current
> size=990 Total jobs dequeued = 10, num queued=10 queue current size=0
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.745
> sec Running com.cloud.network.UpdatePhysicalNetworkTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.312
> sec Running com.cloud.resourcelimit.ResourceLimitManagerImplTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
> Running com.cloud.vm.DeploymentPlanningManagerImplTest
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.713
> sec Running com.cloud.vm.snapshot.VMSnapshotManagerTest
> Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.203
> sec Running com.cloud.vm.UserVmManagerTest Tests run: 10, Failures: 0,
> Errors: 0,
> Skipped: 0, Time elapsed: 0.64 sec Running
> com.cloud.vm.VirtualMachineManagerImplTest
> Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.297
> sec Running com.cloud.vpc.NetworkACLManagerTest
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.406
> sec Running com.cloud.vpc.NetworkACLServiceTest
> Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.203
> sec <<< FAILURE!
> Running com.cloud.vpc.Site2SiteVpnTest Tests run: 1, Failures: 0,
> Errors: 0, Skipped: 0, Time elapsed: 5.491 sec Running
> com.cloud.vpc.VpcTest Tests run: 1, Failures: 0, Errors: 0, Skipped:
> 0, Time
> elapsed: 0.235 sec Running
> org.apache.cloudstack.affinity.AffinityApiUnitTest
> Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.188
> sec Running org.apache.cloudstack.lb.ApplicationLoadBalancerTest
> Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.249
> sec Running
> org.apache.cloudstack.networkoffering.CreateNetworkOfferingTest
> Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.817
> sec Running org.apache.cloudstack.privategw.AclOnPrivateGwTest
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11
> sec Running
> org.apache.cloudstack.region.gslb.GlobalLoadBalancingRulesServiceImplT
> est Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 0.032 sec Running org.apache.cloudstack.region.RegionManagerTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
>
> Results :
>
> Tests in error:
> testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest): ACL item
> with number 1 already exists in ACL: null
>
> -----Original Message-----
> From: Pranav Saxena [mailto:pranav.saxena@citrix.com]
> Sent: woensdag 29 mei 2013 5:58
> To: dev@cloudstack.apache.org
> Cc: Alex Huang
> Subject: RE: master build breaks at
> com.cloud.vpc.NetworkACLServiceTest
>
> Thanks Alex for throwing some light on this . I have myself seen this
> when one of the intenalLb tests were failing on my machine and few
> others but compiling successfully on Alena's machine but that time I
> also didn't know how this could have happened and I ended up reporting
> this to her and then after probably some amendments , unit tests compiled on all the machines.
>
> -----Original Message-----
> From: Alex Huang [mailto:Alex.Huang@citrix.com]
> Sent: Wednesday, May 29, 2013 12:05 AM
> To: dev@cloudstack.apache.org
> Subject: RE: master build breaks at
> com.cloud.vpc.NetworkACLServiceTest
>
> Unit tests are ran in random order because each unit test should be
> self- sufficient. I'm saying I've seen cases where unit tests aren't
> written to be self-sufficient and that's why an unit test fails
> sometimes and passes sometimes.
>
> In those cases, we should fix the unit test and not the order. A
> common problem is sharing the mocked classes. For example, unit test
> A tests the set of a mock object and unit test B tests if the field is
> null but they shared the mock instance. In this case, unit test B
> passes if it is ran before unit test A but fails if it was ran after.
>
> But often times the randomness of how it's ran is actually
> pseudo-random so it works consistently on someone's laptop but as soon
> as it gets ran on another machine, the order changes and it fails. So
> when you see the problem such as this, you should always let the unit
> test writer know the order in which the unit tests were executed so
> that the unit test writer can try it.
>
> --Alex
>
> > -----Original Message-----
> > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > Sent: Monday, May 27, 2013 9:23 AM
> > To: dev@cloudstack.apache.org
> > Subject: RE: master build breaks at
> > com.cloud.vpc.NetworkACLServiceTest
> >
> > And if not how do I manipulate the order of tests?
> >
> > -----Original Message-----
> > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > Sent: maandag 27 mei 2013 17:02
> > To: dev@cloudstack.apache.org
> > Subject: RE: master build breaks at
> > com.cloud.vpc.NetworkACLServiceTest
> >
> > Alex,
> >
> > Can this change out of the box? I had the problem on a clean
> > checkout of master.
> >
> > -----Original Message-----
> > From: Alex Huang [mailto:Alex.Huang@citrix.com]
> > Sent: maandag 27 mei 2013 15:55
> > To: dev@cloudstack.apache.org
> > Subject: RE: master build breaks at
> > com.cloud.vpc.NetworkACLServiceTest
> >
> > Daan,
> >
> > The other possibility is that the test is affected by another test
> > so that it breaks when the tests are in a certain order. Have you
> > noticed what's the order of the tests ran when it breaks?
> >
> > --Alex
> >
> > > -----Original Message-----
> > > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > > Sent: Monday, May 27, 2013 2:37 AM
> > > To: dev@cloudstack.apache.org
> > > Subject: RE: master build breaks at
> > > com.cloud.vpc.NetworkACLServiceTest
> > >
> > > Being in the habit of replying on my own mails as I seems to be on
> > > this list;
> > >
> > > This seems to be related to eclipse refresh/build code. Does
> > > anybody have a solution to it? The problem is not consistently
> > > present but all the more irritating.
> > >
> > > Regards,
> > >
> > > -----Original Message-----
> > > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > > Sent: maandag 27 mei 2013 7:49
> > > To: dev@cloudstack.apache.org
> > > Subject: master build breaks at
> > > com.cloud.vpc.NetworkACLServiceTest
> > >
> > > LS,
> > >
> > > Lately I've been getting,
> > > "com.cloud.exception.InvalidParameterValueException: ACL item with
> > > number 1 already exists in ACL: null", while building cloudstack.
> > > I first thought it was my own hack so I don't know when it was
> > > introduced (it is in a clean master that it happens). The full
> > > surfire report
> > follows:
> > >
> > > ------------------------------------------------------------------
> > > --
> > > --
> > > --------- Test set: com.cloud.vpc.NetworkACLServiceTest
> > > ------------------------------------------------------------------
> > > --
> > > --
> > > --------- Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time
> > > elapsed: 0.219 sec <<< FAILURE!
> > > testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest) Time
> elapsed:
> > > 0.016 sec <<< ERROR!
> > > com.cloud.exception.InvalidParameterValueException: ACL item with
> > > number 1 already exists in ACL: null
> > > at
> > >
> >
> com.cloud.network.vpc.NetworkACLServiceImpl.createNetworkACLItem(Ne
> > > tworkACLServiceImpl.java:270)
> > > at
> > >
> >
> com.cloud.vpc.NetworkACLServiceTest.testCreateACLItem(NetworkACLServ
> > > iceTest.java:141)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> > > at
> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > > Source)
> > > at java.lang.reflect.Method.invoke(Unknown Source)
> > > at
> > >
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(Framework
> > > Method.java:45)
> > > at
> > > org.junit.internal.runners.model.ReflectiveCallable.run(Reflective
> > > Ca
> > > ll
> > > able.jav
> > > a:15)
> > > at
> > >
> org.junit.runners.model.FrameworkMethod.invokeExplosively(Framework
> > > Method.java:42)
> > > at
> > >
> >
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMeth
> > > od.java:20)
> > > at
> > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefor
> > > es
> > > .j
> > > ava
> > > :28)
> > > at
> > >
> >
> org.springframework.test.context.junit4.statements.RunBeforeTestMethod
> > > Callbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
> > > at
> > >
> >
> org.springframework.test.context.junit4.statements.RunAfterTestMethodC
> > > allbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
> > > at
> > > org.springframework.test.context.junit4.statements.SpringRepeat.ev
> > > al
> > > ua
> > > te(
> > > SpringRepeat.java:72)
> > > at
> > > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.ru
> > > nC
> > > hi
> > > ld(S
> > > pringJUnit4ClassRunner.java:231)
> > > at
> > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4Class
> > > Ru
> > > nn
> > > er.j
> > > ava:47)
> > > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> > > at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> > > at
> > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> > > at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> > > at
> > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> > > at
> > > org.springframework.test.context.junit4.statements.RunBeforeTestCl
> > > as
> > > sC
> > > all
> > > backs.evaluate(RunBeforeTestClassCallbacks.java:61)
> > > at
> > > org.springframework.test.context.junit4.statements.RunAfterTestCla
> > > ss
> > > Ca
> > > llb
> > > acks.evaluate(RunAfterTestClassCallbacks.java:71)
> > > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> > > at
> > > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.ru
> > > n(
> > > Sp
> > > ring
> > > JUnit4ClassRunner.java:174)
> > > at
> > > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Prov
> > > id
> > > er
> > > .jav
> > > a:236)
> > > at
> > > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUn
> > > it
> > > 4P
> > > rov
> > > ider.java:134)
> > > at
> > > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.
> > > java
> > > :113)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> > > at
> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > > Source)
> > > at java.lang.reflect.Method.invoke(Unknown Source)
> > > at
> > > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArr
> > > ay
> > > (R
> > > efl
> > > ectionUtils.java:189)
> > > at
> > > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.inv
> > > ok
> > > e(
> > > Pr
> > > oviderFactory.java:165)
> > > at
> > > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(Pr
> > > ov
> > > id
> > > er
> > > Factory.java:85)
> > > at
> > >
> >
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(Forke
> > > d
> > > Booter.java:103)
> > > at
> > >
> >
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:7
> > > 4)
RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Posted by Alex Huang <Al...@citrix.com>.
Kishan,
The problem is caused by these two methods. I hope everyone reads this because I know some of us are new to writing unit tests. The problem is basically as I described in a previous email. In here, both tests makes use of and shares of the _networkACLItemDao and the mock is shared because the DAO is a singleton and is injected. Now, note that testCreateACLItem() invokes the method findByAclAndNumber() via the _networkAclMgr but it didn't specifically mock the return value of the method in the test. If this test is ran before testCreateACLItemDuplicateNumber(), then it will be successful because by default mocks return null. However, testCreateACLItemDuplicateNumber() mocks findByAclAndNumber() to return a value, so if testCreateACLItem() is ran after testCreateACLItemDuplicateNumber(), the test will fail because a value is returned and the condition inside _networkAclMgr will fire just as Daan found it.
I was hoping maven prints out the order of the tests, but from Daan's log, it only printed out the order of the bigger test suite so it's not helpful. However, it's not too hard to find. Just go to the code with the error, check the method that fails and go back to the test and see if your test mocked that method and if it didn't, was it mocked by any other tests.
@Test
public void testCreateACLItem() throws Exception {
Mockito.when(_vpcMgr.getVpc(Mockito.anyLong())).thenReturn(new VpcVO());
Mockito.when(_networkAclMgr.getNetworkACL(Mockito.anyLong())).thenReturn(acl);
Mockito.when(_networkAclMgr.createNetworkACLItem(Mockito.anyInt(), Mockito.anyInt(), Mockito.anyString(), Mockito.anyList(), Mockito.anyInt(), Mockito.anyInt(),
Mockito.any(NetworkACLItem.TrafficType.class), Mockito.anyLong(), Mockito.anyString(), Mockito.anyInt())).thenReturn(new NetworkACLItemVO());
assertNotNull(_aclService.createNetworkACLItem(createACLItemCmd));
}
@Test(expected = InvalidParameterValueException.class)
public void testCreateACLItemDuplicateNumber() throws Exception {
Mockito.when(_vpcMgr.getVpc(Mockito.anyLong())).thenReturn(new VpcVO());
Mockito.when(_networkAclMgr.getNetworkACL(Mockito.anyLong())).thenReturn(acl);
Mockito.when(_networkACLItemDao.findByAclAndNumber(Mockito.anyLong(), Mockito.anyInt())).thenReturn(new NetworkACLItemVO());
_aclService.createNetworkACLItem(createACLItemCmd);
}
> -----Original Message-----
> From: Kishan Kavala
> Sent: Wednesday, May 29, 2013 4:48 AM
> To: dev@cloudstack.apache.org
> Cc: Alex Huang
> Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
>
> Daan,
> I mocked the method causing the failure (I couldn't replicate the error you
> are seeing though). Please check with latest master and let us know if you
> still see issues.
>
> ~kishan
> ________________________________________
> From: Daan Hoogland [DHoogland@schubergphilis.com]
> Sent: Wednesday, May 29, 2013 1:49 PM
> To: dev@cloudstack.apache.org
> Cc: Alex Huang
> Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
>
> H,
>
> I hope this is what Alex means the test programmer needs.
>
> The order that is reported is as follows (I included the server test run only):
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Running com.cloud.alert.AlertControlsUnitTest
> log4j:WARN No appenders could be found for logger
> (com.cloud.alert.AlertControlsUnitTest).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
> more info.
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.546 sec Running
> com.cloud.capacity.CapacityManagerTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11 sec Running
> com.cloud.configuration.ConfigurationManagerTest
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.499 sec Running
> com.cloud.configuration.ValidateIpRangeTest
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 sec Running
> com.cloud.event.EventControlsUnitTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec Running
> com.cloud.keystore.KeystoreTest
> org.apache.cloudstack.api.response.UserVmResponse/null/{"id":"3","securit
> ygroup":[],"nic":[],"tags":[],"affinitygroup":[]}
> org.apache.cloudstack.api.response.AlertResponse/null/{"id":"100","descrip
> tion":"Hello"}
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.109 sec Running
> com.cloud.metadata.ResourceMetaDataManagerTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.047 sec Running
> com.cloud.network.CreatePrivateNetworkTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec Running
> com.cloud.network.dao.NetworkDaoTest
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec Running
> com.cloud.network.DedicateGuestVlanRangesTest
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 sec Running
> com.cloud.network.firewall.FirewallManagerTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec Running
> com.cloud.network.NetworkManagerTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec Running
> com.cloud.network.NetworkModelTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec Running
> com.cloud.network.security.SecurityGroupQueueTest
> Num Vms= 50 Queue size = 50
> Num Vms= 20000 Queue size = 20000 time=827 ms Num Vms= 5000 Queue
> size = 5000 time=2544 ms Num Vms= 1 Queue size = 1 time=0 ms Num Vms=
> 1000000 Queue size = 1000000 time=1171 ms Num Vms= 1 Queue size = 1
> time=1140 ms Total jobs dequeued = 10, num queued=1010 queue current
> size=1000 Total jobs dequeued = 10, num queued=1008 queue current
> size=998 Total jobs dequeued = 1, num queued=1001 queue current
> size=1000 Total jobs dequeued = 10, num queued=1000 queue current
> size=990 Total jobs dequeued = 10, num queued=10 queue current size=0
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.745 sec Running
> com.cloud.network.UpdatePhysicalNetworkTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.312 sec Running
> com.cloud.resourcelimit.ResourceLimitManagerImplTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec Running
> com.cloud.vm.DeploymentPlanningManagerImplTest
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.713 sec Running
> com.cloud.vm.snapshot.VMSnapshotManagerTest
> Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.203 sec Running
> com.cloud.vm.UserVmManagerTest Tests run: 10, Failures: 0, Errors: 0,
> Skipped: 0, Time elapsed: 0.64 sec Running
> com.cloud.vm.VirtualMachineManagerImplTest
> Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.297 sec Running
> com.cloud.vpc.NetworkACLManagerTest
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.406 sec Running
> com.cloud.vpc.NetworkACLServiceTest
> Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.203 sec <<<
> FAILURE!
> Running com.cloud.vpc.Site2SiteVpnTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.491 sec Running
> com.cloud.vpc.VpcTest Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time
> elapsed: 0.235 sec Running org.apache.cloudstack.affinity.AffinityApiUnitTest
> Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.188 sec Running
> org.apache.cloudstack.lb.ApplicationLoadBalancerTest
> Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.249 sec
> Running org.apache.cloudstack.networkoffering.CreateNetworkOfferingTest
> Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.817 sec
> Running org.apache.cloudstack.privategw.AclOnPrivateGwTest
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11 sec Running
> org.apache.cloudstack.region.gslb.GlobalLoadBalancingRulesServiceImplTest
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.032 sec Running
> org.apache.cloudstack.region.RegionManagerTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
>
> Results :
>
> Tests in error:
> testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest): ACL item with
> number 1 already exists in ACL: null
>
> -----Original Message-----
> From: Pranav Saxena [mailto:pranav.saxena@citrix.com]
> Sent: woensdag 29 mei 2013 5:58
> To: dev@cloudstack.apache.org
> Cc: Alex Huang
> Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
>
> Thanks Alex for throwing some light on this . I have myself seen this when
> one of the intenalLb tests were failing on my machine and few others but
> compiling successfully on Alena's machine but that time I also didn't know
> how this could have happened and I ended up reporting this to her and then
> after probably some amendments , unit tests compiled on all the machines.
>
> -----Original Message-----
> From: Alex Huang [mailto:Alex.Huang@citrix.com]
> Sent: Wednesday, May 29, 2013 12:05 AM
> To: dev@cloudstack.apache.org
> Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
>
> Unit tests are ran in random order because each unit test should be self-
> sufficient. I'm saying I've seen cases where unit tests aren't written to be
> self-sufficient and that's why an unit test fails sometimes and passes
> sometimes.
>
> In those cases, we should fix the unit test and not the order. A common
> problem is sharing the mocked classes. For example, unit test A tests the set
> of a mock object and unit test B tests if the field is null but they shared the
> mock instance. In this case, unit test B passes if it is ran before unit test A but
> fails if it was ran after.
>
> But often times the randomness of how it's ran is actually pseudo-random so
> it works consistently on someone's laptop but as soon as it gets ran on
> another machine, the order changes and it fails. So when you see the
> problem such as this, you should always let the unit test writer know the
> order in which the unit tests were executed so that the unit test writer can
> try it.
>
> --Alex
>
> > -----Original Message-----
> > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > Sent: Monday, May 27, 2013 9:23 AM
> > To: dev@cloudstack.apache.org
> > Subject: RE: master build breaks at
> > com.cloud.vpc.NetworkACLServiceTest
> >
> > And if not how do I manipulate the order of tests?
> >
> > -----Original Message-----
> > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > Sent: maandag 27 mei 2013 17:02
> > To: dev@cloudstack.apache.org
> > Subject: RE: master build breaks at
> > com.cloud.vpc.NetworkACLServiceTest
> >
> > Alex,
> >
> > Can this change out of the box? I had the problem on a clean checkout
> > of master.
> >
> > -----Original Message-----
> > From: Alex Huang [mailto:Alex.Huang@citrix.com]
> > Sent: maandag 27 mei 2013 15:55
> > To: dev@cloudstack.apache.org
> > Subject: RE: master build breaks at
> > com.cloud.vpc.NetworkACLServiceTest
> >
> > Daan,
> >
> > The other possibility is that the test is affected by another test so
> > that it breaks when the tests are in a certain order. Have you
> > noticed what's the order of the tests ran when it breaks?
> >
> > --Alex
> >
> > > -----Original Message-----
> > > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > > Sent: Monday, May 27, 2013 2:37 AM
> > > To: dev@cloudstack.apache.org
> > > Subject: RE: master build breaks at
> > > com.cloud.vpc.NetworkACLServiceTest
> > >
> > > Being in the habit of replying on my own mails as I seems to be on
> > > this list;
> > >
> > > This seems to be related to eclipse refresh/build code. Does anybody
> > > have a solution to it? The problem is not consistently present but
> > > all the more irritating.
> > >
> > > Regards,
> > >
> > > -----Original Message-----
> > > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > > Sent: maandag 27 mei 2013 7:49
> > > To: dev@cloudstack.apache.org
> > > Subject: master build breaks at com.cloud.vpc.NetworkACLServiceTest
> > >
> > > LS,
> > >
> > > Lately I've been getting,
> > > "com.cloud.exception.InvalidParameterValueException: ACL item with
> > > number 1 already exists in ACL: null", while building cloudstack. I
> > > first thought it was my own hack so I don't know when it was
> > > introduced (it is in a clean master that it happens). The full
> > > surfire report
> > follows:
> > >
> > > --------------------------------------------------------------------
> > > --
> > > --------- Test set: com.cloud.vpc.NetworkACLServiceTest
> > > --------------------------------------------------------------------
> > > --
> > > --------- Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time
> > > elapsed: 0.219 sec <<< FAILURE!
> > > testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest) Time
> elapsed:
> > > 0.016 sec <<< ERROR!
> > > com.cloud.exception.InvalidParameterValueException: ACL item with
> > > number 1 already exists in ACL: null
> > > at
> > >
> >
> com.cloud.network.vpc.NetworkACLServiceImpl.createNetworkACLItem(Ne
> > > tworkACLServiceImpl.java:270)
> > > at
> > >
> >
> com.cloud.vpc.NetworkACLServiceTest.testCreateACLItem(NetworkACLServ
> > > iceTest.java:141)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> > > at
> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > > Source)
> > > at java.lang.reflect.Method.invoke(Unknown Source)
> > > at
> > >
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(Framework
> > > Method.java:45)
> > > at
> > > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCa
> > > ll
> > > able.jav
> > > a:15)
> > > at
> > >
> org.junit.runners.model.FrameworkMethod.invokeExplosively(Framework
> > > Method.java:42)
> > > at
> > >
> >
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMeth
> > > od.java:20)
> > > at
> > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores
> > > .j
> > > ava
> > > :28)
> > > at
> > >
> >
> org.springframework.test.context.junit4.statements.RunBeforeTestMethod
> > > Callbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
> > > at
> > >
> >
> org.springframework.test.context.junit4.statements.RunAfterTestMethodC
> > > allbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
> > > at
> > > org.springframework.test.context.junit4.statements.SpringRepeat.eval
> > > ua
> > > te(
> > > SpringRepeat.java:72)
> > > at
> > > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runC
> > > hi
> > > ld(S
> > > pringJUnit4ClassRunner.java:231)
> > > at
> > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRu
> > > nn
> > > er.j
> > > ava:47)
> > > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> > > at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> > > at
> > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> > > at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> > > at
> > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> > > at
> > > org.springframework.test.context.junit4.statements.RunBeforeTestClas
> > > sC
> > > all
> > > backs.evaluate(RunBeforeTestClassCallbacks.java:61)
> > > at
> > > org.springframework.test.context.junit4.statements.RunAfterTestClass
> > > Ca
> > > llb
> > > acks.evaluate(RunAfterTestClassCallbacks.java:71)
> > > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> > > at
> > > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(
> > > Sp
> > > ring
> > > JUnit4ClassRunner.java:174)
> > > at
> > > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provid
> > > er
> > > .jav
> > > a:236)
> > > at
> > > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit
> > > 4P
> > > rov
> > > ider.java:134)
> > > at
> > > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.
> > > java
> > > :113)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> > > at
> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > > Source)
> > > at java.lang.reflect.Method.invoke(Unknown Source)
> > > at
> > > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray
> > > (R
> > > efl
> > > ectionUtils.java:189)
> > > at
> > > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invok
> > > e(
> > > Pr
> > > oviderFactory.java:165)
> > > at
> > > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(Prov
> > > id
> > > er
> > > Factory.java:85)
> > > at
> > >
> >
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(Forke
> > > d
> > > Booter.java:103)
> > > at
> > >
> >
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:7
> > > 4)
RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Posted by Kishan Kavala <Ki...@citrix.com>.
Daan,
I mocked the method causing the failure (I couldn't replicate the error you are seeing though). Please check with latest master and let us know if you still see issues.
~kishan
________________________________________
From: Daan Hoogland [DHoogland@schubergphilis.com]
Sent: Wednesday, May 29, 2013 1:49 PM
To: dev@cloudstack.apache.org
Cc: Alex Huang
Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
H,
I hope this is what Alex means the test programmer needs.
The order that is reported is as follows (I included the server test run only):
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.cloud.alert.AlertControlsUnitTest
log4j:WARN No appenders could be found for logger (com.cloud.alert.AlertControlsUnitTest).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.546 sec
Running com.cloud.capacity.CapacityManagerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11 sec
Running com.cloud.configuration.ConfigurationManagerTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.499 sec
Running com.cloud.configuration.ValidateIpRangeTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 sec
Running com.cloud.event.EventControlsUnitTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
Running com.cloud.keystore.KeystoreTest
org.apache.cloudstack.api.response.UserVmResponse/null/{"id":"3","securitygroup":[],"nic":[],"tags":[],"affinitygroup":[]}
org.apache.cloudstack.api.response.AlertResponse/null/{"id":"100","description":"Hello"}
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.109 sec
Running com.cloud.metadata.ResourceMetaDataManagerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.047 sec
Running com.cloud.network.CreatePrivateNetworkTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec
Running com.cloud.network.dao.NetworkDaoTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
Running com.cloud.network.DedicateGuestVlanRangesTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 sec
Running com.cloud.network.firewall.FirewallManagerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec
Running com.cloud.network.NetworkManagerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec
Running com.cloud.network.NetworkModelTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
Running com.cloud.network.security.SecurityGroupQueueTest
Num Vms= 50 Queue size = 50
Num Vms= 20000 Queue size = 20000 time=827 ms
Num Vms= 5000 Queue size = 5000 time=2544 ms
Num Vms= 1 Queue size = 1 time=0 ms
Num Vms= 1000000 Queue size = 1000000 time=1171 ms
Num Vms= 1 Queue size = 1 time=1140 ms
Total jobs dequeued = 10, num queued=1010 queue current size=1000
Total jobs dequeued = 10, num queued=1008 queue current size=998
Total jobs dequeued = 1, num queued=1001 queue current size=1000
Total jobs dequeued = 10, num queued=1000 queue current size=990
Total jobs dequeued = 10, num queued=10 queue current size=0
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.745 sec
Running com.cloud.network.UpdatePhysicalNetworkTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.312 sec
Running com.cloud.resourcelimit.ResourceLimitManagerImplTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
Running com.cloud.vm.DeploymentPlanningManagerImplTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.713 sec
Running com.cloud.vm.snapshot.VMSnapshotManagerTest
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.203 sec
Running com.cloud.vm.UserVmManagerTest
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.64 sec
Running com.cloud.vm.VirtualMachineManagerImplTest
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.297 sec
Running com.cloud.vpc.NetworkACLManagerTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.406 sec
Running com.cloud.vpc.NetworkACLServiceTest
Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.203 sec <<< FAILURE!
Running com.cloud.vpc.Site2SiteVpnTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.491 sec
Running com.cloud.vpc.VpcTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.235 sec
Running org.apache.cloudstack.affinity.AffinityApiUnitTest
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.188 sec
Running org.apache.cloudstack.lb.ApplicationLoadBalancerTest
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.249 sec
Running org.apache.cloudstack.networkoffering.CreateNetworkOfferingTest
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.817 sec
Running org.apache.cloudstack.privategw.AclOnPrivateGwTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11 sec
Running org.apache.cloudstack.region.gslb.GlobalLoadBalancingRulesServiceImplTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.032 sec
Running org.apache.cloudstack.region.RegionManagerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
Results :
Tests in error:
testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest): ACL item with number 1 already exists in ACL: null
-----Original Message-----
From: Pranav Saxena [mailto:pranav.saxena@citrix.com]
Sent: woensdag 29 mei 2013 5:58
To: dev@cloudstack.apache.org
Cc: Alex Huang
Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Thanks Alex for throwing some light on this . I have myself seen this when one of the intenalLb tests were failing on my machine and few others but compiling successfully on Alena's machine but that time I also didn't know how this could have happened and I ended up reporting this to her and then after probably some amendments , unit tests compiled on all the machines.
-----Original Message-----
From: Alex Huang [mailto:Alex.Huang@citrix.com]
Sent: Wednesday, May 29, 2013 12:05 AM
To: dev@cloudstack.apache.org
Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Unit tests are ran in random order because each unit test should be self-sufficient. I'm saying I've seen cases where unit tests aren't written to be self-sufficient and that's why an unit test fails sometimes and passes sometimes.
In those cases, we should fix the unit test and not the order. A common problem is sharing the mocked classes. For example, unit test A tests the set of a mock object and unit test B tests if the field is null but they shared the mock instance. In this case, unit test B passes if it is ran before unit test A but fails if it was ran after.
But often times the randomness of how it's ran is actually pseudo-random so it works consistently on someone's laptop but as soon as it gets ran on another machine, the order changes and it fails. So when you see the problem such as this, you should always let the unit test writer know the order in which the unit tests were executed so that the unit test writer can try it.
--Alex
> -----Original Message-----
> From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> Sent: Monday, May 27, 2013 9:23 AM
> To: dev@cloudstack.apache.org
> Subject: RE: master build breaks at
> com.cloud.vpc.NetworkACLServiceTest
>
> And if not how do I manipulate the order of tests?
>
> -----Original Message-----
> From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> Sent: maandag 27 mei 2013 17:02
> To: dev@cloudstack.apache.org
> Subject: RE: master build breaks at
> com.cloud.vpc.NetworkACLServiceTest
>
> Alex,
>
> Can this change out of the box? I had the problem on a clean checkout
> of master.
>
> -----Original Message-----
> From: Alex Huang [mailto:Alex.Huang@citrix.com]
> Sent: maandag 27 mei 2013 15:55
> To: dev@cloudstack.apache.org
> Subject: RE: master build breaks at
> com.cloud.vpc.NetworkACLServiceTest
>
> Daan,
>
> The other possibility is that the test is affected by another test so
> that it breaks when the tests are in a certain order. Have you
> noticed what's the order of the tests ran when it breaks?
>
> --Alex
>
> > -----Original Message-----
> > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > Sent: Monday, May 27, 2013 2:37 AM
> > To: dev@cloudstack.apache.org
> > Subject: RE: master build breaks at
> > com.cloud.vpc.NetworkACLServiceTest
> >
> > Being in the habit of replying on my own mails as I seems to be on
> > this list;
> >
> > This seems to be related to eclipse refresh/build code. Does anybody
> > have a solution to it? The problem is not consistently present but
> > all the more irritating.
> >
> > Regards,
> >
> > -----Original Message-----
> > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > Sent: maandag 27 mei 2013 7:49
> > To: dev@cloudstack.apache.org
> > Subject: master build breaks at com.cloud.vpc.NetworkACLServiceTest
> >
> > LS,
> >
> > Lately I've been getting,
> > "com.cloud.exception.InvalidParameterValueException: ACL item with
> > number 1 already exists in ACL: null", while building cloudstack. I
> > first thought it was my own hack so I don't know when it was
> > introduced (it is in a clean master that it happens). The full
> > surfire report
> follows:
> >
> > --------------------------------------------------------------------
> > --
> > --------- Test set: com.cloud.vpc.NetworkACLServiceTest
> > --------------------------------------------------------------------
> > --
> > --------- Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time
> > elapsed: 0.219 sec <<< FAILURE!
> > testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest) Time elapsed:
> > 0.016 sec <<< ERROR!
> > com.cloud.exception.InvalidParameterValueException: ACL item with
> > number 1 already exists in ACL: null
> > at
> >
> com.cloud.network.vpc.NetworkACLServiceImpl.createNetworkACLItem(Ne
> > tworkACLServiceImpl.java:270)
> > at
> >
> com.cloud.vpc.NetworkACLServiceTest.testCreateACLItem(NetworkACLServ
> > iceTest.java:141)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > Source)
> > at java.lang.reflect.Method.invoke(Unknown Source)
> > at
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(Framework
> > Method.java:45)
> > at
> > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCa
> > ll
> > able.jav
> > a:15)
> > at
> > org.junit.runners.model.FrameworkMethod.invokeExplosively(Framework
> > Method.java:42)
> > at
> >
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMeth
> > od.java:20)
> > at
> > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores
> > .j
> > ava
> > :28)
> > at
> >
> org.springframework.test.context.junit4.statements.RunBeforeTestMethod
> > Callbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
> > at
> >
> org.springframework.test.context.junit4.statements.RunAfterTestMethodC
> > allbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
> > at
> > org.springframework.test.context.junit4.statements.SpringRepeat.eval
> > ua
> > te(
> > SpringRepeat.java:72)
> > at
> > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runC
> > hi
> > ld(S
> > pringJUnit4ClassRunner.java:231)
> > at
> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRu
> > nn
> > er.j
> > ava:47)
> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> > at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> > at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> > at
> > org.springframework.test.context.junit4.statements.RunBeforeTestClas
> > sC
> > all
> > backs.evaluate(RunBeforeTestClassCallbacks.java:61)
> > at
> > org.springframework.test.context.junit4.statements.RunAfterTestClass
> > Ca
> > llb
> > acks.evaluate(RunAfterTestClassCallbacks.java:71)
> > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> > at
> > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(
> > Sp
> > ring
> > JUnit4ClassRunner.java:174)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provid
> > er
> > .jav
> > a:236)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit
> > 4P
> > rov
> > ider.java:134)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.
> > java
> > :113)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > Source)
> > at java.lang.reflect.Method.invoke(Unknown Source)
> > at
> > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray
> > (R
> > efl
> > ectionUtils.java:189)
> > at
> > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invok
> > e(
> > Pr
> > oviderFactory.java:165)
> > at
> > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(Prov
> > id
> > er
> > Factory.java:85)
> > at
> >
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(Forke
> > d
> > Booter.java:103)
> > at
> >
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:7
> > 4)
RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Posted by Daan Hoogland <DH...@schubergphilis.com>.
H,
I hope this is what Alex means the test programmer needs.
The order that is reported is as follows (I included the server test run only):
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running com.cloud.alert.AlertControlsUnitTest
log4j:WARN No appenders could be found for logger (com.cloud.alert.AlertControlsUnitTest).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.546 sec
Running com.cloud.capacity.CapacityManagerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11 sec
Running com.cloud.configuration.ConfigurationManagerTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.499 sec
Running com.cloud.configuration.ValidateIpRangeTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 sec
Running com.cloud.event.EventControlsUnitTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
Running com.cloud.keystore.KeystoreTest
org.apache.cloudstack.api.response.UserVmResponse/null/{"id":"3","securitygroup":[],"nic":[],"tags":[],"affinitygroup":[]}
org.apache.cloudstack.api.response.AlertResponse/null/{"id":"100","description":"Hello"}
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.109 sec
Running com.cloud.metadata.ResourceMetaDataManagerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.047 sec
Running com.cloud.network.CreatePrivateNetworkTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec
Running com.cloud.network.dao.NetworkDaoTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
Running com.cloud.network.DedicateGuestVlanRangesTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 sec
Running com.cloud.network.firewall.FirewallManagerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec
Running com.cloud.network.NetworkManagerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec
Running com.cloud.network.NetworkModelTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
Running com.cloud.network.security.SecurityGroupQueueTest
Num Vms= 50 Queue size = 50
Num Vms= 20000 Queue size = 20000 time=827 ms
Num Vms= 5000 Queue size = 5000 time=2544 ms
Num Vms= 1 Queue size = 1 time=0 ms
Num Vms= 1000000 Queue size = 1000000 time=1171 ms
Num Vms= 1 Queue size = 1 time=1140 ms
Total jobs dequeued = 10, num queued=1010 queue current size=1000
Total jobs dequeued = 10, num queued=1008 queue current size=998
Total jobs dequeued = 1, num queued=1001 queue current size=1000
Total jobs dequeued = 10, num queued=1000 queue current size=990
Total jobs dequeued = 10, num queued=10 queue current size=0
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.745 sec
Running com.cloud.network.UpdatePhysicalNetworkTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.312 sec
Running com.cloud.resourcelimit.ResourceLimitManagerImplTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
Running com.cloud.vm.DeploymentPlanningManagerImplTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.713 sec
Running com.cloud.vm.snapshot.VMSnapshotManagerTest
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.203 sec
Running com.cloud.vm.UserVmManagerTest
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.64 sec
Running com.cloud.vm.VirtualMachineManagerImplTest
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.297 sec
Running com.cloud.vpc.NetworkACLManagerTest
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.406 sec
Running com.cloud.vpc.NetworkACLServiceTest
Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.203 sec <<< FAILURE!
Running com.cloud.vpc.Site2SiteVpnTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.491 sec
Running com.cloud.vpc.VpcTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.235 sec
Running org.apache.cloudstack.affinity.AffinityApiUnitTest
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.188 sec
Running org.apache.cloudstack.lb.ApplicationLoadBalancerTest
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.249 sec
Running org.apache.cloudstack.networkoffering.CreateNetworkOfferingTest
Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.817 sec
Running org.apache.cloudstack.privategw.AclOnPrivateGwTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.11 sec
Running org.apache.cloudstack.region.gslb.GlobalLoadBalancingRulesServiceImplTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.032 sec
Running org.apache.cloudstack.region.RegionManagerTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
Results :
Tests in error:
testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest): ACL item with number 1 already exists in ACL: null
-----Original Message-----
From: Pranav Saxena [mailto:pranav.saxena@citrix.com]
Sent: woensdag 29 mei 2013 5:58
To: dev@cloudstack.apache.org
Cc: Alex Huang
Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Thanks Alex for throwing some light on this . I have myself seen this when one of the intenalLb tests were failing on my machine and few others but compiling successfully on Alena's machine but that time I also didn't know how this could have happened and I ended up reporting this to her and then after probably some amendments , unit tests compiled on all the machines.
-----Original Message-----
From: Alex Huang [mailto:Alex.Huang@citrix.com]
Sent: Wednesday, May 29, 2013 12:05 AM
To: dev@cloudstack.apache.org
Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Unit tests are ran in random order because each unit test should be self-sufficient. I'm saying I've seen cases where unit tests aren't written to be self-sufficient and that's why an unit test fails sometimes and passes sometimes.
In those cases, we should fix the unit test and not the order. A common problem is sharing the mocked classes. For example, unit test A tests the set of a mock object and unit test B tests if the field is null but they shared the mock instance. In this case, unit test B passes if it is ran before unit test A but fails if it was ran after.
But often times the randomness of how it's ran is actually pseudo-random so it works consistently on someone's laptop but as soon as it gets ran on another machine, the order changes and it fails. So when you see the problem such as this, you should always let the unit test writer know the order in which the unit tests were executed so that the unit test writer can try it.
--Alex
> -----Original Message-----
> From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> Sent: Monday, May 27, 2013 9:23 AM
> To: dev@cloudstack.apache.org
> Subject: RE: master build breaks at
> com.cloud.vpc.NetworkACLServiceTest
>
> And if not how do I manipulate the order of tests?
>
> -----Original Message-----
> From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> Sent: maandag 27 mei 2013 17:02
> To: dev@cloudstack.apache.org
> Subject: RE: master build breaks at
> com.cloud.vpc.NetworkACLServiceTest
>
> Alex,
>
> Can this change out of the box? I had the problem on a clean checkout
> of master.
>
> -----Original Message-----
> From: Alex Huang [mailto:Alex.Huang@citrix.com]
> Sent: maandag 27 mei 2013 15:55
> To: dev@cloudstack.apache.org
> Subject: RE: master build breaks at
> com.cloud.vpc.NetworkACLServiceTest
>
> Daan,
>
> The other possibility is that the test is affected by another test so
> that it breaks when the tests are in a certain order. Have you
> noticed what's the order of the tests ran when it breaks?
>
> --Alex
>
> > -----Original Message-----
> > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > Sent: Monday, May 27, 2013 2:37 AM
> > To: dev@cloudstack.apache.org
> > Subject: RE: master build breaks at
> > com.cloud.vpc.NetworkACLServiceTest
> >
> > Being in the habit of replying on my own mails as I seems to be on
> > this list;
> >
> > This seems to be related to eclipse refresh/build code. Does anybody
> > have a solution to it? The problem is not consistently present but
> > all the more irritating.
> >
> > Regards,
> >
> > -----Original Message-----
> > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > Sent: maandag 27 mei 2013 7:49
> > To: dev@cloudstack.apache.org
> > Subject: master build breaks at com.cloud.vpc.NetworkACLServiceTest
> >
> > LS,
> >
> > Lately I've been getting,
> > "com.cloud.exception.InvalidParameterValueException: ACL item with
> > number 1 already exists in ACL: null", while building cloudstack. I
> > first thought it was my own hack so I don't know when it was
> > introduced (it is in a clean master that it happens). The full
> > surfire report
> follows:
> >
> > --------------------------------------------------------------------
> > --
> > --------- Test set: com.cloud.vpc.NetworkACLServiceTest
> > --------------------------------------------------------------------
> > --
> > --------- Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time
> > elapsed: 0.219 sec <<< FAILURE!
> > testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest) Time elapsed:
> > 0.016 sec <<< ERROR!
> > com.cloud.exception.InvalidParameterValueException: ACL item with
> > number 1 already exists in ACL: null
> > at
> >
> com.cloud.network.vpc.NetworkACLServiceImpl.createNetworkACLItem(Ne
> > tworkACLServiceImpl.java:270)
> > at
> >
> com.cloud.vpc.NetworkACLServiceTest.testCreateACLItem(NetworkACLServ
> > iceTest.java:141)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > Source)
> > at java.lang.reflect.Method.invoke(Unknown Source)
> > at
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(Framework
> > Method.java:45)
> > at
> > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCa
> > ll
> > able.jav
> > a:15)
> > at
> > org.junit.runners.model.FrameworkMethod.invokeExplosively(Framework
> > Method.java:42)
> > at
> >
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMeth
> > od.java:20)
> > at
> > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores
> > .j
> > ava
> > :28)
> > at
> >
> org.springframework.test.context.junit4.statements.RunBeforeTestMethod
> > Callbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
> > at
> >
> org.springframework.test.context.junit4.statements.RunAfterTestMethodC
> > allbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
> > at
> > org.springframework.test.context.junit4.statements.SpringRepeat.eval
> > ua
> > te(
> > SpringRepeat.java:72)
> > at
> > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runC
> > hi
> > ld(S
> > pringJUnit4ClassRunner.java:231)
> > at
> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRu
> > nn
> > er.j
> > ava:47)
> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> > at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> > at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> > at
> > org.springframework.test.context.junit4.statements.RunBeforeTestClas
> > sC
> > all
> > backs.evaluate(RunBeforeTestClassCallbacks.java:61)
> > at
> > org.springframework.test.context.junit4.statements.RunAfterTestClass
> > Ca
> > llb
> > acks.evaluate(RunAfterTestClassCallbacks.java:71)
> > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> > at
> > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(
> > Sp
> > ring
> > JUnit4ClassRunner.java:174)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provid
> > er
> > .jav
> > a:236)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit
> > 4P
> > rov
> > ider.java:134)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.
> > java
> > :113)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > Source)
> > at java.lang.reflect.Method.invoke(Unknown Source)
> > at
> > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray
> > (R
> > efl
> > ectionUtils.java:189)
> > at
> > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invok
> > e(
> > Pr
> > oviderFactory.java:165)
> > at
> > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(Prov
> > id
> > er
> > Factory.java:85)
> > at
> >
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(Forke
> > d
> > Booter.java:103)
> > at
> >
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:7
> > 4)
RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Posted by Pranav Saxena <pr...@citrix.com>.
Thanks Alex for throwing some light on this . I have myself seen this when one of the intenalLb tests were failing on my machine and few others but compiling successfully on Alena's machine but that time I also didn't know how this could have happened and I ended up reporting this to her and then after probably some amendments , unit tests compiled on all the machines.
-----Original Message-----
From: Alex Huang [mailto:Alex.Huang@citrix.com]
Sent: Wednesday, May 29, 2013 12:05 AM
To: dev@cloudstack.apache.org
Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Unit tests are ran in random order because each unit test should be self-sufficient. I'm saying I've seen cases where unit tests aren't written to be self-sufficient and that's why an unit test fails sometimes and passes sometimes.
In those cases, we should fix the unit test and not the order. A common problem is sharing the mocked classes. For example, unit test A tests the set of a mock object and unit test B tests if the field is null but they shared the mock instance. In this case, unit test B passes if it is ran before unit test A but fails if it was ran after.
But often times the randomness of how it's ran is actually pseudo-random so it works consistently on someone's laptop but as soon as it gets ran on another machine, the order changes and it fails. So when you see the problem such as this, you should always let the unit test writer know the order in which the unit tests were executed so that the unit test writer can try it.
--Alex
> -----Original Message-----
> From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> Sent: Monday, May 27, 2013 9:23 AM
> To: dev@cloudstack.apache.org
> Subject: RE: master build breaks at
> com.cloud.vpc.NetworkACLServiceTest
>
> And if not how do I manipulate the order of tests?
>
> -----Original Message-----
> From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> Sent: maandag 27 mei 2013 17:02
> To: dev@cloudstack.apache.org
> Subject: RE: master build breaks at
> com.cloud.vpc.NetworkACLServiceTest
>
> Alex,
>
> Can this change out of the box? I had the problem on a clean checkout
> of master.
>
> -----Original Message-----
> From: Alex Huang [mailto:Alex.Huang@citrix.com]
> Sent: maandag 27 mei 2013 15:55
> To: dev@cloudstack.apache.org
> Subject: RE: master build breaks at
> com.cloud.vpc.NetworkACLServiceTest
>
> Daan,
>
> The other possibility is that the test is affected by another test so
> that it breaks when the tests are in a certain order. Have you
> noticed what's the order of the tests ran when it breaks?
>
> --Alex
>
> > -----Original Message-----
> > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > Sent: Monday, May 27, 2013 2:37 AM
> > To: dev@cloudstack.apache.org
> > Subject: RE: master build breaks at
> > com.cloud.vpc.NetworkACLServiceTest
> >
> > Being in the habit of replying on my own mails as I seems to be on
> > this list;
> >
> > This seems to be related to eclipse refresh/build code. Does anybody
> > have a solution to it? The problem is not consistently present but
> > all the more irritating.
> >
> > Regards,
> >
> > -----Original Message-----
> > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > Sent: maandag 27 mei 2013 7:49
> > To: dev@cloudstack.apache.org
> > Subject: master build breaks at com.cloud.vpc.NetworkACLServiceTest
> >
> > LS,
> >
> > Lately I've been getting,
> > "com.cloud.exception.InvalidParameterValueException: ACL item with
> > number 1 already exists in ACL: null", while building cloudstack. I
> > first thought it was my own hack so I don't know when it was
> > introduced (it is in a clean master that it happens). The full
> > surfire report
> follows:
> >
> > --------------------------------------------------------------------
> > --
> > --------- Test set: com.cloud.vpc.NetworkACLServiceTest
> > --------------------------------------------------------------------
> > --
> > --------- Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time
> > elapsed: 0.219 sec <<< FAILURE!
> > testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest) Time elapsed:
> > 0.016 sec <<< ERROR!
> > com.cloud.exception.InvalidParameterValueException: ACL item with
> > number 1 already exists in ACL: null
> > at
> >
> com.cloud.network.vpc.NetworkACLServiceImpl.createNetworkACLItem(Ne
> > tworkACLServiceImpl.java:270)
> > at
> >
> com.cloud.vpc.NetworkACLServiceTest.testCreateACLItem(NetworkACLServ
> > iceTest.java:141)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > Source)
> > at java.lang.reflect.Method.invoke(Unknown Source)
> > at
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(Framework
> > Method.java:45)
> > at
> > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCa
> > ll
> > able.jav
> > a:15)
> > at
> > org.junit.runners.model.FrameworkMethod.invokeExplosively(Framework
> > Method.java:42)
> > at
> >
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMeth
> > od.java:20)
> > at
> > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores
> > .j
> > ava
> > :28)
> > at
> >
> org.springframework.test.context.junit4.statements.RunBeforeTestMethod
> > Callbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
> > at
> >
> org.springframework.test.context.junit4.statements.RunAfterTestMethodC
> > allbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
> > at
> > org.springframework.test.context.junit4.statements.SpringRepeat.eval
> > ua
> > te(
> > SpringRepeat.java:72)
> > at
> > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runC
> > hi
> > ld(S
> > pringJUnit4ClassRunner.java:231)
> > at
> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRu
> > nn
> > er.j
> > ava:47)
> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> > at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> > at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> > at
> > org.springframework.test.context.junit4.statements.RunBeforeTestClas
> > sC
> > all
> > backs.evaluate(RunBeforeTestClassCallbacks.java:61)
> > at
> > org.springframework.test.context.junit4.statements.RunAfterTestClass
> > Ca
> > llb
> > acks.evaluate(RunAfterTestClassCallbacks.java:71)
> > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> > at
> > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(
> > Sp
> > ring
> > JUnit4ClassRunner.java:174)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provid
> > er
> > .jav
> > a:236)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit
> > 4P
> > rov
> > ider.java:134)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.
> > java
> > :113)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > Source)
> > at java.lang.reflect.Method.invoke(Unknown Source)
> > at
> > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray
> > (R
> > efl
> > ectionUtils.java:189)
> > at
> > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invok
> > e(
> > Pr
> > oviderFactory.java:165)
> > at
> > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(Prov
> > id
> > er
> > Factory.java:85)
> > at
> >
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(Forke
> > d
> > Booter.java:103)
> > at
> >
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:7
> > 4)
Re: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Posted by Daan Hoogland <da...@gmail.com>.
thanks fellows,
It is clear to me, what hte problem and the course to take is. I will
report more on it tomorrow
regards,
Daan
On Tue, May 28, 2013 at 8:35 PM, Alex Huang <Al...@citrix.com> wrote:
> Unit tests are ran in random order because each unit test should be
> self-sufficient. I'm saying I've seen cases where unit tests aren't
> written to be self-sufficient and that's why an unit test fails sometimes
> and passes sometimes.
>
> In those cases, we should fix the unit test and not the order. A common
> problem is sharing the mocked classes. For example, unit test A tests the
> set of a mock object and unit test B tests if the field is null but they
> shared the mock instance. In this case, unit test B passes if it is ran
> before unit test A but fails if it was ran after.
>
> But often times the randomness of how it's ran is actually pseudo-random
> so it works consistently on someone's laptop but as soon as it gets ran on
> another machine, the order changes and it fails. So when you see the
> problem such as this, you should always let the unit test writer know the
> order in which the unit tests were executed so that the unit test writer
> can try it.
>
> --Alex
>
> > -----Original Message-----
> > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > Sent: Monday, May 27, 2013 9:23 AM
> > To: dev@cloudstack.apache.org
> > Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
> >
> > And if not how do I manipulate the order of tests?
> >
> > -----Original Message-----
> > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > Sent: maandag 27 mei 2013 17:02
> > To: dev@cloudstack.apache.org
> > Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
> >
> > Alex,
> >
> > Can this change out of the box? I had the problem on a clean checkout of
> > master.
> >
> > -----Original Message-----
> > From: Alex Huang [mailto:Alex.Huang@citrix.com]
> > Sent: maandag 27 mei 2013 15:55
> > To: dev@cloudstack.apache.org
> > Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
> >
> > Daan,
> >
> > The other possibility is that the test is affected by another test so
> that it
> > breaks when the tests are in a certain order. Have you noticed what's
> the
> > order of the tests ran when it breaks?
> >
> > --Alex
> >
> > > -----Original Message-----
> > > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > > Sent: Monday, May 27, 2013 2:37 AM
> > > To: dev@cloudstack.apache.org
> > > Subject: RE: master build breaks at
> > > com.cloud.vpc.NetworkACLServiceTest
> > >
> > > Being in the habit of replying on my own mails as I seems to be on
> > > this list;
> > >
> > > This seems to be related to eclipse refresh/build code. Does anybody
> > > have a solution to it? The problem is not consistently present but all
> > > the more irritating.
> > >
> > > Regards,
> > >
> > > -----Original Message-----
> > > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > > Sent: maandag 27 mei 2013 7:49
> > > To: dev@cloudstack.apache.org
> > > Subject: master build breaks at com.cloud.vpc.NetworkACLServiceTest
> > >
> > > LS,
> > >
> > > Lately I've been getting,
> > > "com.cloud.exception.InvalidParameterValueException: ACL item with
> > > number 1 already exists in ACL: null", while building cloudstack. I
> > > first thought it was my own hack so I don't know when it was
> > > introduced (it is in a clean master that it happens). The full surfire
> report
> > follows:
> > >
> > > ----------------------------------------------------------------------
> > > --------- Test set: com.cloud.vpc.NetworkACLServiceTest
> > > ----------------------------------------------------------------------
> > > --------- Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time
> > > elapsed: 0.219 sec <<< FAILURE!
> > > testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest) Time elapsed:
> > > 0.016 sec <<< ERROR!
> > > com.cloud.exception.InvalidParameterValueException: ACL item with
> > > number 1 already exists in ACL: null
> > > at
> > >
> > com.cloud.network.vpc.NetworkACLServiceImpl.createNetworkACLItem(Ne
> > > tworkACLServiceImpl.java:270)
> > > at
> > >
> > com.cloud.vpc.NetworkACLServiceTest.testCreateACLItem(NetworkACLServ
> > > iceTest.java:141)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown
> Source)
> > > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > > Source)
> > > at java.lang.reflect.Method.invoke(Unknown Source)
> > > at
> > >
> > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(Framework
> > > Method.java:45)
> > > at
> > > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCall
> > > able.jav
> > > a:15)
> > > at
> > > org.junit.runners.model.FrameworkMethod.invokeExplosively(Framework
> > > Method.java:42)
> > > at
> > >
> > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMeth
> > > od.java:20)
> > > at
> > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.j
> > > ava
> > > :28)
> > > at
> > >
> > org.springframework.test.context.junit4.statements.RunBeforeTestMethod
> > > Callbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
> > > at
> > >
> > org.springframework.test.context.junit4.statements.RunAfterTestMethodC
> > > allbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
> > > at
> > > org.springframework.test.context.junit4.statements.SpringRepeat.evalua
> > > te(
> > > SpringRepeat.java:72)
> > > at
> > > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChi
> > > ld(S
> > > pringJUnit4ClassRunner.java:231)
> > > at
> > > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunn
> > > er.j
> > > ava:47)
> > > at
> org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> > > at
> org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> > > at
> > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> > > at
> org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> > > at
> > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> > > at
> > > org.springframework.test.context.junit4.statements.RunBeforeTestClassC
> > > all
> > > backs.evaluate(RunBeforeTestClassCallbacks.java:61)
> > > at
> > > org.springframework.test.context.junit4.statements.RunAfterTestClassCa
> > > llb
> > > acks.evaluate(RunAfterTestClassCallbacks.java:71)
> > > at
> org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> > > at
> > > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(Sp
> > > ring
> > > JUnit4ClassRunner.java:174)
> > > at
> > > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider
> > > .jav
> > > a:236)
> > > at
> > > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4P
> > > rov
> > > ider.java:134)
> > > at
> > > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.
> > > java
> > > :113)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown
> Source)
> > > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > > Source)
> > > at java.lang.reflect.Method.invoke(Unknown Source)
> > > at
> > > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(R
> > > efl
> > > ectionUtils.java:189)
> > > at
> > > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(
> > > Pr
> > > oviderFactory.java:165)
> > > at
> > > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(Provid
> > > er
> > > Factory.java:85)
> > > at
> > >
> > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(Forke
> > > d
> > > Booter.java:103)
> > > at
> > >
> > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:7
> > > 4)
>
RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Posted by Alex Huang <Al...@citrix.com>.
Unit tests are ran in random order because each unit test should be self-sufficient. I'm saying I've seen cases where unit tests aren't written to be self-sufficient and that's why an unit test fails sometimes and passes sometimes.
In those cases, we should fix the unit test and not the order. A common problem is sharing the mocked classes. For example, unit test A tests the set of a mock object and unit test B tests if the field is null but they shared the mock instance. In this case, unit test B passes if it is ran before unit test A but fails if it was ran after.
But often times the randomness of how it's ran is actually pseudo-random so it works consistently on someone's laptop but as soon as it gets ran on another machine, the order changes and it fails. So when you see the problem such as this, you should always let the unit test writer know the order in which the unit tests were executed so that the unit test writer can try it.
--Alex
> -----Original Message-----
> From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> Sent: Monday, May 27, 2013 9:23 AM
> To: dev@cloudstack.apache.org
> Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
>
> And if not how do I manipulate the order of tests?
>
> -----Original Message-----
> From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> Sent: maandag 27 mei 2013 17:02
> To: dev@cloudstack.apache.org
> Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
>
> Alex,
>
> Can this change out of the box? I had the problem on a clean checkout of
> master.
>
> -----Original Message-----
> From: Alex Huang [mailto:Alex.Huang@citrix.com]
> Sent: maandag 27 mei 2013 15:55
> To: dev@cloudstack.apache.org
> Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
>
> Daan,
>
> The other possibility is that the test is affected by another test so that it
> breaks when the tests are in a certain order. Have you noticed what's the
> order of the tests ran when it breaks?
>
> --Alex
>
> > -----Original Message-----
> > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > Sent: Monday, May 27, 2013 2:37 AM
> > To: dev@cloudstack.apache.org
> > Subject: RE: master build breaks at
> > com.cloud.vpc.NetworkACLServiceTest
> >
> > Being in the habit of replying on my own mails as I seems to be on
> > this list;
> >
> > This seems to be related to eclipse refresh/build code. Does anybody
> > have a solution to it? The problem is not consistently present but all
> > the more irritating.
> >
> > Regards,
> >
> > -----Original Message-----
> > From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> > Sent: maandag 27 mei 2013 7:49
> > To: dev@cloudstack.apache.org
> > Subject: master build breaks at com.cloud.vpc.NetworkACLServiceTest
> >
> > LS,
> >
> > Lately I've been getting,
> > "com.cloud.exception.InvalidParameterValueException: ACL item with
> > number 1 already exists in ACL: null", while building cloudstack. I
> > first thought it was my own hack so I don't know when it was
> > introduced (it is in a clean master that it happens). The full surfire report
> follows:
> >
> > ----------------------------------------------------------------------
> > --------- Test set: com.cloud.vpc.NetworkACLServiceTest
> > ----------------------------------------------------------------------
> > --------- Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time
> > elapsed: 0.219 sec <<< FAILURE!
> > testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest) Time elapsed:
> > 0.016 sec <<< ERROR!
> > com.cloud.exception.InvalidParameterValueException: ACL item with
> > number 1 already exists in ACL: null
> > at
> >
> com.cloud.network.vpc.NetworkACLServiceImpl.createNetworkACLItem(Ne
> > tworkACLServiceImpl.java:270)
> > at
> >
> com.cloud.vpc.NetworkACLServiceTest.testCreateACLItem(NetworkACLServ
> > iceTest.java:141)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > Source)
> > at java.lang.reflect.Method.invoke(Unknown Source)
> > at
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(Framework
> > Method.java:45)
> > at
> > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCall
> > able.jav
> > a:15)
> > at
> > org.junit.runners.model.FrameworkMethod.invokeExplosively(Framework
> > Method.java:42)
> > at
> >
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMeth
> > od.java:20)
> > at
> > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.j
> > ava
> > :28)
> > at
> >
> org.springframework.test.context.junit4.statements.RunBeforeTestMethod
> > Callbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
> > at
> >
> org.springframework.test.context.junit4.statements.RunAfterTestMethodC
> > allbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
> > at
> > org.springframework.test.context.junit4.statements.SpringRepeat.evalua
> > te(
> > SpringRepeat.java:72)
> > at
> > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChi
> > ld(S
> > pringJUnit4ClassRunner.java:231)
> > at
> > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunn
> > er.j
> > ava:47)
> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> > at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> > at
> org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> > at
> > org.springframework.test.context.junit4.statements.RunBeforeTestClassC
> > all
> > backs.evaluate(RunBeforeTestClassCallbacks.java:61)
> > at
> > org.springframework.test.context.junit4.statements.RunAfterTestClassCa
> > llb
> > acks.evaluate(RunAfterTestClassCallbacks.java:71)
> > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> > at
> > org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(Sp
> > ring
> > JUnit4ClassRunner.java:174)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider
> > .jav
> > a:236)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4P
> > rov
> > ider.java:134)
> > at
> > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.
> > java
> > :113)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> > Source)
> > at java.lang.reflect.Method.invoke(Unknown Source)
> > at
> > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(R
> > efl
> > ectionUtils.java:189)
> > at
> > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(
> > Pr
> > oviderFactory.java:165)
> > at
> > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(Provid
> > er
> > Factory.java:85)
> > at
> >
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(Forke
> > d
> > Booter.java:103)
> > at
> >
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:7
> > 4)
RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Posted by Daan Hoogland <DH...@schubergphilis.com>.
And if not how do I manipulate the order of tests?
-----Original Message-----
From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
Sent: maandag 27 mei 2013 17:02
To: dev@cloudstack.apache.org
Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Alex,
Can this change out of the box? I had the problem on a clean checkout of master.
-----Original Message-----
From: Alex Huang [mailto:Alex.Huang@citrix.com]
Sent: maandag 27 mei 2013 15:55
To: dev@cloudstack.apache.org
Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Daan,
The other possibility is that the test is affected by another test so that it breaks when the tests are in a certain order. Have you noticed what's the order of the tests ran when it breaks?
--Alex
> -----Original Message-----
> From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> Sent: Monday, May 27, 2013 2:37 AM
> To: dev@cloudstack.apache.org
> Subject: RE: master build breaks at
> com.cloud.vpc.NetworkACLServiceTest
>
> Being in the habit of replying on my own mails as I seems to be on
> this list;
>
> This seems to be related to eclipse refresh/build code. Does anybody
> have a solution to it? The problem is not consistently present but all
> the more irritating.
>
> Regards,
>
> -----Original Message-----
> From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> Sent: maandag 27 mei 2013 7:49
> To: dev@cloudstack.apache.org
> Subject: master build breaks at com.cloud.vpc.NetworkACLServiceTest
>
> LS,
>
> Lately I've been getting,
> "com.cloud.exception.InvalidParameterValueException: ACL item with
> number 1 already exists in ACL: null", while building cloudstack. I
> first thought it was my own hack so I don't know when it was
> introduced (it is in a clean master that it happens). The full surfire report follows:
>
> ----------------------------------------------------------------------
> --------- Test set: com.cloud.vpc.NetworkACLServiceTest
> ----------------------------------------------------------------------
> --------- Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time
> elapsed: 0.219 sec <<< FAILURE!
> testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest) Time elapsed:
> 0.016 sec <<< ERROR!
> com.cloud.exception.InvalidParameterValueException: ACL item with
> number 1 already exists in ACL: null
> at
> com.cloud.network.vpc.NetworkACLServiceImpl.createNetworkACLItem(Ne
> tworkACLServiceImpl.java:270)
> at
> com.cloud.vpc.NetworkACLServiceTest.testCreateACLItem(NetworkACLServ
> iceTest.java:141)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(Framework
> Method.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCall
> able.jav
> a:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(Framework
> Method.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMeth
> od.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.j
> ava
> :28)
> at
> org.springframework.test.context.junit4.statements.RunBeforeTestMethod
> Callbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
> at
> org.springframework.test.context.junit4.statements.RunAfterTestMethodC
> allbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
> at
> org.springframework.test.context.junit4.statements.SpringRepeat.evalua
> te(
> SpringRepeat.java:72)
> at
> org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChi
> ld(S
> pringJUnit4ClassRunner.java:231)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunn
> er.j
> ava:47)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> at
> org.springframework.test.context.junit4.statements.RunBeforeTestClassC
> all
> backs.evaluate(RunBeforeTestClassCallbacks.java:61)
> at
> org.springframework.test.context.junit4.statements.RunAfterTestClassCa
> llb
> acks.evaluate(RunAfterTestClassCallbacks.java:71)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at
> org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(Sp
> ring
> JUnit4ClassRunner.java:174)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider
> .jav
> a:236)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4P
> rov
> ider.java:134)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.
> java
> :113)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(R
> efl
> ectionUtils.java:189)
> at
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(
> Pr
> oviderFactory.java:165)
> at
> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(Provid
> er
> Factory.java:85)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(Forke
> d
> Booter.java:103)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:7
> 4)
RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Posted by Daan Hoogland <DH...@schubergphilis.com>.
Alex,
Can this change out of the box? I had the problem on a clean checkout of master.
-----Original Message-----
From: Alex Huang [mailto:Alex.Huang@citrix.com]
Sent: maandag 27 mei 2013 15:55
To: dev@cloudstack.apache.org
Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Daan,
The other possibility is that the test is affected by another test so that it breaks when the tests are in a certain order. Have you noticed what's the order of the tests ran when it breaks?
--Alex
> -----Original Message-----
> From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> Sent: Monday, May 27, 2013 2:37 AM
> To: dev@cloudstack.apache.org
> Subject: RE: master build breaks at
> com.cloud.vpc.NetworkACLServiceTest
>
> Being in the habit of replying on my own mails as I seems to be on
> this list;
>
> This seems to be related to eclipse refresh/build code. Does anybody
> have a solution to it? The problem is not consistently present but all
> the more irritating.
>
> Regards,
>
> -----Original Message-----
> From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> Sent: maandag 27 mei 2013 7:49
> To: dev@cloudstack.apache.org
> Subject: master build breaks at com.cloud.vpc.NetworkACLServiceTest
>
> LS,
>
> Lately I've been getting,
> "com.cloud.exception.InvalidParameterValueException: ACL item with
> number 1 already exists in ACL: null", while building cloudstack. I
> first thought it was my own hack so I don't know when it was
> introduced (it is in a clean master that it happens). The full surfire report follows:
>
> ----------------------------------------------------------------------
> --------- Test set: com.cloud.vpc.NetworkACLServiceTest
> ----------------------------------------------------------------------
> --------- Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time
> elapsed: 0.219 sec <<< FAILURE!
> testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest) Time elapsed:
> 0.016 sec <<< ERROR!
> com.cloud.exception.InvalidParameterValueException: ACL item with
> number 1 already exists in ACL: null
> at
> com.cloud.network.vpc.NetworkACLServiceImpl.createNetworkACLItem(Ne
> tworkACLServiceImpl.java:270)
> at
> com.cloud.vpc.NetworkACLServiceTest.testCreateACLItem(NetworkACLServ
> iceTest.java:141)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(Framework
> Method.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCall
> able.jav
> a:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(Framework
> Method.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMeth
> od.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.j
> ava
> :28)
> at
> org.springframework.test.context.junit4.statements.RunBeforeTestMethod
> Callbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
> at
> org.springframework.test.context.junit4.statements.RunAfterTestMethodC
> allbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
> at
> org.springframework.test.context.junit4.statements.SpringRepeat.evalua
> te(
> SpringRepeat.java:72)
> at
> org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChi
> ld(S
> pringJUnit4ClassRunner.java:231)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunn
> er.j
> ava:47)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> at
> org.springframework.test.context.junit4.statements.RunBeforeTestClassC
> all
> backs.evaluate(RunBeforeTestClassCallbacks.java:61)
> at
> org.springframework.test.context.junit4.statements.RunAfterTestClassCa
> llb
> acks.evaluate(RunAfterTestClassCallbacks.java:71)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at
> org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(Sp
> ring
> JUnit4ClassRunner.java:174)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider
> .jav
> a:236)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4P
> rov
> ider.java:134)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.
> java
> :113)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(R
> efl
> ectionUtils.java:189)
> at
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(
> Pr
> oviderFactory.java:165)
> at
> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(Provid
> er
> Factory.java:85)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(Forke
> d
> Booter.java:103)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:7
> 4)
RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Posted by Alex Huang <Al...@citrix.com>.
Daan,
The other possibility is that the test is affected by another test so that it breaks when the tests are in a certain order. Have you noticed what's the order of the tests ran when it breaks?
--Alex
> -----Original Message-----
> From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> Sent: Monday, May 27, 2013 2:37 AM
> To: dev@cloudstack.apache.org
> Subject: RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
>
> Being in the habit of replying on my own mails as I seems to be on this list;
>
> This seems to be related to eclipse refresh/build code. Does anybody have a
> solution to it? The problem is not consistently present but all the more
> irritating.
>
> Regards,
>
> -----Original Message-----
> From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
> Sent: maandag 27 mei 2013 7:49
> To: dev@cloudstack.apache.org
> Subject: master build breaks at com.cloud.vpc.NetworkACLServiceTest
>
> LS,
>
> Lately I've been getting,
> "com.cloud.exception.InvalidParameterValueException: ACL item with
> number 1 already exists in ACL: null", while building cloudstack. I first thought
> it was my own hack so I don't know when it was introduced (it is in a clean
> master that it happens). The full surfire report follows:
>
> -------------------------------------------------------------------------------
> Test set: com.cloud.vpc.NetworkACLServiceTest
> -------------------------------------------------------------------------------
> Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.219 sec <<<
> FAILURE!
> testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest) Time elapsed:
> 0.016 sec <<< ERROR!
> com.cloud.exception.InvalidParameterValueException: ACL item with
> number 1 already exists in ACL: null
> at
> com.cloud.network.vpc.NetworkACLServiceImpl.createNetworkACLItem(Ne
> tworkACLServiceImpl.java:270)
> at
> com.cloud.vpc.NetworkACLServiceTest.testCreateACLItem(NetworkACLServ
> iceTest.java:141)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(Framework
> Method.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.jav
> a:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(Framework
> Method.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMeth
> od.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java
> :28)
> at
> org.springframework.test.context.junit4.statements.RunBeforeTestMethod
> Callbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
> at
> org.springframework.test.context.junit4.statements.RunAfterTestMethodC
> allbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
> at
> org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(
> SpringRepeat.java:72)
> at
> org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(S
> pringJUnit4ClassRunner.java:231)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.j
> ava:47)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
> at
> org.springframework.test.context.junit4.statements.RunBeforeTestClassCall
> backs.evaluate(RunBeforeTestClassCallbacks.java:61)
> at
> org.springframework.test.context.junit4.statements.RunAfterTestClassCallb
> acks.evaluate(RunAfterTestClassCallbacks.java:71)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at
> org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(Spring
> JUnit4ClassRunner.java:174)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.jav
> a:236)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Prov
> ider.java:134)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java
> :113)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(Refl
> ectionUtils.java:189)
> at
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(Pr
> oviderFactory.java:165)
> at
> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(Provider
> Factory.java:85)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(Forked
> Booter.java:103)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:7
> 4)
RE: master build breaks at com.cloud.vpc.NetworkACLServiceTest
Posted by Daan Hoogland <DH...@schubergphilis.com>.
Being in the habit of replying on my own mails as I seems to be on this list;
This seems to be related to eclipse refresh/build code. Does anybody have a solution to it? The problem is not consistently present but all the more irritating.
Regards,
-----Original Message-----
From: Daan Hoogland [mailto:DHoogland@schubergphilis.com]
Sent: maandag 27 mei 2013 7:49
To: dev@cloudstack.apache.org
Subject: master build breaks at com.cloud.vpc.NetworkACLServiceTest
LS,
Lately I've been getting, "com.cloud.exception.InvalidParameterValueException: ACL item with number 1 already exists in ACL: null", while building cloudstack. I first thought it was my own hack so I don't know when it was introduced (it is in a clean master that it happens). The full surfire report follows:
-------------------------------------------------------------------------------
Test set: com.cloud.vpc.NetworkACLServiceTest
-------------------------------------------------------------------------------
Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.219 sec <<< FAILURE!
testCreateACLItem(com.cloud.vpc.NetworkACLServiceTest) Time elapsed: 0.016 sec <<< ERROR!
com.cloud.exception.InvalidParameterValueException: ACL item with number 1 already exists in ACL: null
at com.cloud.network.vpc.NetworkACLServiceImpl.createNetworkACLItem(NetworkACLServiceImpl.java:270)
at com.cloud.vpc.NetworkACLServiceTest.testCreateACLItem(NetworkACLServiceTest.java:141)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:72)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:231)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:71)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:174)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)