You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Jeff Zemerick <jz...@apache.org> on 2017/05/05 14:13:59 UTC
seemingly random junit test errors in 1.2.0-source-release
I'm getting a few random test errors when building the 1.2.0 source
release. The failures seem to be completely random. I have built many times
per the steps below and most times everything is ok but every now and then
one of the tests fails. I can't reproduce any of them in Maven or Eclipse.
Here are my steps:
1. Provision a new Ubuntu 16 VM (t2.small on EC2).
2. wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.2.0/nifi-
1.2.0-source-release.zip
3. unzip nifi-1.2.0-source-release.zip
4. cd nifi-1.2.0-source-release
5. mvn clean install -Pcontrib-check
Other details:
Apache Maven 3.3.9
openjdk version "1.8.0_121"
Ubuntu 16.04.2 LTS
The failing tests (Note that all 3 of these failed separately across
different builds and not in any single build):
Running org.apache.nifi.controller.scheduling.TestStandardProcessScheduler
Tests run: 9, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 16.091 sec
<<< FAILURE! - in
org.apache.nifi.controller.scheduling.TestStandardProcessScheduler
validateServiceEnablementLogicHappensOnlyOnce(org.apache.nifi.controller.scheduling.TestStandardProcessScheduler)
Time elapsed: 0.824 sec <<< FAILURE!
java.lang.AssertionError: expected:<1> but was:<0>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:631)
at
org.apache.nifi.controller.scheduling.TestStandardProcessScheduler.validateServiceEnablementLogicHappensOnlyOnce(TestStandardProcessScheduler.java:266)
Running
org.apache.nifi.controller.service.TestStandardControllerServiceProvider
Tests run: 8, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 125.657 sec
<<< FAILURE! - in
org.apache.nifi.controller.service.TestStandardControllerServiceProvider
testConcurrencyWithEnablingReferencingServicesGraph(org.apache.nifi.controller.service.TestStandardControllerServiceProvider)
Time elapsed: 120.098 sec <<< ERROR!
org.junit.runners.model.TestTimedOutException: test timed out after 120000
milliseconds
at java.util.HashMap.putVal(HashMap.java:630)
at java.util.HashMap.put(HashMap.java:611)
at java.util.HashSet.add(HashSet.java:219)
at org.apache.commons.lang3.ClassUtils.getAllInterfaces(ClassUtils.java:470)
at org.apache.commons.lang3.ClassUtils.getAllInterfaces(ClassUtils.java:471)
at org.apache.commons.lang3.ClassUtils.getAllInterfaces(ClassUtils.java:454)
at
org.apache.nifi.controller.service.StandardControllerServiceProvider.createControllerService(StandardControllerServiceProvider.java:126)
at
org.apache.nifi.controller.service.TestStandardControllerServiceProvider.testEnableReferencingServicesGraph(TestStandardControllerServiceProvider.java:214)
at
org.apache.nifi.controller.service.TestStandardControllerServiceProvider.testConcurrencyWithEnablingReferencingServicesGraph(TestStandardControllerServiceProvider.java:188)
Running org.apache.nifi.controller.scheduling.TestProcessorLifecycle
Tests run: 16, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 47.215 sec
<<< FAILURE! - in
org.apache.nifi.controller.scheduling.TestProcessorLifecycle
validateProcessScheduledAfterAdministrativeDelayDueToTheOnScheduledException(org.apache.nifi.controller.scheduling.TestProcessorLifecycle)
Time elapsed: 3.981 sec <<< FAILURE!
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.nifi.controller.scheduling.TestProcessorLifecycle.validateProcessScheduledAfterAdministrativeDelayDueToTheOnScheduledException(TestProcessorLifecycle.java:373)
Any ideas? Could an EC2 t2.small (2 GB RAM) be insufficient for the build?
Thanks,
Jeff
Re: seemingly random junit test errors in 1.2.0-source-release
Posted by Jeff Zemerick <jz...@apache.org>.
Thanks, Joe! I will document them.
Jeff
On Fri, May 5, 2017 at 10:19 AM, Joe Witt <jo...@gmail.com> wrote:
> Jeff
>
> We have test quality challenges in my opinion. Not new to this release at
> all but we did correct or isolate a lot of tests in this release cycle.
> Please feel free to file jiras for unstable tests as you find them. We
> want unit tests that will prove behavior, verify no regressions, run
> fast/quickly, aren't relying on magic threading timing, avoid
> integration/sockets/etc..
>
> I too have seen the behaviors you describe on some tests and on different
> systems. Happily this release I find far better than previous ones but we
> still have work to do for sure.
>
> Thanks
> Joe
>
>
> On May 5, 2017 10:14 AM, "Jeff Zemerick" <jz...@apache.org> wrote:
>
> I'm getting a few random test errors when building the 1.2.0 source
> release. The failures seem to be completely random. I have built many times
> per the steps below and most times everything is ok but every now and then
> one of the tests fails. I can't reproduce any of them in Maven or Eclipse.
>
> Here are my steps:
>
> 1. Provision a new Ubuntu 16 VM (t2.small on EC2).
> 2. wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.2.0/nifi-
> 1.2.0-source-release.zip
> 3. unzip nifi-1.2.0-source-release.zip
> 4. cd nifi-1.2.0-source-release
> 5. mvn clean install -Pcontrib-check
>
> Other details:
>
> Apache Maven 3.3.9
> openjdk version "1.8.0_121"
> Ubuntu 16.04.2 LTS
>
> The failing tests (Note that all 3 of these failed separately across
> different builds and not in any single build):
>
> Running org.apache.nifi.controller.scheduling.TestStandardProcessScheduler
> Tests run: 9, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 16.091 sec
> <<< FAILURE! - in
> org.apache.nifi.controller.scheduling.TestStandardProcessScheduler
> validateServiceEnablementLogicHappensOnlyOnce(org.apache.
> nifi.controller.scheduling.TestStandardProcessScheduler)
> Time elapsed: 0.824 sec <<< FAILURE!
> java.lang.AssertionError: expected:<1> but was:<0>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:834)
> at org.junit.Assert.assertEquals(Assert.java:645)
> at org.junit.Assert.assertEquals(Assert.java:631)
> at
> org.apache.nifi.controller.scheduling.TestStandardProcessScheduler.
> validateServiceEnablementLogicHappensOnlyOnce(TestStandardPr
> ocessScheduler.
> java:266)
>
> Running
> org.apache.nifi.controller.service.TestStandardControllerServiceProvider
> Tests run: 8, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 125.657 sec
> <<< FAILURE! - in
> org.apache.nifi.controller.service.TestStandardControllerServiceProvider
> testConcurrencyWithEnablingReferencingServicesGraph(org.
> apache.nifi.controller.service.TestStandardControllerServiceProvider)
> Time elapsed: 120.098 sec <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 120000
> milliseconds
> at java.util.HashMap.putVal(HashMap.java:630)
> at java.util.HashMap.put(HashMap.java:611)
> at java.util.HashSet.add(HashSet.java:219)
> at org.apache.commons.lang3.ClassUtils.getAllInterfaces(ClassUt
> ils.java:470)
> at org.apache.commons.lang3.ClassUtils.getAllInterfaces(ClassUt
> ils.java:471)
> at org.apache.commons.lang3.ClassUtils.getAllInterfaces(ClassUt
> ils.java:454)
> at
> org.apache.nifi.controller.service.StandardControllerServiceProvi
> der.createControllerService(StandardControllerServiceProvider.java:126)
> at
> org.apache.nifi.controller.service.TestStandardControllerServiceProvider.
> testEnableReferencingServicesGraph(TestStandardControllerServiceP
> rovider.java:214)
> at
> org.apache.nifi.controller.service.TestStandardControllerServiceProvider.
> testConcurrencyWithEnablingReferencingServicesGraph(
> TestStandardControllerServiceProvider.java:188)
>
> Running org.apache.nifi.controller.scheduling.TestProcessorLifecycle
> Tests run: 16, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 47.215 sec
> <<< FAILURE! - in
> org.apache.nifi.controller.scheduling.TestProcessorLifecycle
> validateProcessScheduledAfterAdministrativeDelayDueToTheOnSc
> heduledException(org.apache.nifi.controller.scheduling.
> TestProcessorLifecycle)
> Time elapsed: 3.981 sec <<< FAILURE!
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.nifi.controller.scheduling.TestProcessorLifecycle.
> validateProcessScheduledAfterAdministrativeDelayDueToTheOnSc
> heduledException(TestProcessorLifecycle.java:373)
>
> Any ideas? Could an EC2 t2.small (2 GB RAM) be insufficient for the build?
>
> Thanks,
> Jeff
>
Re: seemingly random junit test errors in 1.2.0-source-release
Posted by Joe Witt <jo...@gmail.com>.
Jeff
We have test quality challenges in my opinion. Not new to this release at
all but we did correct or isolate a lot of tests in this release cycle.
Please feel free to file jiras for unstable tests as you find them. We
want unit tests that will prove behavior, verify no regressions, run
fast/quickly, aren't relying on magic threading timing, avoid
integration/sockets/etc..
I too have seen the behaviors you describe on some tests and on different
systems. Happily this release I find far better than previous ones but we
still have work to do for sure.
Thanks
Joe
On May 5, 2017 10:14 AM, "Jeff Zemerick" <jz...@apache.org> wrote:
I'm getting a few random test errors when building the 1.2.0 source
release. The failures seem to be completely random. I have built many times
per the steps below and most times everything is ok but every now and then
one of the tests fails. I can't reproduce any of them in Maven or Eclipse.
Here are my steps:
1. Provision a new Ubuntu 16 VM (t2.small on EC2).
2. wget https://dist.apache.org/repos/dist/dev/nifi/nifi-1.2.0/nifi-
1.2.0-source-release.zip
3. unzip nifi-1.2.0-source-release.zip
4. cd nifi-1.2.0-source-release
5. mvn clean install -Pcontrib-check
Other details:
Apache Maven 3.3.9
openjdk version "1.8.0_121"
Ubuntu 16.04.2 LTS
The failing tests (Note that all 3 of these failed separately across
different builds and not in any single build):
Running org.apache.nifi.controller.scheduling.TestStandardProcessScheduler
Tests run: 9, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 16.091 sec
<<< FAILURE! - in
org.apache.nifi.controller.scheduling.TestStandardProcessScheduler
validateServiceEnablementLogicHappensOnlyOnce(org.apache.
nifi.controller.scheduling.TestStandardProcessScheduler)
Time elapsed: 0.824 sec <<< FAILURE!
java.lang.AssertionError: expected:<1> but was:<0>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:631)
at
org.apache.nifi.controller.scheduling.TestStandardProcessScheduler.
validateServiceEnablementLogicHappensOnlyOnce(TestStandardProcessScheduler.
java:266)
Running
org.apache.nifi.controller.service.TestStandardControllerServiceProvider
Tests run: 8, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 125.657 sec
<<< FAILURE! - in
org.apache.nifi.controller.service.TestStandardControllerServiceProvider
testConcurrencyWithEnablingReferencingServicesGraph(org.
apache.nifi.controller.service.TestStandardControllerServiceProvider)
Time elapsed: 120.098 sec <<< ERROR!
org.junit.runners.model.TestTimedOutException: test timed out after 120000
milliseconds
at java.util.HashMap.putVal(HashMap.java:630)
at java.util.HashMap.put(HashMap.java:611)
at java.util.HashSet.add(HashSet.java:219)
at org.apache.commons.lang3.ClassUtils.getAllInterfaces(ClassUtils.java:470)
at org.apache.commons.lang3.ClassUtils.getAllInterfaces(ClassUtils.java:471)
at org.apache.commons.lang3.ClassUtils.getAllInterfaces(ClassUtils.java:454)
at
org.apache.nifi.controller.service.StandardControllerServiceProvi
der.createControllerService(StandardControllerServiceProvider.java:126)
at
org.apache.nifi.controller.service.TestStandardControllerServiceProvider.
testEnableReferencingServicesGraph(TestStandardControllerServiceP
rovider.java:214)
at
org.apache.nifi.controller.service.TestStandardControllerServiceProvider.
testConcurrencyWithEnablingReferencingServicesGraph(
TestStandardControllerServiceProvider.java:188)
Running org.apache.nifi.controller.scheduling.TestProcessorLifecycle
Tests run: 16, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 47.215 sec
<<< FAILURE! - in
org.apache.nifi.controller.scheduling.TestProcessorLifecycle
validateProcessScheduledAfterAdministrativeDelayDueToTheOnSc
heduledException(org.apache.nifi.controller.scheduling.
TestProcessorLifecycle)
Time elapsed: 3.981 sec <<< FAILURE!
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.nifi.controller.scheduling.TestProcessorLifecycle.
validateProcessScheduledAfterAdministrativeDelayDueToTheOnSc
heduledException(TestProcessorLifecycle.java:373)
Any ideas? Could an EC2 t2.small (2 GB RAM) be insufficient for the build?
Thanks,
Jeff