[jira] [Comment Edited] (OOZIE-2550) Flaky tests in TestZKUUIDService.java

["Peter Bacsko (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OOZIE-2550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15314421#comment-15314421 ] 

Peter Bacsko edited comment on OOZIE-2550 at 6/3/16 5:09 PM:
-------------------------------------------------------------

Hi Ferenc,

Array init +1.

I agree that the usage of AtomicRef is debatable here. But in scenarios like this I like to play safe - using simple constructs can lead to subtle and hard-to-debug problems if a shared resource is mutated from multiple threads. However I just read an article (https://www.cs.umd.edu/~pugh/java/memoryModel/jsr-133-faq.html) which states that returning from a join() guarantees visibility so looks like it's safe to change it to primitive boolean array. 


was (Author: pbacsko):
Hi Ferenc,

Array init +1.

I agree that the usage of AtomicRef is debatable here. But in scenarios like this I like to play safe - using simple constructs can lead to subtle and hard-to-debug problems if a shared resource is mutated from multiple threads. However I just read this article which states that returning from a join() guarantees visibility so looks like it's safe to change it to primitive boolean array. 

> Flaky tests in TestZKUUIDService.java
> -------------------------------------
>
>                 Key: OOZIE-2550
>                 URL: https://issues.apache.org/jira/browse/OOZIE-2550
>             Project: Oozie
>          Issue Type: Bug
>          Components: tests
>            Reporter: Peter Bacsko
>            Assignee: Peter Bacsko
>            Priority: Minor
>         Attachments: OOZIE-2550-001.patch, OOZIE-2550-002.patch
>
>
> 1. Test case testMultipleIDGeneration_withMultiThread in TestZKUUIDService uses an ArrayList which is written by two threads simultaneously. This is dangerous and the list must be externally synchronized to prevent race conditions.
> Another problem is that you cannot put items at arbitrary indexes in an ArrayList. For example, if the list is empty, the following code throws ArrayIndexOutOfBoundException:
> {code}
> List<Boolean> test = new ArrayList<>(10000);
> test.add(22, true);
> {code}
> In an unlucky scheduling event, the following can happen:
> 1. The list has a certain number of elements, the value of "size" inside the list is 1000
> 2. Thread-1 retrieves the next ID from ZK UUID service, which is 1001
> 3. Thread-2 retrieves the next ID from ZK UUID service, which is 1002
> 4. Thread-2 runs faster than Thread-1 and tries to call {{list.add(1002, true)}} which fails. 
> The following error was caught during a test run:
> {code}
> Tests run: 8, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 53.571 sec <<< FAILURE!
> testMultipleIDGeneration_withMultiThread(org.apache.oozie.service.TestZKUUIDService)  Time elapsed: 0.02 sec  <<< ERROR!
> java.lang.IndexOutOfBoundsException: Index: 89, Size: 89
>         at java.util.ArrayList.rangeCheck(ArrayList.java:653)
>         at java.util.ArrayList.get(ArrayList.java:429)
>         at org.apache.oozie.service.TestZKUUIDService.testMultipleIDGeneration_withMultiThread(TestZKUUIDService.java:134)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at junit.framework.TestCase.runTest(TestCase.java:168)
>         at junit.framework.TestCase.runBare(TestCase.java:134)
>         at junit.framework.TestResult$1.protect(TestResult.java:110)
>         at junit.framework.TestResult.runProtected(TestResult.java:128)
>         at junit.framework.TestResult.run(TestResult.java:113)
>         at junit.framework.TestCase.run(TestCase.java:124)
>         at junit.framework.TestSuite.runTest(TestSuite.java:243)
>         at junit.framework.TestSuite.run(TestSuite.java:238)
>         at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
>         at org.junit.runners.Suite.runChild(Suite.java:128)
>         at org.junit.runners.Suite.runChild(Suite.java:24)
>         at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> {code}
> 2. The order in which the tests are executed is random. The problem is that testResetSequence sets the maximum sequence number to 900. Because this value is stored in a static variable inside ZKUUIDService, it affects testIDGeneration and testMultipleIDGeneration if these tests are run after testResetSequence.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)