[jira] [Commented] (TEZ-3257) Flaky TestUnorderedPartitionedKVWriter

["Siddharth Seth (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-3257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290412#comment-15290412 ] 

Siddharth Seth commented on TEZ-3257:
-------------------------------------

That makes sense to me. We could create another jira to add a test around the buffer creation.

> Flaky TestUnorderedPartitionedKVWriter
> --------------------------------------
>
>                 Key: TEZ-3257
>                 URL: https://issues.apache.org/jira/browse/TEZ-3257
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Ming Ma
>
> For example,
> {noformat}
> testSkippedPartitions[1](org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter)  Time elapsed: 0.346 sec  <<< FAILURE!
> java.lang.AssertionError: null
> 	at org.junit.Assert.fail(Assert.java:86)
> 	at org.junit.Assert.assertTrue(Assert.java:41)
> 	at org.junit.Assert.assertTrue(Assert.java:52)
> 	at org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter.baseTest(TestUnorderedPartitionedKVWriter.java:661)
> 	at org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter.testSkippedPartitions(TestUnorderedPartitionedKVWriter.java:196)
> {noformat}
> The reason is the spill onSuccess callback can happen between the following two lines, which means the old buffer will be added to {{availableBuffers}} in time and thus there is no need to initialize another buffer.
> {noformat}
>       ListenableFuture<SpillResult> future = spillExecutor.submit(
>           new SpillCallable(currentBuffer, codec, spilledRecordsCounter, spillPathDetails));
>       Futures.addCallback(future, new SpillCallback(spillPathDetails.spillIndex));
>       WrappedBuffer wb = getNextAvailableBuffer();
> {noformat}
> Can we just remove the following check in test code?
> {noformat}
>     // Verify the status of the buffers
>     if (numExpectedSpills == 0) {
>       assertEquals(1, kvWriter.numInitializedBuffers);
>     } else {
>       assertTrue(kvWriter.numInitializedBuffers > 1);
>     }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)