[jira] [Created] (TEZ-3257) Flaky TestUnorderedPartitionedKVWriter

["Ming Ma (JIRA)" <ji...@apache.org>]

Ming Ma created TEZ-3257:
----------------------------

             Summary: Flaky TestUnorderedPartitionedKVWriter
                 Key: TEZ-3257
                 URL: https://issues.apache.org/jira/browse/TEZ-3257
             Project: Apache Tez
          Issue Type: Bug
            Reporter: Ming Ma


For example,

{noformat}
testSkippedPartitions[1](org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter)  Time elapsed: 0.346 sec  <<< FAILURE!
java.lang.AssertionError: null
	at org.junit.Assert.fail(Assert.java:86)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.junit.Assert.assertTrue(Assert.java:52)
	at org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter.baseTest(TestUnorderedPartitionedKVWriter.java:661)
	at org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter.testSkippedPartitions(TestUnorderedPartitionedKVWriter.java:196)
{noformat}


The reason is the spill onSuccess callback can happen between the following two lines, which means the old buffer will be added to {{availableBuffers}} in time and thus there is no need to initialize another buffer.

{noformat}
      ListenableFuture<SpillResult> future = spillExecutor.submit(
          new SpillCallable(currentBuffer, codec, spilledRecordsCounter, spillPathDetails));
      Futures.addCallback(future, new SpillCallback(spillPathDetails.spillIndex));

      WrappedBuffer wb = getNextAvailableBuffer();
{noformat}


Can we just remove the following check in test code?

{noformat}
    // Verify the status of the buffers
    if (numExpectedSpills == 0) {
      assertEquals(1, kvWriter.numInitializedBuffers);
    } else {
      assertTrue(kvWriter.numInitializedBuffers > 1);
    }
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)