You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Jonathan Turner Eagles (Jira)" <ji...@apache.org> on 2020/02/07 14:42:00 UTC
[jira] [Updated] (TEZ-3664) Flaky tests due to writing to /tmp
directory
[ https://issues.apache.org/jira/browse/TEZ-3664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Turner Eagles updated TEZ-3664:
----------------------------------------
Summary: Flaky tests due to writing to /tmp directory (was: testSingleSpill_WithPipelinedShuffle[0] fails intermittently)
> Flaky tests due to writing to /tmp directory
> --------------------------------------------
>
> Key: TEZ-3664
> URL: https://issues.apache.org/jira/browse/TEZ-3664
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Yesha Vora
> Assignee: Jonathan Turner Eagles
> Priority: Major
> Attachments: TEZ-3664.001.patch, TEZ-3664.002.patch, TEZ-3664.003.patch, TEZ-3664.004.patch, TEZ-3664.005.patch, TEZ-3664.006.patch, TEZ-3664.007.patch, TEZ-3664.008.patch
>
>
> testSingleSpill_WithPipelinedShuffle[0] unit test fails intermittently
> {code}
> Stacktrace
> java.lang.AssertionError: null
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter.baseTestWithPipelinedTransfer(TestUnorderedPartitionedKVWriter.java:503)
> at org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter.testSingleSpill_WithPipelinedShuffle(TestUnorderedPartitionedKVWriter.java:414)
> Standard Output
> 2017-03-20 19:50:52,538 INFO [main] writers.TestUnorderedPartitionedKVWriter (TestUnorderedPartitionedKVWriter.java:setup(124)) - Setup. Using test dir: /tmp/TestUnorderedPartitionedKVWriter
> 2017-03-20 19:50:53,065 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
> 2017-03-20 19:50:53,856 INFO [Thread-5] counters.Limits (Limits.java:ensureInitialized(60)) - Counter limits initialized with parameters: GROUP_NAME_MAX=256, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
> 2017-03-20 19:50:53,861 INFO [Thread-5] writers.BaseUnorderedPartitionedKVWriter (BaseUnorderedPartitionedKVWriter.java:<init>(148)) - Instantiating Partitioner: [org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter$PartitionerForTest]
> 2017-03-20 19:50:53,870 INFO [Thread-5] writers.UnorderedPartitionedKVWriter (UnorderedPartitionedKVWriter.java:<init>(221)) - destinationVertexName: numBuffers=2, sizePerBuffer=1024, skipBuffers=false, pipelinedShuffle=true, numPartitions=10
> 2017-03-20 19:50:53,871 INFO [Thread-5] writers.UnorderedPartitionedKVWriter (UnorderedPartitionedKVWriter.java:setupNextBuffer(333)) - destinationVertexName: Moving to next buffer and triggering spill
> 2017-03-20 19:50:53,913 INFO [UnorderedOutSpiller {destinationVertexName}] writers.UnorderedPartitionedKVWriter (UnorderedPartitionedKVWriter.java:callInternal(432)) - destinationVertexName: Finished spill 0
> 2017-03-20 19:50:53,934 INFO [Thread-5] writers.UnorderedPartitionedKVWriter (UnorderedPartitionedKVWriter.java:sendPipelinedEventForSpill(914)) - destinationVertexName: Adding spill event for spill (final update=false), spillId=0
> 2017-03-20 19:50:53,938 INFO [Thread-5] writers.UnorderedPartitionedKVWriter (UnorderedPartitionedKVWriter.java:close(473)) - destinationVertexName: Waiting for all spills to complete : Pending : 0
> 2017-03-20 19:50:53,938 INFO [Thread-5] writers.UnorderedPartitionedKVWriter (UnorderedPartitionedKVWriter.java:close(492)) - destinationVertexName: All spills complete
> 2017-03-20 19:50:53,948 INFO [Thread-5] writers.UnorderedPartitionedKVWriter (UnorderedPartitionedKVWriter.java:callInternal(432)) - destinationVertexName: Finished spill 1
> 2017-03-20 19:50:53,959 INFO [Thread-5] writers.UnorderedPartitionedKVWriter (UnorderedPartitionedKVWriter.java:sendPipelinedEventForSpill(914)) - destinationVertexName: Adding spill event for spill (final update=true), spillId=1
> 2017-03-20 19:50:53,972 INFO [main] writers.TestUnorderedPartitionedKVWriter (TestUnorderedPartitionedKVWriter.java:cleanup(132)) - CleanUp{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)