[jira] [Commented] (MAPREDUCE-4416) Some tests run twice or fail if Clover is enabled

["Kihwal Lee (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409918#comment-13409918 ] 

Kihwal Lee commented on MAPREDUCE-4416:
---------------------------------------

There are actually two different TestJobConf. One in o.a.h.conf and another one in o.a.h.mapred. It's confusing, but not really a problem.

I had 37 failures/errors in jobclient when Clover is enabled.

{noformat}
Failed tests:   testChild(org.apache.hadoop.mapreduce.TestChild)
  testDefaultCleanupAndAbort(org.apache.hadoop.mapreduce.lib.output.TestJobOutputCommitter): Job failed!
  testCustomAbort(org.apache.hadoop.mapreduce.lib.output.TestJobOutputCommitter): Job failed!
  testCustomCleanup(org.apache.hadoop.mapreduce.lib.output.TestJobOutputCommitter): Job failed!
  testValidProxyUser(org.apache.hadoop.mapreduce.v2.TestMiniMRProxyUser)
  testJobSucceed(org.apache.hadoop.mapreduce.v2.TestMROldApiJobs): Job expected to succeed failed
  testJobFail(org.apache.hadoop.mapreduce.v2.TestMROldApiJobs)
  testSleepJob(org.apache.hadoop.mapreduce.v2.TestMRJobs)
  testRandomWriter(org.apache.hadoop.mapreduce.v2.TestMRJobs)
  testDistributedCache(org.apache.hadoop.mapreduce.v2.TestMRJobs)
  testSleepJob(org.apache.hadoop.mapreduce.v2.TestUberAM)
  testRandomWriter(org.apache.hadoop.mapreduce.v2.TestUberAM)
  testFailingMapper(org.apache.hadoop.mapreduce.v2.TestUberAM): expected:<false> but was:<true>
  testSpeculativeExecution(org.apache.hadoop.mapreduce.v2.TestSpeculativeExecution)
  testLazyOutput(org.apache.hadoop.mapreduce.TestMapReduceLazyOutput)
  testHeapUsageCounter(org.apache.hadoop.mapred.TestJobCounters): Job job_1341837408279_0001 failed!
  testDefaultCleanupAndAbort(org.apache.hadoop.mapred.TestJobCleanup): Done file "/home/y/var/builds/thread2/workspace/Cloud-Hadoop-All-2.0-Component/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/test-dir/test-job-cleanup/output-0/_SUCCESS" missing for job job_1341837505379_0001
  testCustomAbort(org.apache.hadoop.mapred.TestJobCleanup): Done file "/home/y/var/builds/thread2/workspace/Cloud-Hadoop-All-2.0-Component/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/test-dir/test-job-cleanup/output-1/_SUCCESS" missing for job job_1341837505379_0002
  testCustomCleanup(org.apache.hadoop.mapred.TestJobCleanup): Done file "/home/y/var/builds/thread2/workspace/Cloud-Hadoop-All-2.0-Component/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/test-dir/test-job-cleanup/output-2/_custom_cleanup" missing for job job_1341837505379_0003
  testTaskTempDir(org.apache.hadoop.mapred.TestMiniMRChildTask)
  testTaskEnv(org.apache.hadoop.mapred.TestMiniMRChildTask): The environment checker job failed.
  testTaskOldEnv(org.apache.hadoop.mapred.TestMiniMRChildTask): The environment checker job failed.
  testJob(org.apache.hadoop.mapred.TestMiniMRClientCluster)

Tests in error: 
  testFailingMapper(org.apache.hadoop.mapreduce.v2.TestMRJobs): 0
  testMR(org.apache.hadoop.mapred.TestClusterMRNotification): Job failed!
  testComplexName(org.apache.hadoop.mapred.TestJobName): Job failed!
  testComplexNameWithRegex(org.apache.hadoop.mapred.TestJobName): Job failed!
  testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem): Job failed!
  testClassPath(org.apache.hadoop.mapred.TestMiniMRClasspath): Job failed!
  testExternalWritable(org.apache.hadoop.mapred.TestMiniMRClasspath): Job failed!
  testWithDFS(org.apache.hadoop.mapred.TestJobSysDirWithDFS): Job failed!
  testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem): Job failed!
  testLazyOutput(org.apache.hadoop.mapred.TestLazyOutput): Job failed!
  testDistinctUsers(org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers): Job failed!
  testMultipleSpills(org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers): Job failed!
  testMapReduce(org.apache.hadoop.mapred.TestClusterMapReduceTestCase): Job failed!
  testMapReduceRestarting(org.apache.hadoop.mapred.TestClusterMapReduceTestCase): Job failed!

Tests run: 381, Failures: 23, Errors: 14, Skipped: 14
{noformat}


For the failing test cases, the container's stderr files contain the following:

{noformat}
[CLOVER] FATAL ERROR: Clover could not be initialised. Are you sure you have
Clover in the runtime classpath? (class
java.lang.NoClassDefFoundError:com_cenqua_clover/CloverVersionInfo)
{noformat}


                
> Some tests run twice or fail if Clover is enabled
> -------------------------------------------------
>
>                 Key: MAPREDUCE-4416
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4416
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: client, mrv2
>    Affects Versions: 2.0.0-alpha, 3.0.0
>            Reporter: Kihwal Lee
>             Fix For: 2.0.1-alpha, 3.0.0
>
>
> Some tests run twice. E.g. try mvn test -Dtest=TestJobConf. It runs under hadoop-mapreduce-client-core and hadoop-mapreduce-client-jobclient.
> There are number of tests running under hadoop-mapreduce-client-jobclient that fail if Clover is enabled. Whenever a job is launched, AM doesn't start because it can't locate the clover jar file.
> It seems this started happening after MAPREDUCE-4253.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira