[jira] [Comment Edited] (TEZ-2340) TestRecoveryParser fails

["Jeff Zhang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14504739#comment-14504739 ] 

Jeff Zhang edited comment on TEZ-2340 at 4/21/15 10:34 AM:
-----------------------------------------------------------

The root cause of the test failure is that all the testcases use the directory for recovery so that the delete operation may fails because the last test case may not close the file stream. Attach the patch to use different recovery path for each test case.

{code}
2015-04-17 07:23:55,672 WARN  [main] fs.FileUtil (FileUtil.java:deleteImpl(187)) - Failed to delete file or dir [D:\w\tez\tez-dag\target\org.apache.tez.dag.app.TestRecoveryParser-tmpDir\recovery\1\.summary.crc]: it still exists.
2015-04-17 07:23:55,674 WARN  [main] fs.FileUtil (FileUtil.java:deleteImpl(187)) - Failed to delete file or dir [D:\w\tez\tez-dag\target\org.apache.tez.dag.app.TestRecoveryParser-tmpDir\recovery\1\summary]: it still exists.
{code}


was (Author: zjffdu):
The root cause of the test failure is that all the testcases use the directory for recovery so that the delete operation may fails because the last test case may not close the file stream. Attach the patch to use different recovery path for each test case 

> TestRecoveryParser fails
> ------------------------
>
>                 Key: TEZ-2340
>                 URL: https://issues.apache.org/jira/browse/TEZ-2340
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Jeff Zhang
>            Assignee: Jeff Zhang
>         Attachments: TEZ-2340-1.patch
>
>
> Stacktrace
> {code}
> java.io.IOException: Not supported
> 	at org.apache.hadoop.fs.ChecksumFileSystem.append(ChecksumFileSystem.java:352)
> 	at org.apache.hadoop.fs.FileSystem.append(FileSystem.java:1174)
> 	at org.apache.tez.dag.history.recovery.RecoveryService.handleSummaryEvent(RecoveryService.java:365)
> 	at org.apache.tez.dag.history.recovery.RecoveryService.handle(RecoveryService.java:285)
> 	at org.apache.tez.dag.app.TestRecoveryParser.testSkipAllOtherEvents_1(TestRecoveryParser.java:138)
> {code}
> Standard Output
> {code}
> 2015-04-17 07:23:55,672 WARN  [main] fs.FileUtil (FileUtil.java:deleteImpl(187)) - Failed to delete file or dir [D:\w\tez\tez-dag\target\org.apache.tez.dag.app.TestRecoveryParser-tmpDir\recovery\1\.summary.crc]: it still exists.
> 2015-04-17 07:23:55,674 WARN  [main] fs.FileUtil (FileUtil.java:deleteImpl(187)) - Failed to delete file or dir [D:\w\tez\tez-dag\target\org.apache.tez.dag.app.TestRecoveryParser-tmpDir\recovery\1\summary]: it still exists.
> 2015-04-17 07:23:55,703 INFO  [Thread-5] impl.TestDAGImpl (TestDAGImpl.java:createTestDAGPlan(446)) - Setting up dag plan
> 2015-04-17 07:23:55,722 INFO  [Thread-5] recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
> 2015-04-17 07:23:55,723 INFO  [Thread-5] recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
> 2015-04-17 07:23:55,724 ERROR [Thread-5] recovery.RecoveryService (RecoveryService.java:handle(314)) - Error handling summary event, eventType=DAG_SUBMITTED
> java.io.IOException: Not supported
> 	at org.apache.hadoop.fs.ChecksumFileSystem.append(ChecksumFileSystem.java:352)
> 	at org.apache.hadoop.fs.FileSystem.append(FileSystem.java:1174)
> 	at org.apache.tez.dag.history.recovery.RecoveryService.handleSummaryEvent(RecoveryService.java:365)
> 	at org.apache.tez.dag.history.recovery.RecoveryService.handle(RecoveryService.java:285)
> 	at org.apache.tez.dag.app.TestRecoveryParser.testSkipAllOtherEvents_1(TestRecoveryParser.java:138)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:601)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> 2015-04-17 07:23:55,724 ERROR [Thread-5] recovery.RecoveryService (RecoveryService.java:handle(318)) - Adding a flag to ensure next AM attempt does not start up, flagFile=target/org.apache.tez.dag.app.TestRecoveryParser-tmpDir/recovery/1/RecoveryFatalErrorOccurred
> 2015-04-17 07:23:55,725 ERROR [Thread-5] recovery.RecoveryService (RecoveryService.java:handle(323)) - Recovery failure occurred. Skipping all events
> 2015-04-17 07:23:55,756 ERROR [RecoveryEventHandlingThread] recovery.RecoveryService (RecoveryService.java:run(146)) - Recovery failure occurred. Stopping recovery thread. Current eventQueueSize=0
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)