You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Nigel Daley (JIRA)" <ji...@apache.org> on 2007/07/02 06:08:04 UTC
[jira] Created: (HADOOP-1556) 9 unit test failures: file.out.index
already exists
9 unit test failures: file.out.index already exists
---------------------------------------------------
Key: HADOOP-1556
URL: https://issues.apache.org/jira/browse/HADOOP-1556
Project: Hadoop
Issue Type: Bug
Components: mapred
Reporter: Nigel Daley
Fix For: 0.14.0
In my nightly unit testing, 9 unit tests failed June 25 and again July 2, all with this exception:
java.io.IOException: Target build/test/mapred/local/map_0005/file.out.index already exists
at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:246)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:93)
at org.apache.hadoop.fs.RawLocalFileSystem.rename(RawLocalFileSystem.java:207)
at org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:476)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:490)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:593)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:190)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:131)
I'm concerned this could be a race conditrion. The unit tests are:
org.apache.hadoop.fs.TestDFSIO.testIOs
org.apache.hadoop.fs.TestFileSystem.testFs
org.apache.hadoop.mapred.TestAggregates.testAggregates
org.apache.hadoop.mapred.TestFieldSelection.testFieldSelection
org.apache.hadoop.mapred.TestLocalMRNotification.testMR
org.apache.hadoop.mapred.TestMapOutputType.testNoMismatch
org.apache.hadoop.mapred.TestMapRed.testMapred
org.apache.hadoop.mapred.TestMapRed.testCompression
org.apache.hadoop.record.TestRecordMR.testMapred
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1556) 9 unit test failures: file.out.index
already exists
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-1556:
--------------------------------
Assignee: Devaraj Das
Status: Patch Available (was: Open)
> 9 unit test failures: file.out.index already exists
> ---------------------------------------------------
>
> Key: HADOOP-1556
> URL: https://issues.apache.org/jira/browse/HADOOP-1556
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Reporter: Nigel Daley
> Assignee: Devaraj Das
> Fix For: 0.14.0
>
> Attachments: 1556.patch
>
>
> In my nightly unit testing, 9 unit tests failed June 25 and again July 2, all with this exception:
> java.io.IOException: Target build/test/mapred/local/map_0005/file.out.index already exists
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:246)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:93)
> at org.apache.hadoop.fs.RawLocalFileSystem.rename(RawLocalFileSystem.java:207)
> at org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:476)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:490)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:593)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:190)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:131)
> I'm concerned this could be a race conditrion. The unit tests are:
> org.apache.hadoop.fs.TestDFSIO.testIOs
> org.apache.hadoop.fs.TestFileSystem.testFs
> org.apache.hadoop.mapred.TestAggregates.testAggregates
> org.apache.hadoop.mapred.TestFieldSelection.testFieldSelection
> org.apache.hadoop.mapred.TestLocalMRNotification.testMR
> org.apache.hadoop.mapred.TestMapOutputType.testNoMismatch
> org.apache.hadoop.mapred.TestMapRed.testMapred
> org.apache.hadoop.mapred.TestMapRed.testCompression
> org.apache.hadoop.record.TestRecordMR.testMapred
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1556) 9 unit test failures: file.out.index
already exists
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated HADOOP-1556:
------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
> 9 unit test failures: file.out.index already exists
> ---------------------------------------------------
>
> Key: HADOOP-1556
> URL: https://issues.apache.org/jira/browse/HADOOP-1556
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Reporter: Nigel Daley
> Assignee: Devaraj Das
> Fix For: 0.14.0
>
> Attachments: 1556.patch
>
>
> In my nightly unit testing, 9 unit tests failed June 25 and again July 2, all with this exception:
> java.io.IOException: Target build/test/mapred/local/map_0005/file.out.index already exists
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:246)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:93)
> at org.apache.hadoop.fs.RawLocalFileSystem.rename(RawLocalFileSystem.java:207)
> at org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:476)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:490)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:593)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:190)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:131)
> I'm concerned this could be a race conditrion. The unit tests are:
> org.apache.hadoop.fs.TestDFSIO.testIOs
> org.apache.hadoop.fs.TestFileSystem.testFs
> org.apache.hadoop.mapred.TestAggregates.testAggregates
> org.apache.hadoop.mapred.TestFieldSelection.testFieldSelection
> org.apache.hadoop.mapred.TestLocalMRNotification.testMR
> org.apache.hadoop.mapred.TestMapOutputType.testNoMismatch
> org.apache.hadoop.mapred.TestMapRed.testMapred
> org.apache.hadoop.mapred.TestMapRed.testCompression
> org.apache.hadoop.record.TestRecordMR.testMapred
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1556) 9 unit test failures:
file.out.index already exists
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12511552 ]
Tom White commented on HADOOP-1556:
-----------------------------------
I've just committed this. Thanks Devaraj!
> 9 unit test failures: file.out.index already exists
> ---------------------------------------------------
>
> Key: HADOOP-1556
> URL: https://issues.apache.org/jira/browse/HADOOP-1556
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Reporter: Nigel Daley
> Assignee: Devaraj Das
> Fix For: 0.14.0
>
> Attachments: 1556.patch
>
>
> In my nightly unit testing, 9 unit tests failed June 25 and again July 2, all with this exception:
> java.io.IOException: Target build/test/mapred/local/map_0005/file.out.index already exists
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:246)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:93)
> at org.apache.hadoop.fs.RawLocalFileSystem.rename(RawLocalFileSystem.java:207)
> at org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:476)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:490)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:593)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:190)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:131)
> I'm concerned this could be a race conditrion. The unit tests are:
> org.apache.hadoop.fs.TestDFSIO.testIOs
> org.apache.hadoop.fs.TestFileSystem.testFs
> org.apache.hadoop.mapred.TestAggregates.testAggregates
> org.apache.hadoop.mapred.TestFieldSelection.testFieldSelection
> org.apache.hadoop.mapred.TestLocalMRNotification.testMR
> org.apache.hadoop.mapred.TestMapOutputType.testNoMismatch
> org.apache.hadoop.mapred.TestMapRed.testMapred
> org.apache.hadoop.mapred.TestMapRed.testCompression
> org.apache.hadoop.record.TestRecordMR.testMapred
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1556) 9 unit test failures:
file.out.index already exists
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12509611 ]
Devaraj Das commented on HADOOP-1556:
-------------------------------------
By the way, couldn't reproduce this problem at my end.
> 9 unit test failures: file.out.index already exists
> ---------------------------------------------------
>
> Key: HADOOP-1556
> URL: https://issues.apache.org/jira/browse/HADOOP-1556
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Reporter: Nigel Daley
> Fix For: 0.14.0
>
> Attachments: 1556.patch
>
>
> In my nightly unit testing, 9 unit tests failed June 25 and again July 2, all with this exception:
> java.io.IOException: Target build/test/mapred/local/map_0005/file.out.index already exists
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:246)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:93)
> at org.apache.hadoop.fs.RawLocalFileSystem.rename(RawLocalFileSystem.java:207)
> at org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:476)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:490)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:593)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:190)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:131)
> I'm concerned this could be a race conditrion. The unit tests are:
> org.apache.hadoop.fs.TestDFSIO.testIOs
> org.apache.hadoop.fs.TestFileSystem.testFs
> org.apache.hadoop.mapred.TestAggregates.testAggregates
> org.apache.hadoop.mapred.TestFieldSelection.testFieldSelection
> org.apache.hadoop.mapred.TestLocalMRNotification.testMR
> org.apache.hadoop.mapred.TestMapOutputType.testNoMismatch
> org.apache.hadoop.mapred.TestMapRed.testMapred
> org.apache.hadoop.mapred.TestMapRed.testCompression
> org.apache.hadoop.record.TestRecordMR.testMapred
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1556) 9 unit test failures: file.out.index
already exists
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj Das updated HADOOP-1556:
--------------------------------
Attachment: 1556.patch
I did some analysis but couldn't pinpoint the exact cause of this issue. I did notice that all the failures emanate out of the LocalJobRunner. What seems to happen is that files are left over from a previous job (test) and hence rename fails for the current test, and all tests thereafter.
I have attached a patch that makes sure that files are deleted at the end of a LocalJob run. It also handles the case of #reduces=0.
> 9 unit test failures: file.out.index already exists
> ---------------------------------------------------
>
> Key: HADOOP-1556
> URL: https://issues.apache.org/jira/browse/HADOOP-1556
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Reporter: Nigel Daley
> Fix For: 0.14.0
>
> Attachments: 1556.patch
>
>
> In my nightly unit testing, 9 unit tests failed June 25 and again July 2, all with this exception:
> java.io.IOException: Target build/test/mapred/local/map_0005/file.out.index already exists
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:246)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:93)
> at org.apache.hadoop.fs.RawLocalFileSystem.rename(RawLocalFileSystem.java:207)
> at org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:476)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:490)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:593)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:190)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:131)
> I'm concerned this could be a race conditrion. The unit tests are:
> org.apache.hadoop.fs.TestDFSIO.testIOs
> org.apache.hadoop.fs.TestFileSystem.testFs
> org.apache.hadoop.mapred.TestAggregates.testAggregates
> org.apache.hadoop.mapred.TestFieldSelection.testFieldSelection
> org.apache.hadoop.mapred.TestLocalMRNotification.testMR
> org.apache.hadoop.mapred.TestMapOutputType.testNoMismatch
> org.apache.hadoop.mapred.TestMapRed.testMapred
> org.apache.hadoop.mapred.TestMapRed.testCompression
> org.apache.hadoop.record.TestRecordMR.testMapred
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1556) 9 unit test failures:
file.out.index already exists
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12511731 ]
Hudson commented on HADOOP-1556:
--------------------------------
Integrated in Hadoop-Nightly #151 (See [http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/151/])
> 9 unit test failures: file.out.index already exists
> ---------------------------------------------------
>
> Key: HADOOP-1556
> URL: https://issues.apache.org/jira/browse/HADOOP-1556
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Reporter: Nigel Daley
> Assignee: Devaraj Das
> Fix For: 0.14.0
>
> Attachments: 1556.patch
>
>
> In my nightly unit testing, 9 unit tests failed June 25 and again July 2, all with this exception:
> java.io.IOException: Target build/test/mapred/local/map_0005/file.out.index already exists
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:246)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:93)
> at org.apache.hadoop.fs.RawLocalFileSystem.rename(RawLocalFileSystem.java:207)
> at org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:476)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:490)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:593)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:190)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:131)
> I'm concerned this could be a race conditrion. The unit tests are:
> org.apache.hadoop.fs.TestDFSIO.testIOs
> org.apache.hadoop.fs.TestFileSystem.testFs
> org.apache.hadoop.mapred.TestAggregates.testAggregates
> org.apache.hadoop.mapred.TestFieldSelection.testFieldSelection
> org.apache.hadoop.mapred.TestLocalMRNotification.testMR
> org.apache.hadoop.mapred.TestMapOutputType.testNoMismatch
> org.apache.hadoop.mapred.TestMapRed.testMapred
> org.apache.hadoop.mapred.TestMapRed.testCompression
> org.apache.hadoop.record.TestRecordMR.testMapred
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1556) 9 unit test failures:
file.out.index already exists
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12509763 ]
Hadoop QA commented on HADOOP-1556:
-----------------------------------
+1
http://issues.apache.org/jira/secure/attachment/12360949/1556.patch applied and successfully tested against trunk revision r552548.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/358/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/358/console
> 9 unit test failures: file.out.index already exists
> ---------------------------------------------------
>
> Key: HADOOP-1556
> URL: https://issues.apache.org/jira/browse/HADOOP-1556
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Reporter: Nigel Daley
> Assignee: Devaraj Das
> Fix For: 0.14.0
>
> Attachments: 1556.patch
>
>
> In my nightly unit testing, 9 unit tests failed June 25 and again July 2, all with this exception:
> java.io.IOException: Target build/test/mapred/local/map_0005/file.out.index already exists
> at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:246)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:93)
> at org.apache.hadoop.fs.RawLocalFileSystem.rename(RawLocalFileSystem.java:207)
> at org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:476)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:490)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:593)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:190)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:131)
> I'm concerned this could be a race conditrion. The unit tests are:
> org.apache.hadoop.fs.TestDFSIO.testIOs
> org.apache.hadoop.fs.TestFileSystem.testFs
> org.apache.hadoop.mapred.TestAggregates.testAggregates
> org.apache.hadoop.mapred.TestFieldSelection.testFieldSelection
> org.apache.hadoop.mapred.TestLocalMRNotification.testMR
> org.apache.hadoop.mapred.TestMapOutputType.testNoMismatch
> org.apache.hadoop.mapred.TestMapRed.testMapred
> org.apache.hadoop.mapred.TestMapRed.testCompression
> org.apache.hadoop.record.TestRecordMR.testMapred
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.