You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org> on 2008/08/13 20:20:44 UTC
[jira] Created: (HADOOP-3950) TestMapRed and TestMiniMRDFSSort
failed on trunk
TestMapRed and TestMiniMRDFSSort failed on trunk
------------------------------------------------
Key: HADOOP-3950
URL: https://issues.apache.org/jira/browse/HADOOP-3950
Project: Hadoop Core
Issue Type: Bug
Reporter: Tsz Wo (Nicholas), SZE
- TestMapRed failed on Windows and Linux
{noformat}
Testcase: testMapred took 7.36 sec
Caused an ERROR
Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
java.io.IOException: Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:195)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:740)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1036)
at org.apache.hadoop.mapred.TestMapRed.launch(TestMapRed.java:614)
at org.apache.hadoop.mapred.TestMapRed.testMapred(TestMapRed.java:248)
{noformat}
- TestMiniMRDFSSort failed on Windows
{noformat}
Testcase: testMapReduceSort took 21.312 sec
Caused an ERROR
File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
java.io.FileNotFoundException: File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:404)
at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:677)
at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1416)
at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1411)
at org.apache.hadoop.mapred.SortValidator$RecordStatsChecker.checkRecords(SortValidator.java:370)
at org.apache.hadoop.mapred.SortValidator.run(SortValidator.java:561)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.mapred.TestMiniMRDFSSort.runSortValidator(TestMiniMRDFSSort.java:73)
at org.apache.hadoop.mapred.TestMiniMRDFSSort.testMapReduceSort(TestMiniMRDFSSort.java:99)
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3950) TestMapRed and TestMiniMRDFSSort
failed on trunk
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated HADOOP-3950:
----------------------------------
Status: Open (was: Patch Available)
> TestMapRed and TestMiniMRDFSSort failed on trunk
> ------------------------------------------------
>
> Key: HADOOP-3950
> URL: https://issues.apache.org/jira/browse/HADOOP-3950
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred, test
> Affects Versions: 0.19.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Enis Soztutar
> Fix For: 0.19.0
>
> Attachments: Hadoop-3950_v1.patch
>
>
> - TestMapRed failed on Windows and Linux
> {noformat}
> Testcase: testMapred took 7.36 sec
> Caused an ERROR
> Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> java.io.IOException: Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:195)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:740)
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1036)
> at org.apache.hadoop.mapred.TestMapRed.launch(TestMapRed.java:614)
> at org.apache.hadoop.mapred.TestMapRed.testMapred(TestMapRed.java:248)
> {noformat}
> - TestMiniMRDFSSort failed on Windows
> {noformat}
> Testcase: testMapReduceSort took 21.312 sec
> Caused an ERROR
> File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> java.io.FileNotFoundException: File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:404)
> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:677)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1416)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1411)
> at org.apache.hadoop.mapred.SortValidator$RecordStatsChecker.checkRecords(SortValidator.java:370)
> at org.apache.hadoop.mapred.SortValidator.run(SortValidator.java:561)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.runSortValidator(TestMiniMRDFSSort.java:73)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.testMapReduceSort(TestMiniMRDFSSort.java:99)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3950) TestMapRed and TestMiniMRDFSSort
failed on trunk
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-3950:
----------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Enis!
> TestMapRed and TestMiniMRDFSSort failed on trunk
> ------------------------------------------------
>
> Key: HADOOP-3950
> URL: https://issues.apache.org/jira/browse/HADOOP-3950
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred, test
> Affects Versions: 0.19.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Enis Soztutar
> Fix For: 0.19.0
>
> Attachments: Hadoop-3950_v1.patch
>
>
> - TestMapRed failed on Windows and Linux
> {noformat}
> Testcase: testMapred took 7.36 sec
> Caused an ERROR
> Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> java.io.IOException: Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:195)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:740)
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1036)
> at org.apache.hadoop.mapred.TestMapRed.launch(TestMapRed.java:614)
> at org.apache.hadoop.mapred.TestMapRed.testMapred(TestMapRed.java:248)
> {noformat}
> - TestMiniMRDFSSort failed on Windows
> {noformat}
> Testcase: testMapReduceSort took 21.312 sec
> Caused an ERROR
> File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> java.io.FileNotFoundException: File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:404)
> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:677)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1416)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1411)
> at org.apache.hadoop.mapred.SortValidator$RecordStatsChecker.checkRecords(SortValidator.java:370)
> at org.apache.hadoop.mapred.SortValidator.run(SortValidator.java:561)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.runSortValidator(TestMiniMRDFSSort.java:73)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.testMapReduceSort(TestMiniMRDFSSort.java:99)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3950) TestMapRed and TestMiniMRDFSSort
failed on trunk
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HADOOP-3950:
----------------------------------
Attachment: Hadoop-3950_v1.patch
Attaching a patch, which fixes MiniMRCluster to wait for all the TaskTrackers to be recognized by JobTracker at initialization.
> TestMapRed and TestMiniMRDFSSort failed on trunk
> ------------------------------------------------
>
> Key: HADOOP-3950
> URL: https://issues.apache.org/jira/browse/HADOOP-3950
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred, test
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Enis Soztutar
> Attachments: Hadoop-3950_v1.patch
>
>
> - TestMapRed failed on Windows and Linux
> {noformat}
> Testcase: testMapred took 7.36 sec
> Caused an ERROR
> Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> java.io.IOException: Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:195)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:740)
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1036)
> at org.apache.hadoop.mapred.TestMapRed.launch(TestMapRed.java:614)
> at org.apache.hadoop.mapred.TestMapRed.testMapred(TestMapRed.java:248)
> {noformat}
> - TestMiniMRDFSSort failed on Windows
> {noformat}
> Testcase: testMapReduceSort took 21.312 sec
> Caused an ERROR
> File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> java.io.FileNotFoundException: File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:404)
> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:677)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1416)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1411)
> at org.apache.hadoop.mapred.SortValidator$RecordStatsChecker.checkRecords(SortValidator.java:370)
> at org.apache.hadoop.mapred.SortValidator.run(SortValidator.java:561)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.runSortValidator(TestMiniMRDFSSort.java:73)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.testMapReduceSort(TestMiniMRDFSSort.java:99)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3950) TestMapRed and TestMiniMRDFSSort
failed on trunk
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622317#action_12622317 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-3950:
------------------------------------------------
After TestMapRed failed, a directory "mapred.loadtest" was created in current directory. We should use "test.build.data" for testing but not the current directory.
> TestMapRed and TestMiniMRDFSSort failed on trunk
> ------------------------------------------------
>
> Key: HADOOP-3950
> URL: https://issues.apache.org/jira/browse/HADOOP-3950
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred, test
> Reporter: Tsz Wo (Nicholas), SZE
>
> - TestMapRed failed on Windows and Linux
> {noformat}
> Testcase: testMapred took 7.36 sec
> Caused an ERROR
> Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> java.io.IOException: Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:195)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:740)
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1036)
> at org.apache.hadoop.mapred.TestMapRed.launch(TestMapRed.java:614)
> at org.apache.hadoop.mapred.TestMapRed.testMapred(TestMapRed.java:248)
> {noformat}
> - TestMiniMRDFSSort failed on Windows
> {noformat}
> Testcase: testMapReduceSort took 21.312 sec
> Caused an ERROR
> File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> java.io.FileNotFoundException: File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:404)
> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:677)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1416)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1411)
> at org.apache.hadoop.mapred.SortValidator$RecordStatsChecker.checkRecords(SortValidator.java:370)
> at org.apache.hadoop.mapred.SortValidator.run(SortValidator.java:561)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.runSortValidator(TestMiniMRDFSSort.java:73)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.testMapReduceSort(TestMiniMRDFSSort.java:99)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3950) TestMapRed and TestMiniMRDFSSort
failed on trunk
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622309#action_12622309 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-3950:
------------------------------------------------
Both tests failed on Hudson in many recent builds.
> TestMapRed and TestMiniMRDFSSort failed on trunk
> ------------------------------------------------
>
> Key: HADOOP-3950
> URL: https://issues.apache.org/jira/browse/HADOOP-3950
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred, test
> Reporter: Tsz Wo (Nicholas), SZE
>
> - TestMapRed failed on Windows and Linux
> {noformat}
> Testcase: testMapred took 7.36 sec
> Caused an ERROR
> Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> java.io.IOException: Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:195)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:740)
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1036)
> at org.apache.hadoop.mapred.TestMapRed.launch(TestMapRed.java:614)
> at org.apache.hadoop.mapred.TestMapRed.testMapred(TestMapRed.java:248)
> {noformat}
> - TestMiniMRDFSSort failed on Windows
> {noformat}
> Testcase: testMapReduceSort took 21.312 sec
> Caused an ERROR
> File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> java.io.FileNotFoundException: File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:404)
> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:677)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1416)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1411)
> at org.apache.hadoop.mapred.SortValidator$RecordStatsChecker.checkRecords(SortValidator.java:370)
> at org.apache.hadoop.mapred.SortValidator.run(SortValidator.java:561)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.runSortValidator(TestMiniMRDFSSort.java:73)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.testMapReduceSort(TestMiniMRDFSSort.java:99)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3950) TestMapRed and TestMiniMRDFSSort
failed on trunk
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622788#action_12622788 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-3950:
------------------------------------------------
After updated with the patch in HADOOP-3946, TestMapRed is passed in Windows and Linux but TestMiniMRDFSSort is still failed in Windows.
> TestMapRed and TestMiniMRDFSSort failed on trunk
> ------------------------------------------------
>
> Key: HADOOP-3950
> URL: https://issues.apache.org/jira/browse/HADOOP-3950
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred, test
> Reporter: Tsz Wo (Nicholas), SZE
>
> - TestMapRed failed on Windows and Linux
> {noformat}
> Testcase: testMapred took 7.36 sec
> Caused an ERROR
> Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> java.io.IOException: Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:195)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:740)
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1036)
> at org.apache.hadoop.mapred.TestMapRed.launch(TestMapRed.java:614)
> at org.apache.hadoop.mapred.TestMapRed.testMapred(TestMapRed.java:248)
> {noformat}
> - TestMiniMRDFSSort failed on Windows
> {noformat}
> Testcase: testMapReduceSort took 21.312 sec
> Caused an ERROR
> File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> java.io.FileNotFoundException: File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:404)
> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:677)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1416)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1411)
> at org.apache.hadoop.mapred.SortValidator$RecordStatsChecker.checkRecords(SortValidator.java:370)
> at org.apache.hadoop.mapred.SortValidator.run(SortValidator.java:561)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.runSortValidator(TestMiniMRDFSSort.java:73)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.testMapReduceSort(TestMiniMRDFSSort.java:99)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3950) TestMapRed and TestMiniMRDFSSort
failed on trunk
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE updated HADOOP-3950:
-------------------------------------------
Component/s: test
mapred
> TestMapRed and TestMiniMRDFSSort failed on trunk
> ------------------------------------------------
>
> Key: HADOOP-3950
> URL: https://issues.apache.org/jira/browse/HADOOP-3950
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred, test
> Reporter: Tsz Wo (Nicholas), SZE
>
> - TestMapRed failed on Windows and Linux
> {noformat}
> Testcase: testMapred took 7.36 sec
> Caused an ERROR
> Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> java.io.IOException: Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:195)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:740)
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1036)
> at org.apache.hadoop.mapred.TestMapRed.launch(TestMapRed.java:614)
> at org.apache.hadoop.mapred.TestMapRed.testMapred(TestMapRed.java:248)
> {noformat}
> - TestMiniMRDFSSort failed on Windows
> {noformat}
> Testcase: testMapReduceSort took 21.312 sec
> Caused an ERROR
> File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> java.io.FileNotFoundException: File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:404)
> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:677)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1416)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1411)
> at org.apache.hadoop.mapred.SortValidator$RecordStatsChecker.checkRecords(SortValidator.java:370)
> at org.apache.hadoop.mapred.SortValidator.run(SortValidator.java:561)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.runSortValidator(TestMiniMRDFSSort.java:73)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.testMapReduceSort(TestMiniMRDFSSort.java:99)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3950) TestMapRed and TestMiniMRDFSSort
failed on trunk
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12623619#action_12623619 ]
Enis Soztutar commented on HADOOP-3950:
---------------------------------------
TestMapRed passes after HADOOP-3946, but TestMiniMRDFSSort fails for me.
I tracked down the issue, which seems to be due to the call to {{JobClient.getClusterStatus().getTaskTrackers()}} returns 0, which in turn causes the RandomWriter to run with 0 maps.
I suspect the tasktracker has not finished its first heathbeat, so a call to getTaskTrackers() returns 0. I will look into it.
> TestMapRed and TestMiniMRDFSSort failed on trunk
> ------------------------------------------------
>
> Key: HADOOP-3950
> URL: https://issues.apache.org/jira/browse/HADOOP-3950
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred, test
> Reporter: Tsz Wo (Nicholas), SZE
>
> - TestMapRed failed on Windows and Linux
> {noformat}
> Testcase: testMapred took 7.36 sec
> Caused an ERROR
> Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> java.io.IOException: Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:195)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:740)
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1036)
> at org.apache.hadoop.mapred.TestMapRed.launch(TestMapRed.java:614)
> at org.apache.hadoop.mapred.TestMapRed.testMapred(TestMapRed.java:248)
> {noformat}
> - TestMiniMRDFSSort failed on Windows
> {noformat}
> Testcase: testMapReduceSort took 21.312 sec
> Caused an ERROR
> File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> java.io.FileNotFoundException: File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:404)
> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:677)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1416)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1411)
> at org.apache.hadoop.mapred.SortValidator$RecordStatsChecker.checkRecords(SortValidator.java:370)
> at org.apache.hadoop.mapred.SortValidator.run(SortValidator.java:561)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.runSortValidator(TestMiniMRDFSSort.java:73)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.testMapReduceSort(TestMiniMRDFSSort.java:99)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3950) TestMapRed and TestMiniMRDFSSort
failed on trunk
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar updated HADOOP-3950:
----------------------------------
Fix Version/s: 0.19.0
Affects Version/s: 0.19.0
Status: Patch Available (was: Open)
> TestMapRed and TestMiniMRDFSSort failed on trunk
> ------------------------------------------------
>
> Key: HADOOP-3950
> URL: https://issues.apache.org/jira/browse/HADOOP-3950
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred, test
> Affects Versions: 0.19.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Enis Soztutar
> Fix For: 0.19.0
>
> Attachments: Hadoop-3950_v1.patch
>
>
> - TestMapRed failed on Windows and Linux
> {noformat}
> Testcase: testMapred took 7.36 sec
> Caused an ERROR
> Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> java.io.IOException: Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:195)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:740)
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1036)
> at org.apache.hadoop.mapred.TestMapRed.launch(TestMapRed.java:614)
> at org.apache.hadoop.mapred.TestMapRed.testMapred(TestMapRed.java:248)
> {noformat}
> - TestMiniMRDFSSort failed on Windows
> {noformat}
> Testcase: testMapReduceSort took 21.312 sec
> Caused an ERROR
> File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> java.io.FileNotFoundException: File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:404)
> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:677)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1416)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1411)
> at org.apache.hadoop.mapred.SortValidator$RecordStatsChecker.checkRecords(SortValidator.java:370)
> at org.apache.hadoop.mapred.SortValidator.run(SortValidator.java:561)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.runSortValidator(TestMiniMRDFSSort.java:73)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.testMapReduceSort(TestMiniMRDFSSort.java:99)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-3950) TestMapRed and TestMiniMRDFSSort
failed on trunk
Posted by "Enis Soztutar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Enis Soztutar reassigned HADOOP-3950:
-------------------------------------
Assignee: Enis Soztutar
> TestMapRed and TestMiniMRDFSSort failed on trunk
> ------------------------------------------------
>
> Key: HADOOP-3950
> URL: https://issues.apache.org/jira/browse/HADOOP-3950
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred, test
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Enis Soztutar
>
> - TestMapRed failed on Windows and Linux
> {noformat}
> Testcase: testMapred took 7.36 sec
> Caused an ERROR
> Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> java.io.IOException: Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:195)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:740)
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1036)
> at org.apache.hadoop.mapred.TestMapRed.launch(TestMapRed.java:614)
> at org.apache.hadoop.mapred.TestMapRed.testMapred(TestMapRed.java:248)
> {noformat}
> - TestMiniMRDFSSort failed on Windows
> {noformat}
> Testcase: testMapReduceSort took 21.312 sec
> Caused an ERROR
> File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> java.io.FileNotFoundException: File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:404)
> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:677)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1416)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1411)
> at org.apache.hadoop.mapred.SortValidator$RecordStatsChecker.checkRecords(SortValidator.java:370)
> at org.apache.hadoop.mapred.SortValidator.run(SortValidator.java:561)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.runSortValidator(TestMiniMRDFSSort.java:73)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.testMapReduceSort(TestMiniMRDFSSort.java:99)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3950) TestMapRed and TestMiniMRDFSSort
failed on trunk
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12625977#action_12625977 ]
Hadoop QA commented on HADOOP-3950:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12388520/Hadoop-3950_v1.patch
against trunk revision 689336.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3115/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3115/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3115/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3115/console
This message is automatically generated.
> TestMapRed and TestMiniMRDFSSort failed on trunk
> ------------------------------------------------
>
> Key: HADOOP-3950
> URL: https://issues.apache.org/jira/browse/HADOOP-3950
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred, test
> Affects Versions: 0.19.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Enis Soztutar
> Fix For: 0.19.0
>
> Attachments: Hadoop-3950_v1.patch
>
>
> - TestMapRed failed on Windows and Linux
> {noformat}
> Testcase: testMapred took 7.36 sec
> Caused an ERROR
> Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> java.io.IOException: Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:195)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:740)
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1036)
> at org.apache.hadoop.mapred.TestMapRed.launch(TestMapRed.java:614)
> at org.apache.hadoop.mapred.TestMapRed.testMapred(TestMapRed.java:248)
> {noformat}
> - TestMiniMRDFSSort failed on Windows
> {noformat}
> Testcase: testMapReduceSort took 21.312 sec
> Caused an ERROR
> File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> java.io.FileNotFoundException: File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:404)
> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:677)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1416)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1411)
> at org.apache.hadoop.mapred.SortValidator$RecordStatsChecker.checkRecords(SortValidator.java:370)
> at org.apache.hadoop.mapred.SortValidator.run(SortValidator.java:561)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.runSortValidator(TestMiniMRDFSSort.java:73)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.testMapReduceSort(TestMiniMRDFSSort.java:99)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3950) TestMapRed and TestMiniMRDFSSort
failed on trunk
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12627487#action_12627487 ]
Hudson commented on HADOOP-3950:
--------------------------------
Integrated in Hadoop-trunk #589 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/589/])
> TestMapRed and TestMiniMRDFSSort failed on trunk
> ------------------------------------------------
>
> Key: HADOOP-3950
> URL: https://issues.apache.org/jira/browse/HADOOP-3950
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred, test
> Affects Versions: 0.19.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Enis Soztutar
> Fix For: 0.19.0
>
> Attachments: Hadoop-3950_v1.patch
>
>
> - TestMapRed failed on Windows and Linux
> {noformat}
> Testcase: testMapred took 7.36 sec
> Caused an ERROR
> Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> java.io.IOException: Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:195)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:740)
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1036)
> at org.apache.hadoop.mapred.TestMapRed.launch(TestMapRed.java:614)
> at org.apache.hadoop.mapred.TestMapRed.testMapred(TestMapRed.java:248)
> {noformat}
> - TestMiniMRDFSSort failed on Windows
> {noformat}
> Testcase: testMapReduceSort took 21.312 sec
> Caused an ERROR
> File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> java.io.FileNotFoundException: File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:404)
> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:677)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1416)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1411)
> at org.apache.hadoop.mapred.SortValidator$RecordStatsChecker.checkRecords(SortValidator.java:370)
> at org.apache.hadoop.mapred.SortValidator.run(SortValidator.java:561)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.runSortValidator(TestMiniMRDFSSort.java:73)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.testMapReduceSort(TestMiniMRDFSSort.java:99)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3950) TestMapRed and TestMiniMRDFSSort
failed on trunk
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622326#action_12622326 ]
Tom White commented on HADOOP-3950:
-----------------------------------
HADOOP-3946 has a fix for the TestMapRed failure.
> TestMapRed and TestMiniMRDFSSort failed on trunk
> ------------------------------------------------
>
> Key: HADOOP-3950
> URL: https://issues.apache.org/jira/browse/HADOOP-3950
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred, test
> Reporter: Tsz Wo (Nicholas), SZE
>
> - TestMapRed failed on Windows and Linux
> {noformat}
> Testcase: testMapred took 7.36 sec
> Caused an ERROR
> Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> java.io.IOException: Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:195)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:740)
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1036)
> at org.apache.hadoop.mapred.TestMapRed.launch(TestMapRed.java:614)
> at org.apache.hadoop.mapred.TestMapRed.testMapred(TestMapRed.java:248)
> {noformat}
> - TestMiniMRDFSSort failed on Windows
> {noformat}
> Testcase: testMapReduceSort took 21.312 sec
> Caused an ERROR
> File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> java.io.FileNotFoundException: File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:404)
> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:677)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1416)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1411)
> at org.apache.hadoop.mapred.SortValidator$RecordStatsChecker.checkRecords(SortValidator.java:370)
> at org.apache.hadoop.mapred.SortValidator.run(SortValidator.java:561)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.runSortValidator(TestMiniMRDFSSort.java:73)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.testMapReduceSort(TestMiniMRDFSSort.java:99)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3950) TestMapRed and TestMiniMRDFSSort
failed on trunk
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated HADOOP-3950:
----------------------------------
Status: Patch Available (was: Open)
This works for me. Trying Hudson
> TestMapRed and TestMiniMRDFSSort failed on trunk
> ------------------------------------------------
>
> Key: HADOOP-3950
> URL: https://issues.apache.org/jira/browse/HADOOP-3950
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred, test
> Affects Versions: 0.19.0
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Enis Soztutar
> Fix For: 0.19.0
>
> Attachments: Hadoop-3950_v1.patch
>
>
> - TestMapRed failed on Windows and Linux
> {noformat}
> Testcase: testMapred took 7.36 sec
> Caused an ERROR
> Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> java.io.IOException: Not a file: file:/d:/@sze/hadoop/trunk/mapred.loadtest/intermediateouts/part-00000/data
> at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:195)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:740)
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1036)
> at org.apache.hadoop.mapred.TestMapRed.launch(TestMapRed.java:614)
> at org.apache.hadoop.mapred.TestMapRed.testMapred(TestMapRed.java:248)
> {noformat}
> - TestMiniMRDFSSort failed on Windows
> {noformat}
> Testcase: testMapReduceSort took 21.312 sec
> Caused an ERROR
> File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> java.io.FileNotFoundException: File does not exist: /tmp/sortvalidate/recordstatschecker/part-00000
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:404)
> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:677)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1416)
> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1411)
> at org.apache.hadoop.mapred.SortValidator$RecordStatsChecker.checkRecords(SortValidator.java:370)
> at org.apache.hadoop.mapred.SortValidator.run(SortValidator.java:561)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.runSortValidator(TestMiniMRDFSSort.java:73)
> at org.apache.hadoop.mapred.TestMiniMRDFSSort.testMapReduceSort(TestMiniMRDFSSort.java:99)
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.