You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Mukund Madhugiri (JIRA)" <ji...@apache.org> on 2008/04/05 01:18:24 UTC
[jira] Created: (HADOOP-3183) Unit test fails on Windows:
TestJobShell.testJobShell
Unit test fails on Windows: TestJobShell.testJobShell
-----------------------------------------------------
Key: HADOOP-3183
URL: https://issues.apache.org/jira/browse/HADOOP-3183
Project: Hadoop Core
Issue Type: Bug
Components: mapred
Affects Versions: 0.17.0
Reporter: Mukund Madhugiri
Unit test fails on Windows: TestJobShell.testJobShell
junit.framework.AssertionFailedError: not failed
at org.apache.hadoop.mapred.TestJobShell.testJobShell(TestJobShell.java:74)
The MR job is failing:
task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,795 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000001_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,858 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
task_200804021954_0001_m_000001_0: 2008-04-02 19:54:36,904 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
task_200804021954_0001_m_000001_0: 2008-04-02 19:54:40,356 INFO mapred.TaskRunner (Task.java:done(387)) - Task 'task_200804021954_0001_m_000001_0' done.
2008-04-02 19:54:41,730 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
2008-04-02 19:54:45,104 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_0 done; removing files.
2008-04-02 19:54:45,244 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_0' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2085'
2008-04-02 19:54:45,385 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_0' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_0
2008-04-02 19:54:45,556 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_0, Status : FAILED
task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,060 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/0_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,185 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
task_200804021954_0001_m_000000_0: 2008-04-02 19:54:39,028 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
task_200804021954_0001_m_000000_0: 2008-04-02 19:54:40,153 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
2008-04-02 19:54:45,963 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1087)) - Choosing rack-local task tip_200804021954_0001_m_000000
2008-04-02 19:54:45,963 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_1' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
2008-04-02 19:54:45,978 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_1
2008-04-02 19:54:45,994 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
2008-04-02 19:54:46,087 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
2008-04-02 19:54:46,228 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
2008-04-02 19:54:47,618 INFO mapred.JobClient (JobClient.java:runJob(995)) - map 50% reduce 16%
2008-04-02 19:54:49,180 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
2008-04-02 19:54:49,242 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
2008-04-02 19:54:51,085 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
2008-04-02 19:54:52,241 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
2008-04-02 19:54:53,709 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_1 done; removing files.
2008-04-02 19:54:53,787 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_2' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_1' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
2008-04-02 19:54:53,803 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_2
2008-04-02 19:54:53,834 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
2008-04-02 19:54:53,897 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_1' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_1
2008-04-02 19:54:53,975 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
2008-04-02 19:54:54,662 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_1, Status : FAILED
task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,383 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_1/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,415 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,649 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
task_200804021954_0001_m_000000_1: 2008-04-02 19:54:49,180 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
2008-04-02 19:54:55,240 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
2008-04-02 19:54:56,520 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
2008-04-02 19:54:57,223 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
2008-04-02 19:55:01,065 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_2 done; removing files.
2008-04-02 19:55:01,159 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_3' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_2' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
2008-04-02 19:55:01,175 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_3
2008-04-02 19:55:01,206 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
2008-04-02 19:55:01,253 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
2008-04-02 19:55:01,284 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_2' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_2
2008-04-02 19:55:01,346 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
2008-04-02 19:55:01,690 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_2, Status : FAILED
task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,708 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_2/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,724 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,974 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
task_200804021954_0001_m_000000_2: 2008-04-02 19:54:56,520 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
2008-04-02 19:55:03,673 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
2008-04-02 19:55:04,001 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_3: java.io.IOException: file file_tmpfile not found
at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
2008-04-02 19:55:07,250 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
2008-04-02 19:55:08,218 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_3 done; removing files.
2008-04-02 19:55:08,281 INFO mapred.TaskInProgress (TaskInProgress.java:incompleteSubTask(519)) - TaskInProgress tip_200804021954_0001_m_000000 has failed 4 times.
2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:addTrackerTaskFailure(741)) - TaskTracker at 'tracker_seemeight-dx.ds.corp.yahoo.com' turned 'flaky'
2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:failedTask(1556)) - Aborting job job_200804021954_0001
2008-04-02 19:55:08,312 INFO dfs.StateChange (FSNamesystem.java:allocateBlock(1332)) - BLOCK* NameSystem.allocateBlock: /test/output/_logs/history/localhost_1207191264285_job_200804021954_0001_hadoopqa_external job. blk_1520406918918394204
2008-04-02 19:55:08,328 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2241 dest: /127.0.0.1:2064
2008-04-02 19:55:08,343 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2242 dest: /127.0.0.1:2066
2008-04-02 19:55:08,359 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2066 is added to blk_1520406918918394204 size 3263
2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1943)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1961)) - PacketResponder 0 for block blk_1520406918918394204 terminating
2008-04-02 19:55:08,375 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2064 is added to blk_1520406918918394204 size 3263
2008-04-02 19:55:08,375 INFO dfs.DataNode (DataNode.java:run(2037)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
2008-04-02 19:55:08,390 INFO dfs.DataNode (DataNode.java:run(2095)) - PacketResponder 1 for block blk_1520406918918394204 terminating
2008-04-02 19:55:08,437 INFO mapred.JobInProgress (JobInProgress.java:kill(1409)) - Killing job 'job_200804021954_0001'
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3183) Unit test fails on Windows:
TestJobShell.testJobShell
Posted by "Chris Douglas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Douglas updated HADOOP-3183:
----------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Mahadev
> Unit test fails on Windows: TestJobShell.testJobShell
> -----------------------------------------------------
>
> Key: HADOOP-3183
> URL: https://issues.apache.org/jira/browse/HADOOP-3183
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Mukund Madhugiri
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3183_1.patch
>
>
> Unit test fails on Windows: TestJobShell.testJobShell
> junit.framework.AssertionFailedError: not failed
> at org.apache.hadoop.mapred.TestJobShell.testJobShell(TestJobShell.java:74)
> The MR job is failing:
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,795 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000001_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,858 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:36,904 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:40,356 INFO mapred.TaskRunner (Task.java:done(387)) - Task 'task_200804021954_0001_m_000001_0' done.
> 2008-04-02 19:54:41,730 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,104 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_0 done; removing files.
> 2008-04-02 19:54:45,244 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_0' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2085'
> 2008-04-02 19:54:45,385 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_0' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_0
> 2008-04-02 19:54:45,556 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_0, Status : FAILED
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,060 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/0_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,185 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:39,028 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:40,153 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,963 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1087)) - Choosing rack-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:45,963 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_1' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:45,978 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:45,994 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:46,087 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:46,228 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:47,618 INFO mapred.JobClient (JobClient.java:runJob(995)) - map 50% reduce 16%
> 2008-04-02 19:54:49,180 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:49,242 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:51,085 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:52,241 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:53,709 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_1 done; removing files.
> 2008-04-02 19:54:53,787 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_2' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_1' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,803 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_2
> 2008-04-02 19:54:53,834 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:53,897 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_1' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:53,975 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:54,662 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_1, Status : FAILED
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,383 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_1/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,415 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,649 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:49,180 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:55,240 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:56,520 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:57,223 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:01,065 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_2 done; removing files.
> 2008-04-02 19:55:01,159 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_3' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_2' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,175 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_3
> 2008-04-02 19:55:01,206 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:55:01,253 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:01,284 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_2' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_2
> 2008-04-02 19:55:01,346 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:55:01,690 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_2, Status : FAILED
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,708 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_2/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,724 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,974 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:56,520 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:03,673 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:55:04,001 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_3: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:07,250 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:08,218 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_3 done; removing files.
> 2008-04-02 19:55:08,281 INFO mapred.TaskInProgress (TaskInProgress.java:incompleteSubTask(519)) - TaskInProgress tip_200804021954_0001_m_000000 has failed 4 times.
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:addTrackerTaskFailure(741)) - TaskTracker at 'tracker_seemeight-dx.ds.corp.yahoo.com' turned 'flaky'
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:failedTask(1556)) - Aborting job job_200804021954_0001
> 2008-04-02 19:55:08,312 INFO dfs.StateChange (FSNamesystem.java:allocateBlock(1332)) - BLOCK* NameSystem.allocateBlock: /test/output/_logs/history/localhost_1207191264285_job_200804021954_0001_hadoopqa_external job. blk_1520406918918394204
> 2008-04-02 19:55:08,328 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2241 dest: /127.0.0.1:2064
> 2008-04-02 19:55:08,343 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2242 dest: /127.0.0.1:2066
> 2008-04-02 19:55:08,359 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2066 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1943)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1961)) - PacketResponder 0 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,375 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2064 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,375 INFO dfs.DataNode (DataNode.java:run(2037)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,390 INFO dfs.DataNode (DataNode.java:run(2095)) - PacketResponder 1 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,437 INFO mapred.JobInProgress (JobInProgress.java:kill(1409)) - Killing job 'job_200804021954_0001'
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3183) Unit test fails on Windows:
TestJobShell.testJobShell
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated HADOOP-3183:
----------------------------------
Status: Open (was: Patch Available)
> Unit test fails on Windows: TestJobShell.testJobShell
> -----------------------------------------------------
>
> Key: HADOOP-3183
> URL: https://issues.apache.org/jira/browse/HADOOP-3183
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Mukund Madhugiri
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3183_1.patch
>
>
> Unit test fails on Windows: TestJobShell.testJobShell
> junit.framework.AssertionFailedError: not failed
> at org.apache.hadoop.mapred.TestJobShell.testJobShell(TestJobShell.java:74)
> The MR job is failing:
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,795 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000001_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,858 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:36,904 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:40,356 INFO mapred.TaskRunner (Task.java:done(387)) - Task 'task_200804021954_0001_m_000001_0' done.
> 2008-04-02 19:54:41,730 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,104 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_0 done; removing files.
> 2008-04-02 19:54:45,244 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_0' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2085'
> 2008-04-02 19:54:45,385 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_0' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_0
> 2008-04-02 19:54:45,556 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_0, Status : FAILED
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,060 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/0_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,185 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:39,028 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:40,153 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,963 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1087)) - Choosing rack-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:45,963 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_1' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:45,978 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:45,994 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:46,087 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:46,228 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:47,618 INFO mapred.JobClient (JobClient.java:runJob(995)) - map 50% reduce 16%
> 2008-04-02 19:54:49,180 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:49,242 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:51,085 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:52,241 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:53,709 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_1 done; removing files.
> 2008-04-02 19:54:53,787 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_2' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_1' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,803 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_2
> 2008-04-02 19:54:53,834 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:53,897 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_1' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:53,975 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:54,662 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_1, Status : FAILED
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,383 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_1/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,415 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,649 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:49,180 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:55,240 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:56,520 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:57,223 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:01,065 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_2 done; removing files.
> 2008-04-02 19:55:01,159 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_3' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_2' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,175 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_3
> 2008-04-02 19:55:01,206 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:55:01,253 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:01,284 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_2' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_2
> 2008-04-02 19:55:01,346 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:55:01,690 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_2, Status : FAILED
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,708 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_2/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,724 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,974 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:56,520 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:03,673 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:55:04,001 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_3: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:07,250 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:08,218 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_3 done; removing files.
> 2008-04-02 19:55:08,281 INFO mapred.TaskInProgress (TaskInProgress.java:incompleteSubTask(519)) - TaskInProgress tip_200804021954_0001_m_000000 has failed 4 times.
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:addTrackerTaskFailure(741)) - TaskTracker at 'tracker_seemeight-dx.ds.corp.yahoo.com' turned 'flaky'
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:failedTask(1556)) - Aborting job job_200804021954_0001
> 2008-04-02 19:55:08,312 INFO dfs.StateChange (FSNamesystem.java:allocateBlock(1332)) - BLOCK* NameSystem.allocateBlock: /test/output/_logs/history/localhost_1207191264285_job_200804021954_0001_hadoopqa_external job. blk_1520406918918394204
> 2008-04-02 19:55:08,328 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2241 dest: /127.0.0.1:2064
> 2008-04-02 19:55:08,343 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2242 dest: /127.0.0.1:2066
> 2008-04-02 19:55:08,359 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2066 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1943)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1961)) - PacketResponder 0 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,375 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2064 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,375 INFO dfs.DataNode (DataNode.java:run(2037)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,390 INFO dfs.DataNode (DataNode.java:run(2095)) - PacketResponder 1 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,437 INFO mapred.JobInProgress (JobInProgress.java:kill(1409)) - Killing job 'job_200804021954_0001'
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3183) Unit test fails on Windows:
TestJobShell.testJobShell
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12587252#action_12587252 ]
Hudson commented on HADOOP-3183:
--------------------------------
Integrated in Hadoop-trunk #455 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/455/])
> Unit test fails on Windows: TestJobShell.testJobShell
> -----------------------------------------------------
>
> Key: HADOOP-3183
> URL: https://issues.apache.org/jira/browse/HADOOP-3183
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Mukund Madhugiri
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3183_1.patch
>
>
> Unit test fails on Windows: TestJobShell.testJobShell
> junit.framework.AssertionFailedError: not failed
> at org.apache.hadoop.mapred.TestJobShell.testJobShell(TestJobShell.java:74)
> The MR job is failing:
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,795 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000001_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,858 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:36,904 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:40,356 INFO mapred.TaskRunner (Task.java:done(387)) - Task 'task_200804021954_0001_m_000001_0' done.
> 2008-04-02 19:54:41,730 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,104 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_0 done; removing files.
> 2008-04-02 19:54:45,244 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_0' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2085'
> 2008-04-02 19:54:45,385 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_0' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_0
> 2008-04-02 19:54:45,556 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_0, Status : FAILED
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,060 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/0_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,185 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:39,028 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:40,153 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,963 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1087)) - Choosing rack-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:45,963 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_1' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:45,978 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:45,994 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:46,087 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:46,228 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:47,618 INFO mapred.JobClient (JobClient.java:runJob(995)) - map 50% reduce 16%
> 2008-04-02 19:54:49,180 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:49,242 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:51,085 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:52,241 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:53,709 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_1 done; removing files.
> 2008-04-02 19:54:53,787 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_2' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_1' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,803 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_2
> 2008-04-02 19:54:53,834 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:53,897 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_1' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:53,975 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:54,662 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_1, Status : FAILED
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,383 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_1/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,415 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,649 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:49,180 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:55,240 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:56,520 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:57,223 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:01,065 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_2 done; removing files.
> 2008-04-02 19:55:01,159 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_3' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_2' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,175 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_3
> 2008-04-02 19:55:01,206 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:55:01,253 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:01,284 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_2' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_2
> 2008-04-02 19:55:01,346 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:55:01,690 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_2, Status : FAILED
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,708 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_2/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,724 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,974 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:56,520 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:03,673 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:55:04,001 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_3: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:07,250 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:08,218 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_3 done; removing files.
> 2008-04-02 19:55:08,281 INFO mapred.TaskInProgress (TaskInProgress.java:incompleteSubTask(519)) - TaskInProgress tip_200804021954_0001_m_000000 has failed 4 times.
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:addTrackerTaskFailure(741)) - TaskTracker at 'tracker_seemeight-dx.ds.corp.yahoo.com' turned 'flaky'
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:failedTask(1556)) - Aborting job job_200804021954_0001
> 2008-04-02 19:55:08,312 INFO dfs.StateChange (FSNamesystem.java:allocateBlock(1332)) - BLOCK* NameSystem.allocateBlock: /test/output/_logs/history/localhost_1207191264285_job_200804021954_0001_hadoopqa_external job. blk_1520406918918394204
> 2008-04-02 19:55:08,328 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2241 dest: /127.0.0.1:2064
> 2008-04-02 19:55:08,343 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2242 dest: /127.0.0.1:2066
> 2008-04-02 19:55:08,359 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2066 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1943)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1961)) - PacketResponder 0 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,375 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2064 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,375 INFO dfs.DataNode (DataNode.java:run(2037)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,390 INFO dfs.DataNode (DataNode.java:run(2095)) - PacketResponder 1 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,437 INFO mapred.JobInProgress (JobInProgress.java:kill(1409)) - Killing job 'job_200804021954_0001'
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3183) Unit test fails on Windows:
TestJobShell.testJobShell
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated HADOOP-3183:
----------------------------------
Attachment: HADOOP-3183_1.patch
this patch fixes the test to work on both windows and linux.
> Unit test fails on Windows: TestJobShell.testJobShell
> -----------------------------------------------------
>
> Key: HADOOP-3183
> URL: https://issues.apache.org/jira/browse/HADOOP-3183
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Mukund Madhugiri
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3183_1.patch
>
>
> Unit test fails on Windows: TestJobShell.testJobShell
> junit.framework.AssertionFailedError: not failed
> at org.apache.hadoop.mapred.TestJobShell.testJobShell(TestJobShell.java:74)
> The MR job is failing:
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,795 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000001_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,858 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:36,904 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:40,356 INFO mapred.TaskRunner (Task.java:done(387)) - Task 'task_200804021954_0001_m_000001_0' done.
> 2008-04-02 19:54:41,730 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,104 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_0 done; removing files.
> 2008-04-02 19:54:45,244 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_0' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2085'
> 2008-04-02 19:54:45,385 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_0' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_0
> 2008-04-02 19:54:45,556 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_0, Status : FAILED
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,060 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/0_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,185 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:39,028 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:40,153 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,963 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1087)) - Choosing rack-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:45,963 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_1' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:45,978 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:45,994 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:46,087 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:46,228 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:47,618 INFO mapred.JobClient (JobClient.java:runJob(995)) - map 50% reduce 16%
> 2008-04-02 19:54:49,180 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:49,242 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:51,085 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:52,241 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:53,709 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_1 done; removing files.
> 2008-04-02 19:54:53,787 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_2' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_1' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,803 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_2
> 2008-04-02 19:54:53,834 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:53,897 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_1' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:53,975 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:54,662 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_1, Status : FAILED
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,383 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_1/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,415 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,649 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:49,180 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:55,240 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:56,520 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:57,223 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:01,065 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_2 done; removing files.
> 2008-04-02 19:55:01,159 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_3' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_2' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,175 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_3
> 2008-04-02 19:55:01,206 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:55:01,253 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:01,284 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_2' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_2
> 2008-04-02 19:55:01,346 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:55:01,690 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_2, Status : FAILED
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,708 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_2/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,724 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,974 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:56,520 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:03,673 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:55:04,001 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_3: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:07,250 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:08,218 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_3 done; removing files.
> 2008-04-02 19:55:08,281 INFO mapred.TaskInProgress (TaskInProgress.java:incompleteSubTask(519)) - TaskInProgress tip_200804021954_0001_m_000000 has failed 4 times.
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:addTrackerTaskFailure(741)) - TaskTracker at 'tracker_seemeight-dx.ds.corp.yahoo.com' turned 'flaky'
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:failedTask(1556)) - Aborting job job_200804021954_0001
> 2008-04-02 19:55:08,312 INFO dfs.StateChange (FSNamesystem.java:allocateBlock(1332)) - BLOCK* NameSystem.allocateBlock: /test/output/_logs/history/localhost_1207191264285_job_200804021954_0001_hadoopqa_external job. blk_1520406918918394204
> 2008-04-02 19:55:08,328 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2241 dest: /127.0.0.1:2064
> 2008-04-02 19:55:08,343 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2242 dest: /127.0.0.1:2066
> 2008-04-02 19:55:08,359 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2066 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1943)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1961)) - PacketResponder 0 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,375 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2064 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,375 INFO dfs.DataNode (DataNode.java:run(2037)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,390 INFO dfs.DataNode (DataNode.java:run(2095)) - PacketResponder 1 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,437 INFO mapred.JobInProgress (JobInProgress.java:kill(1409)) - Killing job 'job_200804021954_0001'
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3183) Unit test fails on Windows:
TestJobShell.testJobShell
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12586477#action_12586477 ]
Mahadev konar commented on HADOOP-3183:
---------------------------------------
i think the problem occurs because java is not able to read the symlinks created on cygwin which means the command line features of -files will not work on cygwin. I will update the test and check in documentation which will reflect this.
> Unit test fails on Windows: TestJobShell.testJobShell
> -----------------------------------------------------
>
> Key: HADOOP-3183
> URL: https://issues.apache.org/jira/browse/HADOOP-3183
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Mukund Madhugiri
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.17.0
>
>
> Unit test fails on Windows: TestJobShell.testJobShell
> junit.framework.AssertionFailedError: not failed
> at org.apache.hadoop.mapred.TestJobShell.testJobShell(TestJobShell.java:74)
> The MR job is failing:
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,795 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000001_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,858 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:36,904 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:40,356 INFO mapred.TaskRunner (Task.java:done(387)) - Task 'task_200804021954_0001_m_000001_0' done.
> 2008-04-02 19:54:41,730 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,104 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_0 done; removing files.
> 2008-04-02 19:54:45,244 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_0' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2085'
> 2008-04-02 19:54:45,385 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_0' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_0
> 2008-04-02 19:54:45,556 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_0, Status : FAILED
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,060 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/0_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,185 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:39,028 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:40,153 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,963 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1087)) - Choosing rack-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:45,963 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_1' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:45,978 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:45,994 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:46,087 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:46,228 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:47,618 INFO mapred.JobClient (JobClient.java:runJob(995)) - map 50% reduce 16%
> 2008-04-02 19:54:49,180 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:49,242 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:51,085 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:52,241 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:53,709 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_1 done; removing files.
> 2008-04-02 19:54:53,787 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_2' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_1' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,803 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_2
> 2008-04-02 19:54:53,834 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:53,897 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_1' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:53,975 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:54,662 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_1, Status : FAILED
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,383 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_1/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,415 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,649 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:49,180 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:55,240 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:56,520 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:57,223 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:01,065 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_2 done; removing files.
> 2008-04-02 19:55:01,159 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_3' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_2' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,175 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_3
> 2008-04-02 19:55:01,206 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:55:01,253 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:01,284 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_2' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_2
> 2008-04-02 19:55:01,346 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:55:01,690 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_2, Status : FAILED
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,708 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_2/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,724 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,974 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:56,520 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:03,673 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:55:04,001 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_3: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:07,250 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:08,218 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_3 done; removing files.
> 2008-04-02 19:55:08,281 INFO mapred.TaskInProgress (TaskInProgress.java:incompleteSubTask(519)) - TaskInProgress tip_200804021954_0001_m_000000 has failed 4 times.
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:addTrackerTaskFailure(741)) - TaskTracker at 'tracker_seemeight-dx.ds.corp.yahoo.com' turned 'flaky'
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:failedTask(1556)) - Aborting job job_200804021954_0001
> 2008-04-02 19:55:08,312 INFO dfs.StateChange (FSNamesystem.java:allocateBlock(1332)) - BLOCK* NameSystem.allocateBlock: /test/output/_logs/history/localhost_1207191264285_job_200804021954_0001_hadoopqa_external job. blk_1520406918918394204
> 2008-04-02 19:55:08,328 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2241 dest: /127.0.0.1:2064
> 2008-04-02 19:55:08,343 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2242 dest: /127.0.0.1:2066
> 2008-04-02 19:55:08,359 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2066 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1943)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1961)) - PacketResponder 0 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,375 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2064 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,375 INFO dfs.DataNode (DataNode.java:run(2037)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,390 INFO dfs.DataNode (DataNode.java:run(2095)) - PacketResponder 1 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,437 INFO mapred.JobInProgress (JobInProgress.java:kill(1409)) - Killing job 'job_200804021954_0001'
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3183) Unit test fails on Windows:
TestJobShell.testJobShell
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12586671#action_12586671 ]
Hadoop QA commented on HADOOP-3183:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12379617/HADOOP-3183_1.patch
against trunk revision 643282.
@author +1. The patch does not contain any @author tags.
tests included +1. The patch appears to include 3 new or modified tests.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new javac compiler warnings.
release audit +1. The applied patch does not generate any new release audit warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests -1. The patch failed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2180/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2180/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2180/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2180/console
This message is automatically generated.
> Unit test fails on Windows: TestJobShell.testJobShell
> -----------------------------------------------------
>
> Key: HADOOP-3183
> URL: https://issues.apache.org/jira/browse/HADOOP-3183
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Mukund Madhugiri
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3183_1.patch
>
>
> Unit test fails on Windows: TestJobShell.testJobShell
> junit.framework.AssertionFailedError: not failed
> at org.apache.hadoop.mapred.TestJobShell.testJobShell(TestJobShell.java:74)
> The MR job is failing:
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,795 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000001_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,858 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:36,904 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:40,356 INFO mapred.TaskRunner (Task.java:done(387)) - Task 'task_200804021954_0001_m_000001_0' done.
> 2008-04-02 19:54:41,730 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,104 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_0 done; removing files.
> 2008-04-02 19:54:45,244 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_0' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2085'
> 2008-04-02 19:54:45,385 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_0' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_0
> 2008-04-02 19:54:45,556 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_0, Status : FAILED
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,060 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/0_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,185 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:39,028 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:40,153 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,963 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1087)) - Choosing rack-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:45,963 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_1' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:45,978 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:45,994 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:46,087 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:46,228 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:47,618 INFO mapred.JobClient (JobClient.java:runJob(995)) - map 50% reduce 16%
> 2008-04-02 19:54:49,180 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:49,242 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:51,085 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:52,241 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:53,709 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_1 done; removing files.
> 2008-04-02 19:54:53,787 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_2' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_1' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,803 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_2
> 2008-04-02 19:54:53,834 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:53,897 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_1' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:53,975 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:54,662 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_1, Status : FAILED
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,383 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_1/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,415 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,649 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:49,180 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:55,240 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:56,520 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:57,223 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:01,065 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_2 done; removing files.
> 2008-04-02 19:55:01,159 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_3' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_2' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,175 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_3
> 2008-04-02 19:55:01,206 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:55:01,253 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:01,284 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_2' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_2
> 2008-04-02 19:55:01,346 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:55:01,690 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_2, Status : FAILED
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,708 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_2/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,724 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,974 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:56,520 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:03,673 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:55:04,001 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_3: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:07,250 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:08,218 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_3 done; removing files.
> 2008-04-02 19:55:08,281 INFO mapred.TaskInProgress (TaskInProgress.java:incompleteSubTask(519)) - TaskInProgress tip_200804021954_0001_m_000000 has failed 4 times.
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:addTrackerTaskFailure(741)) - TaskTracker at 'tracker_seemeight-dx.ds.corp.yahoo.com' turned 'flaky'
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:failedTask(1556)) - Aborting job job_200804021954_0001
> 2008-04-02 19:55:08,312 INFO dfs.StateChange (FSNamesystem.java:allocateBlock(1332)) - BLOCK* NameSystem.allocateBlock: /test/output/_logs/history/localhost_1207191264285_job_200804021954_0001_hadoopqa_external job. blk_1520406918918394204
> 2008-04-02 19:55:08,328 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2241 dest: /127.0.0.1:2064
> 2008-04-02 19:55:08,343 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2242 dest: /127.0.0.1:2066
> 2008-04-02 19:55:08,359 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2066 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1943)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1961)) - PacketResponder 0 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,375 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2064 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,375 INFO dfs.DataNode (DataNode.java:run(2037)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,390 INFO dfs.DataNode (DataNode.java:run(2095)) - PacketResponder 1 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,437 INFO mapred.JobInProgress (JobInProgress.java:kill(1409)) - Killing job 'job_200804021954_0001'
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3183) Unit test fails on Windows:
TestJobShell.testJobShell
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12586746#action_12586746 ]
Hadoop QA commented on HADOOP-3183:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12379617/HADOOP-3183_1.patch
against trunk revision 645773.
@author +1. The patch does not contain any @author tags.
tests included +1. The patch appears to include 3 new or modified tests.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new javac compiler warnings.
release audit +1. The applied patch does not generate any new release audit warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2183/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2183/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2183/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2183/console
This message is automatically generated.
> Unit test fails on Windows: TestJobShell.testJobShell
> -----------------------------------------------------
>
> Key: HADOOP-3183
> URL: https://issues.apache.org/jira/browse/HADOOP-3183
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Mukund Madhugiri
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3183_1.patch
>
>
> Unit test fails on Windows: TestJobShell.testJobShell
> junit.framework.AssertionFailedError: not failed
> at org.apache.hadoop.mapred.TestJobShell.testJobShell(TestJobShell.java:74)
> The MR job is failing:
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,795 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000001_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,858 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:36,904 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:40,356 INFO mapred.TaskRunner (Task.java:done(387)) - Task 'task_200804021954_0001_m_000001_0' done.
> 2008-04-02 19:54:41,730 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,104 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_0 done; removing files.
> 2008-04-02 19:54:45,244 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_0' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2085'
> 2008-04-02 19:54:45,385 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_0' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_0
> 2008-04-02 19:54:45,556 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_0, Status : FAILED
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,060 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/0_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,185 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:39,028 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:40,153 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,963 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1087)) - Choosing rack-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:45,963 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_1' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:45,978 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:45,994 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:46,087 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:46,228 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:47,618 INFO mapred.JobClient (JobClient.java:runJob(995)) - map 50% reduce 16%
> 2008-04-02 19:54:49,180 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:49,242 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:51,085 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:52,241 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:53,709 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_1 done; removing files.
> 2008-04-02 19:54:53,787 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_2' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_1' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,803 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_2
> 2008-04-02 19:54:53,834 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:53,897 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_1' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:53,975 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:54,662 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_1, Status : FAILED
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,383 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_1/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,415 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,649 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:49,180 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:55,240 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:56,520 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:57,223 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:01,065 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_2 done; removing files.
> 2008-04-02 19:55:01,159 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_3' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_2' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,175 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_3
> 2008-04-02 19:55:01,206 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:55:01,253 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:01,284 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_2' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_2
> 2008-04-02 19:55:01,346 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:55:01,690 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_2, Status : FAILED
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,708 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_2/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,724 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,974 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:56,520 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:03,673 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:55:04,001 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_3: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:07,250 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:08,218 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_3 done; removing files.
> 2008-04-02 19:55:08,281 INFO mapred.TaskInProgress (TaskInProgress.java:incompleteSubTask(519)) - TaskInProgress tip_200804021954_0001_m_000000 has failed 4 times.
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:addTrackerTaskFailure(741)) - TaskTracker at 'tracker_seemeight-dx.ds.corp.yahoo.com' turned 'flaky'
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:failedTask(1556)) - Aborting job job_200804021954_0001
> 2008-04-02 19:55:08,312 INFO dfs.StateChange (FSNamesystem.java:allocateBlock(1332)) - BLOCK* NameSystem.allocateBlock: /test/output/_logs/history/localhost_1207191264285_job_200804021954_0001_hadoopqa_external job. blk_1520406918918394204
> 2008-04-02 19:55:08,328 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2241 dest: /127.0.0.1:2064
> 2008-04-02 19:55:08,343 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2242 dest: /127.0.0.1:2066
> 2008-04-02 19:55:08,359 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2066 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1943)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1961)) - PacketResponder 0 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,375 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2064 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,375 INFO dfs.DataNode (DataNode.java:run(2037)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,390 INFO dfs.DataNode (DataNode.java:run(2095)) - PacketResponder 1 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,437 INFO mapred.JobInProgress (JobInProgress.java:kill(1409)) - Killing job 'job_200804021954_0001'
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3183) Unit test fails on Windows:
TestJobShell.testJobShell
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated HADOOP-3183:
----------------------------------
Status: Patch Available (was: Open)
tests failed with testbalancer. patch does not change anything in dfs infact just chages a testcase. so trying hudson again.
> Unit test fails on Windows: TestJobShell.testJobShell
> -----------------------------------------------------
>
> Key: HADOOP-3183
> URL: https://issues.apache.org/jira/browse/HADOOP-3183
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Mukund Madhugiri
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3183_1.patch
>
>
> Unit test fails on Windows: TestJobShell.testJobShell
> junit.framework.AssertionFailedError: not failed
> at org.apache.hadoop.mapred.TestJobShell.testJobShell(TestJobShell.java:74)
> The MR job is failing:
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,795 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000001_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,858 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:36,904 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:40,356 INFO mapred.TaskRunner (Task.java:done(387)) - Task 'task_200804021954_0001_m_000001_0' done.
> 2008-04-02 19:54:41,730 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,104 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_0 done; removing files.
> 2008-04-02 19:54:45,244 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_0' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2085'
> 2008-04-02 19:54:45,385 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_0' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_0
> 2008-04-02 19:54:45,556 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_0, Status : FAILED
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,060 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/0_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,185 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:39,028 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:40,153 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,963 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1087)) - Choosing rack-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:45,963 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_1' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:45,978 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:45,994 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:46,087 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:46,228 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:47,618 INFO mapred.JobClient (JobClient.java:runJob(995)) - map 50% reduce 16%
> 2008-04-02 19:54:49,180 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:49,242 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:51,085 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:52,241 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:53,709 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_1 done; removing files.
> 2008-04-02 19:54:53,787 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_2' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_1' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,803 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_2
> 2008-04-02 19:54:53,834 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:53,897 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_1' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:53,975 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:54,662 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_1, Status : FAILED
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,383 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_1/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,415 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,649 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:49,180 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:55,240 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:56,520 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:57,223 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:01,065 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_2 done; removing files.
> 2008-04-02 19:55:01,159 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_3' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_2' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,175 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_3
> 2008-04-02 19:55:01,206 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:55:01,253 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:01,284 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_2' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_2
> 2008-04-02 19:55:01,346 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:55:01,690 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_2, Status : FAILED
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,708 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_2/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,724 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,974 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:56,520 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:03,673 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:55:04,001 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_3: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:07,250 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:08,218 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_3 done; removing files.
> 2008-04-02 19:55:08,281 INFO mapred.TaskInProgress (TaskInProgress.java:incompleteSubTask(519)) - TaskInProgress tip_200804021954_0001_m_000000 has failed 4 times.
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:addTrackerTaskFailure(741)) - TaskTracker at 'tracker_seemeight-dx.ds.corp.yahoo.com' turned 'flaky'
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:failedTask(1556)) - Aborting job job_200804021954_0001
> 2008-04-02 19:55:08,312 INFO dfs.StateChange (FSNamesystem.java:allocateBlock(1332)) - BLOCK* NameSystem.allocateBlock: /test/output/_logs/history/localhost_1207191264285_job_200804021954_0001_hadoopqa_external job. blk_1520406918918394204
> 2008-04-02 19:55:08,328 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2241 dest: /127.0.0.1:2064
> 2008-04-02 19:55:08,343 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2242 dest: /127.0.0.1:2066
> 2008-04-02 19:55:08,359 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2066 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1943)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1961)) - PacketResponder 0 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,375 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2064 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,375 INFO dfs.DataNode (DataNode.java:run(2037)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,390 INFO dfs.DataNode (DataNode.java:run(2095)) - PacketResponder 1 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,437 INFO mapred.JobInProgress (JobInProgress.java:kill(1409)) - Killing job 'job_200804021954_0001'
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3183) Unit test fails on Windows:
TestJobShell.testJobShell
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nigel Daley updated HADOOP-3183:
--------------------------------
Priority: Blocker (was: Major)
Fix Version/s: 0.17.0
> Unit test fails on Windows: TestJobShell.testJobShell
> -----------------------------------------------------
>
> Key: HADOOP-3183
> URL: https://issues.apache.org/jira/browse/HADOOP-3183
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Mukund Madhugiri
> Priority: Blocker
> Fix For: 0.17.0
>
>
> Unit test fails on Windows: TestJobShell.testJobShell
> junit.framework.AssertionFailedError: not failed
> at org.apache.hadoop.mapred.TestJobShell.testJobShell(TestJobShell.java:74)
> The MR job is failing:
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,795 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000001_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,858 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:36,904 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:40,356 INFO mapred.TaskRunner (Task.java:done(387)) - Task 'task_200804021954_0001_m_000001_0' done.
> 2008-04-02 19:54:41,730 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,104 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_0 done; removing files.
> 2008-04-02 19:54:45,244 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_0' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2085'
> 2008-04-02 19:54:45,385 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_0' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_0
> 2008-04-02 19:54:45,556 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_0, Status : FAILED
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,060 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/0_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,185 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:39,028 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:40,153 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,963 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1087)) - Choosing rack-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:45,963 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_1' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:45,978 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:45,994 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:46,087 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:46,228 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:47,618 INFO mapred.JobClient (JobClient.java:runJob(995)) - map 50% reduce 16%
> 2008-04-02 19:54:49,180 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:49,242 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:51,085 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:52,241 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:53,709 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_1 done; removing files.
> 2008-04-02 19:54:53,787 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_2' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_1' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,803 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_2
> 2008-04-02 19:54:53,834 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:53,897 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_1' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:53,975 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:54,662 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_1, Status : FAILED
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,383 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_1/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,415 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,649 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:49,180 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:55,240 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:56,520 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:57,223 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:01,065 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_2 done; removing files.
> 2008-04-02 19:55:01,159 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_3' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_2' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,175 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_3
> 2008-04-02 19:55:01,206 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:55:01,253 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:01,284 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_2' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_2
> 2008-04-02 19:55:01,346 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:55:01,690 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_2, Status : FAILED
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,708 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_2/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,724 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,974 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:56,520 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:03,673 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:55:04,001 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_3: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:07,250 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:08,218 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_3 done; removing files.
> 2008-04-02 19:55:08,281 INFO mapred.TaskInProgress (TaskInProgress.java:incompleteSubTask(519)) - TaskInProgress tip_200804021954_0001_m_000000 has failed 4 times.
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:addTrackerTaskFailure(741)) - TaskTracker at 'tracker_seemeight-dx.ds.corp.yahoo.com' turned 'flaky'
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:failedTask(1556)) - Aborting job job_200804021954_0001
> 2008-04-02 19:55:08,312 INFO dfs.StateChange (FSNamesystem.java:allocateBlock(1332)) - BLOCK* NameSystem.allocateBlock: /test/output/_logs/history/localhost_1207191264285_job_200804021954_0001_hadoopqa_external job. blk_1520406918918394204
> 2008-04-02 19:55:08,328 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2241 dest: /127.0.0.1:2064
> 2008-04-02 19:55:08,343 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2242 dest: /127.0.0.1:2066
> 2008-04-02 19:55:08,359 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2066 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1943)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1961)) - PacketResponder 0 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,375 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2064 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,375 INFO dfs.DataNode (DataNode.java:run(2037)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,390 INFO dfs.DataNode (DataNode.java:run(2095)) - PacketResponder 1 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,437 INFO mapred.JobInProgress (JobInProgress.java:kill(1409)) - Killing job 'job_200804021954_0001'
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-3183) Unit test fails on Windows:
TestJobShell.testJobShell
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar reassigned HADOOP-3183:
-------------------------------------
Assignee: Mahadev konar
> Unit test fails on Windows: TestJobShell.testJobShell
> -----------------------------------------------------
>
> Key: HADOOP-3183
> URL: https://issues.apache.org/jira/browse/HADOOP-3183
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Mukund Madhugiri
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.17.0
>
>
> Unit test fails on Windows: TestJobShell.testJobShell
> junit.framework.AssertionFailedError: not failed
> at org.apache.hadoop.mapred.TestJobShell.testJobShell(TestJobShell.java:74)
> The MR job is failing:
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,795 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000001_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,858 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:36,904 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:40,356 INFO mapred.TaskRunner (Task.java:done(387)) - Task 'task_200804021954_0001_m_000001_0' done.
> 2008-04-02 19:54:41,730 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,104 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_0 done; removing files.
> 2008-04-02 19:54:45,244 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_0' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2085'
> 2008-04-02 19:54:45,385 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_0' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_0
> 2008-04-02 19:54:45,556 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_0, Status : FAILED
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,060 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/0_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,185 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:39,028 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:40,153 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,963 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1087)) - Choosing rack-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:45,963 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_1' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:45,978 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:45,994 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:46,087 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:46,228 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:47,618 INFO mapred.JobClient (JobClient.java:runJob(995)) - map 50% reduce 16%
> 2008-04-02 19:54:49,180 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:49,242 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:51,085 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:52,241 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:53,709 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_1 done; removing files.
> 2008-04-02 19:54:53,787 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_2' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_1' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,803 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_2
> 2008-04-02 19:54:53,834 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:53,897 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_1' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:53,975 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:54,662 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_1, Status : FAILED
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,383 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_1/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,415 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,649 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:49,180 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:55,240 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:56,520 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:57,223 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:01,065 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_2 done; removing files.
> 2008-04-02 19:55:01,159 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_3' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_2' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,175 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_3
> 2008-04-02 19:55:01,206 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:55:01,253 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:01,284 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_2' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_2
> 2008-04-02 19:55:01,346 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:55:01,690 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_2, Status : FAILED
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,708 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_2/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,724 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,974 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:56,520 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:03,673 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:55:04,001 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_3: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:07,250 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:08,218 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_3 done; removing files.
> 2008-04-02 19:55:08,281 INFO mapred.TaskInProgress (TaskInProgress.java:incompleteSubTask(519)) - TaskInProgress tip_200804021954_0001_m_000000 has failed 4 times.
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:addTrackerTaskFailure(741)) - TaskTracker at 'tracker_seemeight-dx.ds.corp.yahoo.com' turned 'flaky'
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:failedTask(1556)) - Aborting job job_200804021954_0001
> 2008-04-02 19:55:08,312 INFO dfs.StateChange (FSNamesystem.java:allocateBlock(1332)) - BLOCK* NameSystem.allocateBlock: /test/output/_logs/history/localhost_1207191264285_job_200804021954_0001_hadoopqa_external job. blk_1520406918918394204
> 2008-04-02 19:55:08,328 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2241 dest: /127.0.0.1:2064
> 2008-04-02 19:55:08,343 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2242 dest: /127.0.0.1:2066
> 2008-04-02 19:55:08,359 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2066 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1943)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1961)) - PacketResponder 0 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,375 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2064 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,375 INFO dfs.DataNode (DataNode.java:run(2037)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,390 INFO dfs.DataNode (DataNode.java:run(2095)) - PacketResponder 1 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,437 INFO mapred.JobInProgress (JobInProgress.java:kill(1409)) - Killing job 'job_200804021954_0001'
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3183) Unit test fails on Windows:
TestJobShell.testJobShell
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated HADOOP-3183:
----------------------------------
Status: Patch Available (was: Open)
> Unit test fails on Windows: TestJobShell.testJobShell
> -----------------------------------------------------
>
> Key: HADOOP-3183
> URL: https://issues.apache.org/jira/browse/HADOOP-3183
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Mukund Madhugiri
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3183_1.patch
>
>
> Unit test fails on Windows: TestJobShell.testJobShell
> junit.framework.AssertionFailedError: not failed
> at org.apache.hadoop.mapred.TestJobShell.testJobShell(TestJobShell.java:74)
> The MR job is failing:
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,795 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000001_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,858 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:36,904 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:40,356 INFO mapred.TaskRunner (Task.java:done(387)) - Task 'task_200804021954_0001_m_000001_0' done.
> 2008-04-02 19:54:41,730 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,104 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_0 done; removing files.
> 2008-04-02 19:54:45,244 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_0' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2085'
> 2008-04-02 19:54:45,385 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_0' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_0
> 2008-04-02 19:54:45,556 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_0, Status : FAILED
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,060 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/0_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,185 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:39,028 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:40,153 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,963 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1087)) - Choosing rack-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:45,963 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_1' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:45,978 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:45,994 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:46,087 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:46,228 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:47,618 INFO mapred.JobClient (JobClient.java:runJob(995)) - map 50% reduce 16%
> 2008-04-02 19:54:49,180 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:49,242 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:51,085 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:52,241 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:53,709 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_1 done; removing files.
> 2008-04-02 19:54:53,787 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_2' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_1' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,803 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_2
> 2008-04-02 19:54:53,834 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:53,897 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_1' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:53,975 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:54,662 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_1, Status : FAILED
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,383 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_1/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,415 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,649 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:49,180 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:55,240 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:56,520 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:57,223 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:01,065 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_2 done; removing files.
> 2008-04-02 19:55:01,159 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_3' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_2' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,175 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_3
> 2008-04-02 19:55:01,206 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:55:01,253 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:01,284 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_2' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_2
> 2008-04-02 19:55:01,346 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:55:01,690 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_2, Status : FAILED
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,708 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_2/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,724 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,974 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:56,520 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:03,673 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:55:04,001 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_3: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:07,250 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:08,218 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_3 done; removing files.
> 2008-04-02 19:55:08,281 INFO mapred.TaskInProgress (TaskInProgress.java:incompleteSubTask(519)) - TaskInProgress tip_200804021954_0001_m_000000 has failed 4 times.
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:addTrackerTaskFailure(741)) - TaskTracker at 'tracker_seemeight-dx.ds.corp.yahoo.com' turned 'flaky'
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:failedTask(1556)) - Aborting job job_200804021954_0001
> 2008-04-02 19:55:08,312 INFO dfs.StateChange (FSNamesystem.java:allocateBlock(1332)) - BLOCK* NameSystem.allocateBlock: /test/output/_logs/history/localhost_1207191264285_job_200804021954_0001_hadoopqa_external job. blk_1520406918918394204
> 2008-04-02 19:55:08,328 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2241 dest: /127.0.0.1:2064
> 2008-04-02 19:55:08,343 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2242 dest: /127.0.0.1:2066
> 2008-04-02 19:55:08,359 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2066 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1943)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1961)) - PacketResponder 0 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,375 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2064 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,375 INFO dfs.DataNode (DataNode.java:run(2037)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,390 INFO dfs.DataNode (DataNode.java:run(2095)) - PacketResponder 1 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,437 INFO mapred.JobInProgress (JobInProgress.java:kill(1409)) - Killing job 'job_200804021954_0001'
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3183) Unit test fails on Windows:
TestJobShell.testJobShell
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated HADOOP-3183:
----------------------------------
Status: Open (was: Patch Available)
trying hudson again.
> Unit test fails on Windows: TestJobShell.testJobShell
> -----------------------------------------------------
>
> Key: HADOOP-3183
> URL: https://issues.apache.org/jira/browse/HADOOP-3183
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Mukund Madhugiri
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3183_1.patch
>
>
> Unit test fails on Windows: TestJobShell.testJobShell
> junit.framework.AssertionFailedError: not failed
> at org.apache.hadoop.mapred.TestJobShell.testJobShell(TestJobShell.java:74)
> The MR job is failing:
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,795 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000001_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,858 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:36,904 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:40,356 INFO mapred.TaskRunner (Task.java:done(387)) - Task 'task_200804021954_0001_m_000001_0' done.
> 2008-04-02 19:54:41,730 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,104 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_0 done; removing files.
> 2008-04-02 19:54:45,244 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_0' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2085'
> 2008-04-02 19:54:45,385 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_0' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_0
> 2008-04-02 19:54:45,556 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_0, Status : FAILED
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,060 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/0_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,185 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:39,028 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:40,153 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,963 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1087)) - Choosing rack-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:45,963 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_1' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:45,978 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:45,994 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:46,087 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:46,228 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:47,618 INFO mapred.JobClient (JobClient.java:runJob(995)) - map 50% reduce 16%
> 2008-04-02 19:54:49,180 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:49,242 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:51,085 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:52,241 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:53,709 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_1 done; removing files.
> 2008-04-02 19:54:53,787 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_2' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_1' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,803 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_2
> 2008-04-02 19:54:53,834 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:53,897 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_1' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:53,975 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:54,662 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_1, Status : FAILED
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,383 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_1/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,415 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,649 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:49,180 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:55,240 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:56,520 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:57,223 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:01,065 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_2 done; removing files.
> 2008-04-02 19:55:01,159 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_3' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_2' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,175 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_3
> 2008-04-02 19:55:01,206 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:55:01,253 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:01,284 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_2' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_2
> 2008-04-02 19:55:01,346 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:55:01,690 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_2, Status : FAILED
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,708 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_2/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,724 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,974 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:56,520 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:03,673 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:55:04,001 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_3: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:07,250 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:08,218 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_3 done; removing files.
> 2008-04-02 19:55:08,281 INFO mapred.TaskInProgress (TaskInProgress.java:incompleteSubTask(519)) - TaskInProgress tip_200804021954_0001_m_000000 has failed 4 times.
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:addTrackerTaskFailure(741)) - TaskTracker at 'tracker_seemeight-dx.ds.corp.yahoo.com' turned 'flaky'
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:failedTask(1556)) - Aborting job job_200804021954_0001
> 2008-04-02 19:55:08,312 INFO dfs.StateChange (FSNamesystem.java:allocateBlock(1332)) - BLOCK* NameSystem.allocateBlock: /test/output/_logs/history/localhost_1207191264285_job_200804021954_0001_hadoopqa_external job. blk_1520406918918394204
> 2008-04-02 19:55:08,328 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2241 dest: /127.0.0.1:2064
> 2008-04-02 19:55:08,343 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2242 dest: /127.0.0.1:2066
> 2008-04-02 19:55:08,359 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2066 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1943)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1961)) - PacketResponder 0 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,375 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2064 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,375 INFO dfs.DataNode (DataNode.java:run(2037)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,390 INFO dfs.DataNode (DataNode.java:run(2095)) - PacketResponder 1 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,437 INFO mapred.JobInProgress (JobInProgress.java:kill(1409)) - Killing job 'job_200804021954_0001'
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-3183) Unit test fails on Windows:
TestJobShell.testJobShell
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated HADOOP-3183:
----------------------------------
Status: Patch Available (was: Open)
> Unit test fails on Windows: TestJobShell.testJobShell
> -----------------------------------------------------
>
> Key: HADOOP-3183
> URL: https://issues.apache.org/jira/browse/HADOOP-3183
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.17.0
> Reporter: Mukund Madhugiri
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 0.17.0
>
> Attachments: HADOOP-3183_1.patch
>
>
> Unit test fails on Windows: TestJobShell.testJobShell
> junit.framework.AssertionFailedError: not failed
> at org.apache.hadoop.mapred.TestJobShell.testJobShell(TestJobShell.java:74)
> The MR job is failing:
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,795 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000001_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,858 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:36,904 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:40,356 INFO mapred.TaskRunner (Task.java:done(387)) - Task 'task_200804021954_0001_m_000001_0' done.
> 2008-04-02 19:54:41,730 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,104 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_0 done; removing files.
> 2008-04-02 19:54:45,244 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_0' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2085'
> 2008-04-02 19:54:45,385 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_0' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_0
> 2008-04-02 19:54:45,556 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_0, Status : FAILED
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,060 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/0_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_0/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,185 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:39,028 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:40,153 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_0: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_0: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,963 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1087)) - Choosing rack-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:45,963 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_1' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:45,978 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:45,994 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:46,087 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:46,228 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:47,618 INFO mapred.JobClient (JobClient.java:runJob(995)) - map 50% reduce 16%
> 2008-04-02 19:54:49,180 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:49,242 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:51,085 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:52,241 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:53,709 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_1 done; removing files.
> 2008-04-02 19:54:53,787 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_2' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,787 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_1' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,803 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_2
> 2008-04-02 19:54:53,834 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:54:53,897 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_1' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:53,975 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:54,662 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_1, Status : FAILED
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,383 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_1/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,415 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,649 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:49,180 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_1: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_1: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:55,240 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:54:56,520 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:57,223 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:01,065 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_2 done; removing files.
> 2008-04-02 19:55:01,159 INFO mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084)) - Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:createTaskEntry(879)) - Adding task 'task_200804021954_0001_m_000000_3' to tip tip_200804021954_0001_m_000000, for tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,159 INFO mapred.JobTracker (JobTracker.java:removeMarkedTasks(974)) - Removed completed task 'task_200804021954_0001_m_000000_2' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,175 INFO mapred.TaskTracker (TaskTracker.java:startNewTask(1261)) - LaunchTaskAction: task_200804021954_0001_m_000000_3
> 2008-04-02 19:55:01,206 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> 2008-04-02 19:55:01,253 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:01,284 INFO mapred.TaskRunner (Task.java:discardTaskOutput(502)) - Discarded output of task 'task_200804021954_0001_m_000000_2' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_2
> 2008-04-02 19:55:01,346 WARN fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052" is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:55:01,690 INFO mapred.JobClient (JobClient.java:runJob(1046)) - Task Id : task_200804021954_0001_m_000000_2, Status : FAILED
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,708 WARN conf.Configuration (Configuration.java:loadResource(893)) - C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_2/job.xml:a attempt to override final parameter: hadoop.tmp.dir; Ignoring.
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,724 INFO jvm.JvmMetrics (JvmMetrics.java:init(67)) - Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,974 INFO mapred.MapTask (MapTask.java:run(182)) - numReduceTasks: 1
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:56,520 WARN mapred.TaskTracker (TaskTracker.java:main(2118)) - Error running child
> task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_2: at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_2: at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:03,673 INFO dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064 Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:55:04,001 INFO mapred.TaskInProgress (TaskInProgress.java:updateStatus(420)) - Error from task_200804021954_0001_m_000000_3: java.io.IOException: file file_tmpfile not found
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:07,250 INFO mapred.TaskTracker (TaskTracker.java:reportProgress(1522)) - task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >
> 2008-04-02 19:55:08,218 INFO mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_3 done; removing files.
> 2008-04-02 19:55:08,281 INFO mapred.TaskInProgress (TaskInProgress.java:incompleteSubTask(519)) - TaskInProgress tip_200804021954_0001_m_000000 has failed 4 times.
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:addTrackerTaskFailure(741)) - TaskTracker at 'tracker_seemeight-dx.ds.corp.yahoo.com' turned 'flaky'
> 2008-04-02 19:55:08,281 INFO mapred.JobInProgress (JobInProgress.java:failedTask(1556)) - Aborting job job_200804021954_0001
> 2008-04-02 19:55:08,312 INFO dfs.StateChange (FSNamesystem.java:allocateBlock(1332)) - BLOCK* NameSystem.allocateBlock: /test/output/_logs/history/localhost_1207191264285_job_200804021954_0001_hadoopqa_external job. blk_1520406918918394204
> 2008-04-02 19:55:08,328 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2241 dest: /127.0.0.1:2064
> 2008-04-02 19:55:08,343 INFO dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving block blk_1520406918918394204 src: /127.0.0.1:2242 dest: /127.0.0.1:2066
> 2008-04-02 19:55:08,359 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2066 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1943)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,359 INFO dfs.DataNode (DataNode.java:lastDataNodeRun(1961)) - PacketResponder 0 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,375 INFO dfs.StateChange (FSNamesystem.java:addStoredBlock(2833)) - BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2064 is added to blk_1520406918918394204 size 3263
> 2008-04-02 19:55:08,375 INFO dfs.DataNode (DataNode.java:run(2037)) - Received block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,390 INFO dfs.DataNode (DataNode.java:run(2095)) - PacketResponder 1 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,437 INFO mapred.JobInProgress (JobInProgress.java:kill(1409)) - Killing job 'job_200804021954_0001'
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.