You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org> on 2008/09/25 09:09:44 UTC
[jira] Commented: (HADOOP-4264) DFSIO is failing on 500 nodes
cluster
[ https://issues.apache.org/jira/browse/HADOOP-4264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12634392#action_12634392 ]
Amareshwari Sriramadasu commented on HADOOP-4264:
-------------------------------------------------
TestDFSIO is creating files in DATA_DIR, which is not output directory for the mapreduce job. If speculative execution is on for the test, two attempts of the task are trying to create same file. This test should have speculative execution off.
> DFSIO is failing on 500 nodes cluster
> -------------------------------------
>
> Key: HADOOP-4264
> URL: https://issues.apache.org/jira/browse/HADOOP-4264
> Project: Hadoop Core
> Issue Type: Bug
> Components: io, test
> Affects Versions: 0.19.0
> Reporter: Suman Sehgal
>
> On executing following command :
> bin/hadoop jar ~/hadoop/hadoop-0.19.0-test.jar TestDFSIO -write -nrFiles 990 -fileSize 320
> This error occurs:
> 08/09/24 06:15:03 INFO mapred.JobClient: map 98% reduce 32%
> java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1201)
> at org.apache.hadoop.fs.TestDFSIO.runIOTest(TestDFSIO.java:236)
> at org.apache.hadoop.fs.TestDFSIO.writeTest(TestDFSIO.java:218)
> at org.apache.hadoop.fs.TestDFSIO.main(TestDFSIO.java:354)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
> at org.apache.hadoop.test.AllTestDriver.main(AllTestDriver.java:77)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:165)
> at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)
> On looking at hadoop logs, It seems that file names are clashing
> 2008-09-24 06:21:41,618 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_200809240600_0005_m_000802_2_1222236048515' from 'tracker_xxxx/client x.x.x.x:xxxxx'
> 2008-09-24 06:21:41,627 INFO org.apache.hadoop.mapred.JobTracker: Adding task 'attempt_200809240600_0005_m_000802_4_1222236048515' to tip task_200809240600_0005_m_000802, for tracker 'tracker_xxxx/client x.x.x.x:xxxxx'
> 2008-09-24 06:21:41,627 INFO org.apache.hadoop.mapred.JobInProgress: Choosing rack-local task task_200809240600_0005_m_000802
> 2008-09-24 06:21:41,724 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_200809240600_0005_m_000900_2_1222236048515: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException: failed to create file /benchmarks/TestDFSIO/io_data/test_io_20 for DFSClient_attempt_200809240600_0005_m_000900_2_1222236048515 on client client x.x.x.x, because this file is already being created by DFSClient_attempt_200809240600_0005_m_000900_0_1222236048515 on client x.x.x.x
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.