You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Lucian Iordache <lu...@gmail.com> on 2011/05/16 17:31:09 UTC
Job works only when TaskTracker & JobTracker on the same machine
Hello,
I have a Hadoop/Hbase cluster, cloudera cdh3u0 version, with several
machines. I've created a job that does some work using the information from
a HBase table.
The next example *works* fine:
- start JobTracker
- start TaskTracker *on the same machine* as the JobTracker
- run the job
- the job completes successfully
The next example *does not work*:
- start JobTracker
- start TaskTracker *on another machine* (not the same as the JobTracker)
- run the job
- I get the next error in the console:
11/04/21 12:21:01 INFO mapred.JobClient: Running job: job_201104201820_0007
11/04/21 12:21:02 INFO mapred.JobClient: map 0% reduce 0%
11/04/21 12:21:02 INFO mapred.JobClient: Task Id :
attempt_201104201820_0007_m_000002_0, Status : FAILED
Error initializing attempt_201104201820_0007_m_000002_0:
java.io.FileNotFoundException: File
file:/var/ui/hadoop/mapred/system/job_201104201820_0007/jobToken<file:///var/ui/hadoop/mapred/system/job_201104201820_0007/jobToken>does
not exist.
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:383)
at
org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
at
org.apache.hadoop.mapred.TaskTracker.localizeJobTokenFile(TaskTracker.java:4033)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1148)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1089)
at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:2257)
at
org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:2221)
11/04/21 12:21:02 WARN mapred.JobClient: Error reading task outputhttp://
cene.ro.schlund.net:50060/tasklog?plaintext=true&attemptid=attempt_201104201820_0007_m_000002_0&filter=stdout
11/04/21 12:21:02 WARN mapred.JobClient: Error reading task outputhttp://
cene.ro.schlund.net:50060/tasklog?plaintext=true&attemptid=attempt_201104201820_0007_m_000002_0&filter=stderr
11/04/21 12:21:02 INFO mapred.JobClient: Task Id :
attempt_201104201820_0007_r_000002_0, Status : FAILED
Can you help me with this problem?
Best Regards,
Lucian
Re: Job works only when TaskTracker & JobTracker on the same machine
Posted by Lucian Iordache <lu...@gmail.com>.
Hi Todd,
I think the cluster is well configured, the HDFS and HBase work fine in
distributed mode (several datanodes and regionservers are started and work
correctly).
- all the slaves are present in the ../conf/slaves file
- the jobtracker host and port are well set in mapred-site on the
TaskTracker machine
I know that it must be some configuration problem, but I don't manage to
find out which is it.
By the way, the problem appeared after the upgrade of the cluster from
CDH3B4 to CDH3U0
Thanks,
Lucian
On Tue, May 17, 2011 at 8:42 PM, Todd Lipcon <to...@cloudera.com> wrote:
> Hi Lucian,
>
> Sounds like you didn't configure MapReduce to run on top of HDFS? If you
> want to run on a distributed cluster, you need a distributed file system set
> up as well.
>
> -Todd
>
>
> On Mon, May 16, 2011 at 8:31 AM, Lucian Iordache <
> lucian.george.iordache@gmail.com> wrote:
>
>> Hello,
>>
>> I have a Hadoop/Hbase cluster, cloudera cdh3u0 version, with several
>> machines. I've created a job that does some work using the information from
>> a HBase table.
>>
>> The next example *works* fine:
>> - start JobTracker
>> - start TaskTracker *on the same machine* as the JobTracker
>> - run the job
>> - the job completes successfully
>>
>> The next example *does not work*:
>> - start JobTracker
>> - start TaskTracker *on another machine* (not the same as the JobTracker)
>> - run the job
>> - I get the next error in the console:
>> 11/04/21 12:21:01 INFO mapred.JobClient: Running job:
>> job_201104201820_0007
>> 11/04/21 12:21:02 INFO mapred.JobClient: map 0% reduce 0%
>> 11/04/21 12:21:02 INFO mapred.JobClient: Task Id :
>> attempt_201104201820_0007_m_000002_0, Status : FAILED
>> Error initializing attempt_201104201820_0007_m_000002_0:
>> java.io.FileNotFoundException: File
>> file:/var/ui/hadoop/mapred/system/job_201104201820_0007/jobToken does not
>> exist.
>> at
>> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:383)
>> at
>> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
>> at
>> org.apache.hadoop.mapred.TaskTracker.localizeJobTokenFile(TaskTracker.java:4033)
>> at
>> org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1148)
>> at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1089)
>> at
>> org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:2257)
>> at
>> org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:2221)
>> 11/04/21 12:21:02 WARN mapred.JobClient: Error reading task outputhttp://
>> cene.ro.schlund.net:50060/tasklog?plaintext=true&attemptid=attempt_201104201820_0007_m_000002_0&filter=stdout
>> 11/04/21 12:21:02 WARN mapred.JobClient: Error reading task outputhttp://
>> cene.ro.schlund.net:50060/tasklog?plaintext=true&attemptid=attempt_201104201820_0007_m_000002_0&filter=stderr
>> 11/04/21 12:21:02 INFO mapred.JobClient: Task Id :
>> attempt_201104201820_0007_r_000002_0, Status : FAILED
>>
>> Can you help me with this problem?
>>
>> Best Regards,
>> Lucian
>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
Re: Job works only when TaskTracker & JobTracker on the same machine
Posted by Todd Lipcon <to...@cloudera.com>.
Hi Lucian,
Sounds like you didn't configure MapReduce to run on top of HDFS? If you
want to run on a distributed cluster, you need a distributed file system set
up as well.
-Todd
On Mon, May 16, 2011 at 8:31 AM, Lucian Iordache <
lucian.george.iordache@gmail.com> wrote:
> Hello,
>
> I have a Hadoop/Hbase cluster, cloudera cdh3u0 version, with several
> machines. I've created a job that does some work using the information from
> a HBase table.
>
> The next example *works* fine:
> - start JobTracker
> - start TaskTracker *on the same machine* as the JobTracker
> - run the job
> - the job completes successfully
>
> The next example *does not work*:
> - start JobTracker
> - start TaskTracker *on another machine* (not the same as the JobTracker)
> - run the job
> - I get the next error in the console:
> 11/04/21 12:21:01 INFO mapred.JobClient: Running job: job_201104201820_0007
> 11/04/21 12:21:02 INFO mapred.JobClient: map 0% reduce 0%
> 11/04/21 12:21:02 INFO mapred.JobClient: Task Id :
> attempt_201104201820_0007_m_000002_0, Status : FAILED
> Error initializing attempt_201104201820_0007_m_000002_0:
> java.io.FileNotFoundException: File
> file:/var/ui/hadoop/mapred/system/job_201104201820_0007/jobToken does not
> exist.
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:383)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
> at
> org.apache.hadoop.mapred.TaskTracker.localizeJobTokenFile(TaskTracker.java:4033)
> at
> org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1148)
> at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1089)
> at org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:2257)
> at
> org.apache.hadoop.mapred.TaskTracker$TaskLauncher.run(TaskTracker.java:2221)
> 11/04/21 12:21:02 WARN mapred.JobClient: Error reading task outputhttp://
> cene.ro.schlund.net:50060/tasklog?plaintext=true&attemptid=attempt_201104201820_0007_m_000002_0&filter=stdout
> 11/04/21 12:21:02 WARN mapred.JobClient: Error reading task outputhttp://
> cene.ro.schlund.net:50060/tasklog?plaintext=true&attemptid=attempt_201104201820_0007_m_000002_0&filter=stderr
> 11/04/21 12:21:02 INFO mapred.JobClient: Task Id :
> attempt_201104201820_0007_r_000002_0, Status : FAILED
>
> Can you help me with this problem?
>
> Best Regards,
> Lucian
>
--
Todd Lipcon
Software Engineer, Cloudera