You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Ross Boucher <bo...@apple.com> on 2007/09/19 23:30:36 UTC
Running Custom Job
I've got a hadoop cluster running the example files just fine, but am
having difficulty running a custom job.
Specifically, the job starts, and then each task that is scheduled
fails, with the following error:
Error initializing task_0007_m_000063_0:
java.io.IOException: /DFS_ROOT/tmp/mapred/system/submit_i849v1/
job.xml: No such file or directory
I was under the impression that when starting hadoop using bin/hadoop
jar myJarFile.jar <options>, hadoop would do the magic of actually
transferring the jar for me. In fact, I do see it temporarily appear
in tmp/mapred/local/jobTracker (a jar and xml file) on the DFS drive
I am using. But for whatever reason, this is not making it to the
client machines (or locating it in the correct place).
The job itself is very simple, literally a copy of wordcount, with a
slightly changed map function.
If anyone has any ideas, I would greatly appreciate it.
Thanks.
Ross Boucher
boucher@apple.com
RE: Running Custom Job
Posted by Eric Zhang <ez...@yahoo-inc.com>.
I am not sure if you could do this in one job, since you have to do two
sorts. You could run another round of map/reduce job by using InverseMapper
and IdentityReducer (part of org.apache.hadoop.mapred.lib package).
Calling setNumReduceTasks to set num of reduce tasks to 1 can give you one
final output.
For you original questin of missing file. I am not sure if this helps, but I
had similar problem for missing jar file before because I ran multiple jobs
in a program and reused the JobConf object without resetting job's jar file
(setJarByClass), I think what happened was that hadoop would reset the
location of the jar file to the tmp directory every time you run a job.
Eric Zhang
Vespa content @Yahoo!
Work: 408-349-2466
-----Original Message-----
From: Ross Boucher [mailto:boucher@apple.com]
Sent: Wednesday, September 19, 2007 3:43 PM
To: hadoop-user@lucene.apache.org
Subject: Re: Running Custom Job
This problem seems to have gone away by itself.
Now I have my job running, but I'm not entirely sure how to get the output
into something useful to me.
I've counting word frequencies, and I would like the output sorted by
frequency, rather than alphabetically. I would also like the final
output to be in one file, though I'm not sure if this is possible
given that its computed separately. I suppose it wouldn't be too difficult
to post process the files to get them sorted the way I would like and in one
file, but if anyone has some tips on how to do this in my job itself, that
would be great.
Thanks.
Ross Boucher
boucher@apple.com
On Sep 19, 2007, at 2:59 PM, Owen O'Malley wrote:
>
> On Sep 19, 2007, at 2:30 PM, Ross Boucher wrote:
>
>> Specifically, the job starts, and then each task that is scheduled
>> fails, with the following error:
>>
>> Error initializing task_0007_m_000063_0:
>> java.io.IOException: /DFS_ROOT/tmp/mapred/system/submit_i849v1/
>> job.xml: No such file or directory
>
> Look at the configuration of your mapred.system.dir. It MUST be the
> same on both the cluster and submitting node. Note that
> mapred.system.dir must be in the default file system, which must
> also be the same on the cluster and submitting node. Note that
> there is a jira (HADOOP-1100) that would have the cluster pass the
> system directory to the client, which would get rid of this issue.
>
> -- Owen
Re: Running Custom Job
Posted by Ross Boucher <bo...@apple.com>.
Since I was doing terms instead of just words, it was slightly more
complicated then just calling sort.
A simple solution might have been to reverse the order in which the
hadoop job outputs results, i.e.
value key instead of key value
But instead, I used the following set of commands to process the file
and sort it appropriately:
cat * | tr '\t' ' ' | sed 's/^\(.*\) \([0-9]*\)$/\2 \1/' | sort -n >
sorted.txt
Which basically takes the number at the end of each line, and puts it
at the beginning, then calls sort.
Works like a charm. Thanks for the help.
On Sep 19, 2007, at 4:03 PM, Ted Dunning wrote:
>
> I use something like this:
>
> bin/hadoop -getmerge <output-directory> | sort +1n
>
> This wokrs very well because the final counts are relatively small
> compared
> to the original input. There is nothing that says you can't mix MR
> programming with conventional code.
>
> On 9/19/07 3:42 PM, "Ross Boucher" <bo...@apple.com> wrote:
>
>> This problem seems to have gone away by itself.
>>
>> Now I have my job running, but I'm not entirely sure how to get the
>> output into something useful to me.
>>
>> I've counting word frequencies, and I would like the output sorted by
>> frequency, rather than alphabetically. I would also like the final
>> output to be in one file, though I'm not sure if this is possible
>> given that its computed separately. I suppose it wouldn't be too
>> difficult to post process the files to get them sorted the way I
>> would like and in one file, but if anyone has some tips on how to do
>> this in my job itself, that would be great.
>>
>> Thanks.
>>
>> Ross Boucher
>> boucher@apple.com
>>
>>
>> On Sep 19, 2007, at 2:59 PM, Owen O'Malley wrote:
>>
>>>
>>> On Sep 19, 2007, at 2:30 PM, Ross Boucher wrote:
>>>
>>>> Specifically, the job starts, and then each task that is scheduled
>>>> fails, with the following error:
>>>>
>>>> Error initializing task_0007_m_000063_0:
>>>> java.io.IOException: /DFS_ROOT/tmp/mapred/system/submit_i849v1/
>>>> job.xml: No such file or directory
>>>
>>> Look at the configuration of your mapred.system.dir. It MUST be the
>>> same on both the cluster and submitting node. Note that
>>> mapred.system.dir must be in the default file system, which must
>>> also be the same on the cluster and submitting node. Note that
>>> there is a jira (HADOOP-1100) that would have the cluster pass the
>>> system directory to the client, which would get rid of this issue.
>>>
>>> -- Owen
>>
>
Re: Running Custom Job
Posted by Ted Dunning <td...@veoh.com>.
I use something like this:
bin/hadoop -getmerge <output-directory> | sort +1n
This wokrs very well because the final counts are relatively small compared
to the original input. There is nothing that says you can't mix MR
programming with conventional code.
On 9/19/07 3:42 PM, "Ross Boucher" <bo...@apple.com> wrote:
> This problem seems to have gone away by itself.
>
> Now I have my job running, but I'm not entirely sure how to get the
> output into something useful to me.
>
> I've counting word frequencies, and I would like the output sorted by
> frequency, rather than alphabetically. I would also like the final
> output to be in one file, though I'm not sure if this is possible
> given that its computed separately. I suppose it wouldn't be too
> difficult to post process the files to get them sorted the way I
> would like and in one file, but if anyone has some tips on how to do
> this in my job itself, that would be great.
>
> Thanks.
>
> Ross Boucher
> boucher@apple.com
>
>
> On Sep 19, 2007, at 2:59 PM, Owen O'Malley wrote:
>
>>
>> On Sep 19, 2007, at 2:30 PM, Ross Boucher wrote:
>>
>>> Specifically, the job starts, and then each task that is scheduled
>>> fails, with the following error:
>>>
>>> Error initializing task_0007_m_000063_0:
>>> java.io.IOException: /DFS_ROOT/tmp/mapred/system/submit_i849v1/
>>> job.xml: No such file or directory
>>
>> Look at the configuration of your mapred.system.dir. It MUST be the
>> same on both the cluster and submitting node. Note that
>> mapred.system.dir must be in the default file system, which must
>> also be the same on the cluster and submitting node. Note that
>> there is a jira (HADOOP-1100) that would have the cluster pass the
>> system directory to the client, which would get rid of this issue.
>>
>> -- Owen
>
Re: Running Custom Job
Posted by Ross Boucher <bo...@apple.com>.
This problem seems to have gone away by itself.
Now I have my job running, but I'm not entirely sure how to get the
output into something useful to me.
I've counting word frequencies, and I would like the output sorted by
frequency, rather than alphabetically. I would also like the final
output to be in one file, though I'm not sure if this is possible
given that its computed separately. I suppose it wouldn't be too
difficult to post process the files to get them sorted the way I
would like and in one file, but if anyone has some tips on how to do
this in my job itself, that would be great.
Thanks.
Ross Boucher
boucher@apple.com
On Sep 19, 2007, at 2:59 PM, Owen O'Malley wrote:
>
> On Sep 19, 2007, at 2:30 PM, Ross Boucher wrote:
>
>> Specifically, the job starts, and then each task that is scheduled
>> fails, with the following error:
>>
>> Error initializing task_0007_m_000063_0:
>> java.io.IOException: /DFS_ROOT/tmp/mapred/system/submit_i849v1/
>> job.xml: No such file or directory
>
> Look at the configuration of your mapred.system.dir. It MUST be the
> same on both the cluster and submitting node. Note that
> mapred.system.dir must be in the default file system, which must
> also be the same on the cluster and submitting node. Note that
> there is a jira (HADOOP-1100) that would have the cluster pass the
> system directory to the client, which would get rid of this issue.
>
> -- Owen
Re: Running Custom Job
Posted by Ross Boucher <bo...@apple.com>.
I re ran into this problem yesterday, and just thought I'd share the
cause for the record.
After looking through my logs, I was seeing a bunch of exceptions
trying to bound to already in use ports. As a result, the namenode
and jobtracker were failing to launch, which I had not noticed. In
fact, jps reported that they were not running. But in reality, they
were running, as some sort of zombie process on my system, which was
blocking the startup. After killing the processes, everything went
back to working well.
On Sep 19, 2007, at 2:59 PM, Owen O'Malley wrote:
>
> On Sep 19, 2007, at 2:30 PM, Ross Boucher wrote:
>
>> Specifically, the job starts, and then each task that is scheduled
>> fails, with the following error:
>>
>> Error initializing task_0007_m_000063_0:
>> java.io.IOException: /DFS_ROOT/tmp/mapred/system/submit_i849v1/
>> job.xml: No such file or directory
>
> Look at the configuration of your mapred.system.dir. It MUST be the
> same on both the cluster and submitting node. Note that
> mapred.system.dir must be in the default file system, which must
> also be the same on the cluster and submitting node. Note that
> there is a jira (HADOOP-1100) that would have the cluster pass the
> system directory to the client, which would get rid of this issue.
>
> -- Owen
Re: Running Custom Job
Posted by Owen O'Malley <oo...@yahoo-inc.com>.
On Sep 19, 2007, at 2:30 PM, Ross Boucher wrote:
> Specifically, the job starts, and then each task that is scheduled
> fails, with the following error:
>
> Error initializing task_0007_m_000063_0:
> java.io.IOException: /DFS_ROOT/tmp/mapred/system/submit_i849v1/
> job.xml: No such file or directory
Look at the configuration of your mapred.system.dir. It MUST be the
same on both the cluster and submitting node. Note that
mapred.system.dir must be in the default file system, which must also
be the same on the cluster and submitting node. Note that there is a
jira (HADOOP-1100) that would have the cluster pass the system
directory to the client, which would get rid of this issue.
-- Owen