You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Andrew McNabb <am...@mcnabbs.org> on 2007/01/09 03:16:50 UTC
Streaming in Hadoop 0.10
In the past, I would get a job failure if a streaming mapper or reducer
died (for example, due to a Python exception). However, in 0.10, I've
been getting output in the Hadoop task list that looks like:
tip_0002_m_000000 100.00% MRErr: ImportError: No module named
scipy
On the job page, it says that the job succeeded and that there were no
failures. Naturally, it would be nice to know when errors like this
happen. Is this a bug in 0.10, or am I looking at things wrong?
Thanks.
--
Andrew McNabb
http://www.mcnabbs.org/andrew/
PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868
Re: Streaming in Hadoop 0.10
Posted by Andrew McNabb <am...@mcnabbs.org>.
On Tue, Jan 09, 2007 at 04:59:27PM -0700, Andrew McNabb wrote:
>
> The Java documentation doesn't say that exec can return null. I guess
> I'm confused about what's causing this exception to be thrown. Does
> anyone have any ideas? Thanks.
>
Ok. I dug around in some other logs and found that there was an
java.io.IOException thrown because it couldn't find the executable. I'm
still not quite sure how that got turned into a NullPointerException.
I'm having a real hard time with some of these error messages.
--
Andrew McNabb
http://www.mcnabbs.org/andrew/
PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868
RE: Streaming in Hadoop 0.10
Posted by Mahadev Konar <ma...@yahoo-inc.com>.
The only way I can see this happening would be when exec() fails and throws
out an exception. Did you take a look at the per task logs to see if there
is anything being logged in there?
Which Hadoop version are you using? Are you transferring your mapper files
using -file option?
Regards
Mahadev
> -----Original Message-----
> From: Andrew McNabb [mailto:amcnabb@mcnabbs.org]
> Sent: Tuesday, January 09, 2007 3:59 PM
> To: hadoop-dev@lucene.apache.org
> Subject: Re: Streaming in Hadoop 0.10
>
> On Tue, Jan 09, 2007 at 12:36:10PM -0700, Andrew McNabb wrote:
> >
> > java.lang.NullPointerException
> > at
> org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:4
> 55)
> > at
> org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:578)
> > at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:109)
> > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:51)
> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:183)
> > at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1341)
> >
>
> Even when I fixed the bug in my mapper, it still gives this error. I
> can't see anything else in the logs. I went in and looked at the code.
> Line 455 of PipeMapRed.java is:
>
> sim.waitFor();
>
> The Java documentation doesn't say anything about waitFor throwing a
> NullPointerException, and I don't see anything about how sim (a Process
> object) would get set to null. Sim is set at line 301:
>
> sim = Runtime.getRuntime().exec(argvSplit, childEnv.toArray());
>
> The Java documentation doesn't say that exec can return null. I guess
> I'm confused about what's causing this exception to be thrown. Does
> anyone have any ideas? Thanks.
>
>
> --
> Andrew McNabb
> http://www.mcnabbs.org/andrew/
> PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868
Re: Streaming in Hadoop 0.10
Posted by Andrew McNabb <am...@mcnabbs.org>.
On Tue, Jan 09, 2007 at 12:36:10PM -0700, Andrew McNabb wrote:
>
> java.lang.NullPointerException
> at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:455)
> at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:578)
> at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:109)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:51)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:183)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1341)
>
Even when I fixed the bug in my mapper, it still gives this error. I
can't see anything else in the logs. I went in and looked at the code.
Line 455 of PipeMapRed.java is:
sim.waitFor();
The Java documentation doesn't say anything about waitFor throwing a
NullPointerException, and I don't see anything about how sim (a Process
object) would get set to null. Sim is set at line 301:
sim = Runtime.getRuntime().exec(argvSplit, childEnv.toArray());
The Java documentation doesn't say that exec can return null. I guess
I'm confused about what's causing this exception to be thrown. Does
anyone have any ideas? Thanks.
--
Andrew McNabb
http://www.mcnabbs.org/andrew/
PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868
Re: Streaming in Hadoop 0.10
Posted by Andrew McNabb <am...@mcnabbs.org>.
On Mon, Jan 08, 2007 at 07:16:50PM -0700, Andrew McNabb wrote:
> In the past, I would get a job failure if a streaming mapper or reducer
> died (for example, due to a Python exception). However, in 0.10, I've
> been getting output in the Hadoop task list that looks like:
>
> tip_0002_m_000000 100.00% MRErr: ImportError: No module named
> scipy
On the same note, I'm currently getting the following traceback in the
errors section of the map task list:
java.lang.NullPointerException
at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:455)
at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:578)
at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:109)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:51)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:183)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1341)
The map task failed, and it was the mappers fault, but this error
message makes it appear that something in Hadoop was to blame. However,
the real cause of failure is that the mapper raised a an AttributeError
and died.
Since it sounded like someone was working on improving the display of
error messages on the task list page, I thought this might be useful.
Thanks for everything.
--
Andrew McNabb
http://www.mcnabbs.org/andrew/
PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868