You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by yu...@ISI.EDU on 2011/07/14 04:44:07 UTC

shuffle access error to class o.a.h.m.JobInitializationPoller$JobInitializationThread

Greetings,

I'm running common/hdfs/mapreduce trunk version
-r1146503; I'm getting the following error at the reduce phase:

Error: tried to access class org.apache.hadoop.mapred.JobInitializationPoller$JobInitializationThread from class org.apache.hadoop.mapreduce.task.reduce.Shuffle

All reduce tasks die this death.

Any clue would be appreciated.

Thanks,

  -Yuri

Re: shuffle access error to class o.a.h.m.JobInitializationPoller$JobInitializationThread

Posted by Yuri Pradkin <yu...@isi.edu>.
This problem didn't show when I ran tests, which led me to conclude that it's 
rooted in my local setup.

The problem magically disappeared after I cleaned up the classpath a bit 
(removing fault injection jars and a few others).  This experience taught me a 
few things:

1. Fault injection compiles by default (why? how do I disable its build? -- it 
doesn't build cleanly btw - some confusion about version numbers).

2. Classpaths are a mess.  The scripts driving hadoop 
(bin/{hadoop,mapred,hdfs}) loook for jars in a bunch of places and are not 
consistent with one another.  Related to that: there doesn't seem to be a 
build target for mapred,hdfs,common that would deploy all necessary jars where 
they can be picked up by the scripts; I think this needs to be fixed.

Please correct me if I'm wrong on any of these observations.

Thanks,

  -Yuri

On Wednesday 13 July 2011 19:44:07 yuri@isi.edu wrote:
> Greetings,
> 
> I'm running common/hdfs/mapreduce trunk version
> -r1146503; I'm getting the following error at the reduce phase:
> 
> Error: tried to access class
> org.apache.hadoop.mapred.JobInitializationPoller$JobInitializationThread
> from class org.apache.hadoop.mapreduce.task.reduce.Shuffle
> 
> All reduce tasks die this death.
> 
> Any clue would be appreciated.
> 
> Thanks,
> 
>   -Yuri

Re: shuffle access error to class o.a.h.m.JobInitializationPoller$JobInitializationThread

Posted by yu...@ISI.EDU.
Hi Arun,

Before I file a jira, I want to make sure it's not a simple 
misconfiguration on my part, or a "wrong" jar in the classpath,
or something else obvious to current developers.

I was running a streaming job, here's all I could find in the 
tasktracker's log:

2011-07-14 07:24:20,307 WARN org.apache.hadoop.ipc.Server: IPC Server Responder, call fatalError(attempt_201107131536_0003_r_000134_0, tried to access class org.apache.hadoop.mapred.JobInitializationPoller$JobInitializationThread from class org.apache.hadoop.mapreduce.task.reduce.Shuffle), rpc version=1, client version=19, methodsFingerPrint=937413979 from 127.0.0.1:42895: output error
2011-07-14 07:24:20,308 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 35633 caught an exception
java.nio.channels.ClosedChannelException
    at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:144)
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:342)
    at org.apache.hadoop.ipc.Server.channelWrite(Server.java:1844)
    at org.apache.hadoop.ipc.Server.access$1900(Server.java:97)
    at org.apache.hadoop.ipc.Server$Responder.processResponse(Server.java:797)
    at org.apache.hadoop.ipc.Server$Responder.doRespond(Server.java:861)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1529)

Thanks,

  -Yuri

On Wed, Jul 13, 2011 at 07:46:25PM -0700, acm@hortonworks.com wrote:
> Yuri,
> 
> Do you have the stack trace?
> 
> Pls file a jira. Thanks.
> 
> Sent from my iPhone
> 
> On Jul 13, 2011, at 7:44 PM, yuri@ISI.EDU wrote:
> 
> > Greetings,
> >
> > I'm running common/hdfs/mapreduce trunk version
> > -r1146503; I'm getting the following error at the reduce phase:
> >
> > Error: tried to access class org.apache.hadoop.mapred.JobInitializationPoller$JobInitializationThread from class org.apache.hadoop.mapreduce.task.reduce.Shuffle
> >
> > All reduce tasks die this death.
> >
> > Any clue would be appreciated.
> >
> > Thanks,
> >
> >  -Yuri

Re: shuffle access error to class o.a.h.m.JobInitializationPoller$JobInitializationThread

Posted by Arun Murthy <ac...@hortonworks.com>.
Yuri,

Do you have the stack trace?

Pls file a jira. Thanks.

Sent from my iPhone

On Jul 13, 2011, at 7:44 PM, yuri@ISI.EDU wrote:

> Greetings,
>
> I'm running common/hdfs/mapreduce trunk version
> -r1146503; I'm getting the following error at the reduce phase:
>
> Error: tried to access class org.apache.hadoop.mapred.JobInitializationPoller$JobInitializationThread from class org.apache.hadoop.mapreduce.task.reduce.Shuffle
>
> All reduce tasks die this death.
>
> Any clue would be appreciated.
>
> Thanks,
>
>  -Yuri