You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Sagar Kohli <sa...@impetus.co.in> on 2011/03/11 06:53:02 UTC

running a job without user

Hi ,

I am trying to run a job which does not require reducer, I commented out the reducer configuration from the job, but now its throwing following error

java.lang.ClassCastException: class java.nio.ByteBuffer
      at java.lang.Class.asSubclass(Unknown Source)
      at org.apache.hadoop.mapred.JobConf.getOutputKeyComparator(JobConf.java:599)
      at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:791)
      at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:524)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
      at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
11/03/11 11:19:12 INFO mapred.JobClient:  map 0% reduce 0%


Is it possible to run a job only with mapper??


Regards
Sagar

________________________________

Are you exploring a Big Data Strategy ? Listen to this recorded webinar on Planning your Hadoop/ NoSQL projects for 2011 at www.impetus.com/featured_webinar?eventid=37

Follow us on www.twitter.com/impetuscalling or visit www.impetus.com to know more.


NOTE: This message may contain information that is confidential, proprietary, privileged or otherwise protected by law. The message is intended solely for the named addressee. If received in error, please destroy and notify the sender. Any use of this email is prohibited when received in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this communication has been maintained nor that the communication is free of errors, virus, interception or interference.

Re: running a job without user

Posted by Harsh J <qw...@gmail.com>.
You need to explictly set number of reducers to 0 to disable reducers;
otherwise, the default value is applied (which is non-zero).

On Fri, Mar 11, 2011 at 11:23 AM, Sagar Kohli <sa...@impetus.co.in> wrote:
> Hi ,
>
>
>
> I am trying to run a job which does not require reducer, I commented out the
> reducer configuration from the job, but now its throwing following error
>
>
>
> java.lang.ClassCastException: class java.nio.ByteBuffer
>
>       at java.lang.Class.asSubclass(Unknown Source)
>
>       at
> org.apache.hadoop.mapred.JobConf.getOutputKeyComparator(JobConf.java:599)
>
>       at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:791)
>
>       at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:524)
>
>       at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
>
>       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
>
>       at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
>
> 11/03/11 11:19:12 INFO mapred.JobClient:  map 0% reduce 0%
>
>
>
>
>
> Is it possible to run a job only with mapper??
>
>
>
>
>
> Regards
>
> Sagar
>
> ________________________________
> Are you exploring a Big Data Strategy ? Listen to this recorded webinar on
> Planning your Hadoop/ NoSQL projects for 2011 at
> www.impetus.com/featured_webinar?eventid=37
>
> Follow us on www.twitter.com/impetuscalling or visit www.impetus.com to know
> more.
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>



-- 
Harsh J
www.harshj.com

Re: running a job without user

Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Sagar,

Set the following property in your JobConf (or using -D)

*  mapred*.*reduce*.*tasks=0*
*
*

That should do the trick.


Alejandro
***
*
On Fri, Mar 11, 2011 at 1:53 PM, Sagar Kohli <sa...@impetus.co.in>wrote:

>  Hi ,
>
>
>
> I am trying to run a job which does not require reducer, I commented out
> the reducer configuration from the job, but now its throwing following error
>
>
>
> *java.lang.ClassCastException*: class java.nio.ByteBuffer
>
>       at java.lang.Class.asSubclass(Unknown Source)
>
>       at org.apache.hadoop.mapred.JobConf.getOutputKeyComparator(*
> JobConf.java:599*)
>
>       at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(*
> MapTask.java:791*)
>
>       at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(*
> MapTask.java:524*)
>
>       at org.apache.hadoop.mapred.MapTask.runNewMapper(*MapTask.java:613*)
>
>       at org.apache.hadoop.mapred.MapTask.run(*MapTask.java:305*)
>
>       at org.apache.hadoop.mapred.LocalJobRunner$Job.run(*
> LocalJobRunner.java:177*)
>
> 11/03/11 11:19:12 INFO mapred.JobClient:  map 0% reduce 0%
>
>
>
>
>
> *Is it possible to run a job only with mapper??*
>
> * *
>
> * *
>
> Regards
>
> Sagar
>
> ------------------------------
>
> Are you exploring a Big Data Strategy ? Listen to this recorded webinar on
> Planning your Hadoop/ NoSQL projects for 2011 at
> www.impetus.com/featured_webinar?eventid=37
>
> Follow us on www.twitter.com/impetuscalling or visit www.impetus.com to
> know more.
>
>
> NOTE: This message may contain information that is confidential,
> proprietary, privileged or otherwise protected by law. The message is
> intended solely for the named addressee. If received in error, please
> destroy and notify the sender. Any use of this email is prohibited when
> received in error. Impetus does not represent, warrant and/or guarantee,
> that the integrity of this communication has been maintained nor that the
> communication is free of errors, virus, interception or interference.
>