You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Mark Kerzner <ma...@gmail.com> on 2011/09/15 04:48:53 UTC
Am i crazy? - question about hadoop streaming
Hi,
I am using the latest Cloudera distribution, and with that I am able to use
the latest Hadoop API, which I believe is 0.21, for such things as
import org.apache.hadoop.mapreduce.Reducer;
So I am using mapreduce, not mapred, and everything works fine.
However, in a small streaming job, trying it out with Java classes first, I
get this error
Exception in thread "main" java.lang.RuntimeException: class mypackage.Map
not org.apache.hadoop.mapred.Mapper -- which it really is not, it is a
mapreduce.Mapper.
So it seems that Cloudera backports some of the advances but for streaming
it is still the old API.
So it is me or the world?
Thank you,
Mark
Re: Am i crazy? - question about hadoop streaming
Posted by Mark Kerzner <ma...@gmail.com>.
I am sorry, you are right.
mark
On Wed, Sep 14, 2011 at 9:52 PM, Konstantin Boudnik <co...@apache.org> wrote:
> I am sure if you ask at provider's specific list you'll get a better answer
> than from common Hadoop list ;)
>
> Cos
>
> On Wed, Sep 14, 2011 at 09:48PM, Mark Kerzner wrote:
> > Hi,
> >
> > I am using the latest Cloudera distribution, and with that I am able to
> use
> > the latest Hadoop API, which I believe is 0.21, for such things as
> >
> > import org.apache.hadoop.mapreduce.Reducer;
> >
> > So I am using mapreduce, not mapred, and everything works fine.
> >
> > However, in a small streaming job, trying it out with Java classes first,
> I
> > get this error
> >
> > Exception in thread "main" java.lang.RuntimeException: class
> mypackage.Map
> > not org.apache.hadoop.mapred.Mapper -- which it really is not, it is a
> > mapreduce.Mapper.
> >
> > So it seems that Cloudera backports some of the advances but for
> streaming
> > it is still the old API.
> >
> > So it is me or the world?
> >
> > Thank you,
> > Mark
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
>
> iF4EAREIAAYFAk5xaGIACgkQenyFlstYjhKtZAEAmNtHK9DqBFmZ2DTJgAxEbF+p
> P0Tek1iW1P1ZwlqGDRIA/AuVVaNiul1bQM0NRYuAVxLn7sJOTSCQG5PRGJUQdvjq
> =Z/hO
> -----END PGP SIGNATURE-----
>
>
Re: Am i crazy? - question about hadoop streaming
Posted by Konstantin Boudnik <co...@apache.org>.
I am sure if you ask at provider's specific list you'll get a better answer
than from common Hadoop list ;)
Cos
On Wed, Sep 14, 2011 at 09:48PM, Mark Kerzner wrote:
> Hi,
>
> I am using the latest Cloudera distribution, and with that I am able to use
> the latest Hadoop API, which I believe is 0.21, for such things as
>
> import org.apache.hadoop.mapreduce.Reducer;
>
> So I am using mapreduce, not mapred, and everything works fine.
>
> However, in a small streaming job, trying it out with Java classes first, I
> get this error
>
> Exception in thread "main" java.lang.RuntimeException: class mypackage.Map
> not org.apache.hadoop.mapred.Mapper -- which it really is not, it is a
> mapreduce.Mapper.
>
> So it seems that Cloudera backports some of the advances but for streaming
> it is still the old API.
>
> So it is me or the world?
>
> Thank you,
> Mark
Re: Am i crazy? - question about hadoop streaming
Posted by Mark Kerzner <ma...@gmail.com>.
Thank you, Prashant, it seems so. I already verified this by refactoring the
code to use 0.20 API as well as 0.21 API in two different packages, and
streaming happily works with 0.20.
Mark
On Wed, Sep 14, 2011 at 11:46 PM, Prashant <pr...@imaginea.com> wrote:
> On 09/15/2011 08:18 AM, Mark Kerzner wrote:
>
>> Hi,
>>
>> I am using the latest Cloudera distribution, and with that I am able to
>> use
>> the latest Hadoop API, which I believe is 0.21, for such things as
>>
>> import org.apache.hadoop.mapreduce.**Reducer;
>>
>> So I am using mapreduce, not mapred, and everything works fine.
>>
>> However, in a small streaming job, trying it out with Java classes first,
>> I
>> get this error
>>
>> Exception in thread "main" java.lang.RuntimeException: class mypackage.Map
>> not org.apache.hadoop.mapred.**Mapper -- which it really is not, it is a
>> mapreduce.Mapper.
>>
>> So it seems that Cloudera backports some of the advances but for streaming
>> it is still the old API.
>>
>> So it is me or the world?
>>
>> Thank you,
>> Mark
>>
>> The world!
>
Re: Am i crazy? - question about hadoop streaming
Posted by Prashant <pr...@imaginea.com>.
On 09/15/2011 08:18 AM, Mark Kerzner wrote:
> Hi,
>
> I am using the latest Cloudera distribution, and with that I am able to use
> the latest Hadoop API, which I believe is 0.21, for such things as
>
> import org.apache.hadoop.mapreduce.Reducer;
>
> So I am using mapreduce, not mapred, and everything works fine.
>
> However, in a small streaming job, trying it out with Java classes first, I
> get this error
>
> Exception in thread "main" java.lang.RuntimeException: class mypackage.Map
> not org.apache.hadoop.mapred.Mapper -- which it really is not, it is a
> mapreduce.Mapper.
>
> So it seems that Cloudera backports some of the advances but for streaming
> it is still the old API.
>
> So it is me or the world?
>
> Thank you,
> Mark
>
The world!