Can we use Inheritance hierarchy to specify the outputvalue class for mapper which is also inputvalue class for the reducer ?

[Anuja Kulkarni <an...@yahoo.com>]

Hi,

We have class hierarchy for output value for both mapper as well as reducer class   as parent (abstract class) , child1,child2,… 
 
We have mapper class which is 
specified with its outputvalue class as parent class ; the map function 
will emit  either  child1 or child2 depending on the logic we used ( 
and reducer is having its inputvalue class as parent class).
 
But we are getting error as  "java.io.IOException: Type mismatch in value from map: expected parent classreceived child class”
 So is this possible to specify parent class as output/inputvalue class in case of mapper as well as in reducer ; since we want to follow some object-oriented approach. What will be the correct way of achieving this ?


- Anuja

Re: Can we use Inheritance hierarchy to specify the outputvalue class for mapper which is also inputvalue class for the reducer ?

[Kamesh <ka...@imaginea.com>]

On Tuesday 04 October 2011 10:16 PM, Anuja Kulkarni wrote:
> Hi,
>
> We have class hierarchy for output value for both mapper as well as 
> reducer class   as parent (abstract class) , child1,child2,…
> We have mapper class which is specified with its outputvalue class as 
> parent class ; the map function will emit  either  child1 or child2 
> depending on the logic we used ( and reducer is having its inputvalue 
> class as parent class).
> But we are getting error as "*java.io.IOException: Type mismatch in 
> value from map: expected **parent classreceived child class”*
> So is this possible to specify parent class as output/inputvalue class 
> in case of mapper as well as in reducer ; since we want to follow some 
> object-oriented approach. What will be the correct way of achieving this ?
>
>
> - Anuja
Hi Anuja,

bq. So is this possible to specify parent class as output/inputvalue 
class in case of mapper as well as in reducer ;

I **think** it is not possible.

bq. What will be the correct way of achieving this ?

I feel the following approach works (there could be a better way of 
doing this)

Create a TupleWritable instance with either child1 or child2 array as an 
argument based on your logic at the mapper side and write the same 
TupleWritable instance as a value of Mapper output value.

At the reducer side, get the writable type from the TupleWritable and do 
instanceof check to determine the appropriate value type.

-- 
Thanks&Regards,
Bh.V.S.Kamesh.