getStructFieldData method on StructObjectInspector

["kulkarni.swarnim@gmail.com" <ku...@gmail.com>]

I am trying to write a custom ObjectInspector extending the
StructObjectInspector and got a little confused about the use of the
getStructFieldData method on the inspector. Looking at the definition of
the method:

public Object getStructFieldData(Object data, StructField fieldRef);

I understand that the use of this method is to retrieve the specific given
field from the buffer. However, what I don't understand is what is it
expected to return. I looked around the tests and related code and mostly
stuff returned was either a LazyPrimitive or a LazyNonPrimitive, but I
couldn't find anything that enforces this(specially given that the return
type is a plain "Object")! Does this mean that I am free to return even my
custom object as a return type of this method? If so, what is the guarantee
that it will be interpreted correctly down the pipeline?

Thanks,
-- 
Swarnim

Re: getStructFieldData method on StructObjectInspector

["kulkarni.swarnim@gmail.com" <ku...@gmail.com>]

Thanks Edward for your reply on this.

Would you mind giving a very small example on how a struct corresponds to a
Map? I am having hard time understanding what the K/V pairs in the map
would look like.

Thanks again.

On Tue, May 29, 2012 at 10:16 AM, Edward Capriolo <ed...@gmail.com>wrote:

> Returning custom writables will not work. In most cases the methods
> return Object because the types can be many things that do not fall
> under a single superclass other then object. like Integer,IntWritable,
> Array<IntWritable>, or Map. In your case, a struct corresponds to a
> Map.
>
> On Tue, May 29, 2012 at 11:08 AM, kulkarni.swarnim@gmail.com
> <ku...@gmail.com> wrote:
> > If someone can help understand this, I would really appreciate.
> >
> > On Fri, May 25, 2012 at 3:58 PM, kulkarni.swarnim@gmail.com
> > <ku...@gmail.com> wrote:
> >>
> >> I am trying to write a custom ObjectInspector extending the
> >> StructObjectInspector and got a little confused about the use of the
> >> getStructFieldData method on the inspector. Looking at the definition
> of the
> >> method:
> >>
> >> public Object getStructFieldData(Object data, StructField fieldRef);
> >>
> >> I understand that the use of this method is to retrieve the specific
> given
> >> field from the buffer. However, what I don't understand is what is it
> >> expected to return. I looked around the tests and related code and
> mostly
> >> stuff returned was either a LazyPrimitive or a LazyNonPrimitive, but I
> >> couldn't find anything that enforces this(specially given that the
> return
> >> type is a plain "Object")! Does this mean that I am free to return even
> my
> >> custom object as a return type of this method? If so, what is the
> guarantee
> >> that it will be interpreted correctly down the pipeline?
> >>
> >> Thanks,
> >> --
> >> Swarnim
> >
> >
> >
> >
> > --
> > Swarnim
>



-- 
Swarnim

Re: getStructFieldData method on StructObjectInspector

[Edward Capriolo <ed...@gmail.com>]

Returning custom writables will not work. In most cases the methods
return Object because the types can be many things that do not fall
under a single superclass other then object. like Integer,IntWritable,
Array<IntWritable>, or Map. In your case, a struct corresponds to a
Map.

On Tue, May 29, 2012 at 11:08 AM, kulkarni.swarnim@gmail.com
<ku...@gmail.com> wrote:
> If someone can help understand this, I would really appreciate.
>
> On Fri, May 25, 2012 at 3:58 PM, kulkarni.swarnim@gmail.com
> <ku...@gmail.com> wrote:
>>
>> I am trying to write a custom ObjectInspector extending the
>> StructObjectInspector and got a little confused about the use of the
>> getStructFieldData method on the inspector. Looking at the definition of the
>> method:
>>
>> public Object getStructFieldData(Object data, StructField fieldRef);
>>
>> I understand that the use of this method is to retrieve the specific given
>> field from the buffer. However, what I don't understand is what is it
>> expected to return. I looked around the tests and related code and mostly
>> stuff returned was either a LazyPrimitive or a LazyNonPrimitive, but I
>> couldn't find anything that enforces this(specially given that the return
>> type is a plain "Object")! Does this mean that I am free to return even my
>> custom object as a return type of this method? If so, what is the guarantee
>> that it will be interpreted correctly down the pipeline?
>>
>> Thanks,
>> --
>> Swarnim
>
>
>
>
> --
> Swarnim

Re: getStructFieldData method on StructObjectInspector

["kulkarni.swarnim@gmail.com" <ku...@gmail.com>]

If someone can help understand this, I would really appreciate.

On Fri, May 25, 2012 at 3:58 PM, kulkarni.swarnim@gmail.com <
kulkarni.swarnim@gmail.com> wrote:

> I am trying to write a custom ObjectInspector extending the
> StructObjectInspector and got a little confused about the use of the
> getStructFieldData method on the inspector. Looking at the definition of
> the method:
>
> public Object getStructFieldData(Object data, StructField fieldRef);
>
> I understand that the use of this method is to retrieve the specific given
> field from the buffer. However, what I don't understand is what is it
> expected to return. I looked around the tests and related code and mostly
> stuff returned was either a LazyPrimitive or a LazyNonPrimitive, but I
> couldn't find anything that enforces this(specially given that the return
> type is a plain "Object")! Does this mean that I am free to return even my
> custom object as a return type of this method? If so, what is the guarantee
> that it will be interpreted correctly down the pipeline?
>
> Thanks,
> --
> Swarnim
>



-- 
Swarnim