You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@hive.apache.org by Kumar V <ku...@yahoo.com> on 2014/01/29 16:21:39 UTC

Returning variable number of fields from Hive UDTF

Hi all,
    I have a UDTF that returns fields from an XML/SerDe based table.  So I have to return varying number of fields from the UDTF based on the number of xpath's in the query file.  I am not sure how to access the values of the parameters in the initialize() method of the UDTF.  It only has the ObjectInspector array. The UDTF itself works fine if I always return a fixed number of fields.

Something like 

select * from myudtf(xpathsfile);

Should return as many fields as there are xpaths in the file.  Or I can also modify the UDTF to take in a parameter for the number of fields, like

select * from myudtf(5,xpathsfile);

I am kind of against a wall here. Any help is appreciated.

Regards,
Murali.

Re: Returning variable number of fields from Hive UDTF

Posted by Kumar V <ku...@yahoo.com>.

Thanks a Lot Edward.  constantobjectinspector solved my problem.  It works now.

Regards,
Murali.

On Wednesday, January 29, 2014 10:29 AM, Edward Capriolo <ed...@gmail.com> wrote:

if and only if the number is truely a constant, you can use the constant Object Inspector in initialize. The value but be a constant at query compile time. 
So this should work:
myudtf(5,xpathsfile);

THhis can not work
myudtf(colx,xpathsfile);

On Wed, Jan 29, 2014 at 10:21 AM, Kumar V <ku...@yahoo.com> wrote:

Hi all,
>    I have a UDTF that returns fields from an XML/SerDe based table.  So I have to return varying number of fields from the UDTF based on the number of xpath's in the query file.  I am not sure how to access the values of the parameters in the initialize() method of the UDTF.  It only has the ObjectInspector array. The UDTF itself works fine if I always return a fixed number of fields.
>
>
>Something like 
>
>
>select * from myudtf(xpathsfile);
>
>
>Should return as many fields as there are xpaths in the file.  Or I can also modify the UDTF to take in a parameter for the number of fields, like
>
>
>select * from myudtf(5,xpathsfile);
>
>
>I am kind of against a wall here. Any help is appreciated.
>
>
>Regards,
>Murali.

Re: Returning variable number of fields from Hive UDTF

Posted by Edward Capriolo <ed...@gmail.com>.

if and only if the number is truely a constant, you can use the constant
Object Inspector in initialize. The value but be a constant at query
compile time.
So this should work:
myudtf(5,xpathsfile);

THhis can not work
myudtf(colx,xpathsfile);


On Wed, Jan 29, 2014 at 10:21 AM, Kumar V <ku...@yahoo.com> wrote:

> Hi all,
>     I have a UDTF that returns fields from an XML/SerDe based table.  So I
> have to return varying number of fields from the UDTF based on the number
> of xpath's in the query file.  I am not sure how to access the values of
> the parameters in the initialize() method of the UDTF.  It only has the
> ObjectInspector array. The UDTF itself works fine if I always return a
> fixed number of fields.
>
> Something like
>
> select * from myudtf(xpathsfile);
>
> Should return as many fields as there are xpaths in the file.  Or I can
> also modify the UDTF to take in a parameter for the number of fields, like
>
> select * from myudtf(5,xpathsfile);
>
> I am kind of against a wall here. Any help is appreciated.
>
> Regards,
> Murali.
>