You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Jim Donofrio <do...@gmail.com> on 2012/09/15 07:13:57 UTC
access schema defined in LOAD statement in custom LoadFunc?
Is there anyway within a LoadFunc to access the schema that a user
defines after AS in a LOAD statement? Is there some property I can
access in the UDFContext or ? pushProjection provides the schema from a
FOREACH GENERATE and getSchema seems to be only meant to read part of
the actual file or a JSON schema file off disk.
Re: access schema defined in LOAD statement in custom LoadFunc?
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
I am not sure why pushProjection doesn't solve your dilemma?
This is what we use in HBaseStorage, and ElephantBird uses in thrift
and protobuf loaders.
D
On Sun, Sep 16, 2012 at 8:11 PM, Jim Donofrio <do...@gmail.com> wrote:
> I guess a workaround could be to Base64 decode the pig.script property and
> look for AS statements
>
>
> On 09/15/2012 12:38 PM, Alan Gates wrote:
>>
>> Unfortunately, no. I agree we should add that to the LoadFunc interface.
>>
>> Alan.
>>
>> On Sep 15, 2012, at 1:13 AM, Jim Donofrio wrote:
>>
>>> Is there anyway within a LoadFunc to access the schema that a user
>>> defines after AS in a LOAD statement? Is there some property I can access in
>>> the UDFContext or ? pushProjection provides the schema from a FOREACH
>>> GENERATE and getSchema seems to be only meant to read part of the actual
>>> file or a JSON schema file off disk.
>>
>>
>
Re: access schema defined in LOAD statement in custom LoadFunc?
Posted by Jim Donofrio <do...@gmail.com>.
I guess a workaround could be to Base64 decode the pig.script property
and look for AS statements
On 09/15/2012 12:38 PM, Alan Gates wrote:
> Unfortunately, no. I agree we should add that to the LoadFunc interface.
>
> Alan.
>
> On Sep 15, 2012, at 1:13 AM, Jim Donofrio wrote:
>
>> Is there anyway within a LoadFunc to access the schema that a user defines after AS in a LOAD statement? Is there some property I can access in the UDFContext or ? pushProjection provides the schema from a FOREACH GENERATE and getSchema seems to be only meant to read part of the actual file or a JSON schema file off disk.
>
Re: access schema defined in LOAD statement in custom LoadFunc?
Posted by Alan Gates <ga...@hortonworks.com>.
Unfortunately, no. I agree we should add that to the LoadFunc interface.
Alan.
On Sep 15, 2012, at 1:13 AM, Jim Donofrio wrote:
> Is there anyway within a LoadFunc to access the schema that a user defines after AS in a LOAD statement? Is there some property I can access in the UDFContext or ? pushProjection provides the schema from a FOREACH GENERATE and getSchema seems to be only meant to read part of the actual file or a JSON schema file off disk.