You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@pig.apache.org by Benoit Mathieu <bm...@deezer.com> on 2013/12/03 17:09:51 UTC

escape field names

Hello,

Is there any way to escape field names so that Pig do not throw an error
when field name is a reserved keyword ?

Here is my script:
A = LOAD 'logs.stream5' USING org.apache.hcatalog.pig.HCatLoader();
B = FILTER A BY lt>120;
DUMP B;

Pig throws an ERROR because "lt" is a reserved keyword.
It's annoying because I can't change this field name, and I don't really
want to use $15 because it will break on future schema changes.
Is there any magic syntax ?

Thanks for help.

Benoit

Re: escape field names

Posted by Raviteja Chirala <ra...@gmail.com>.

After 'load' can you use 'Define' to give alias? That's a guess. 

On Dec 3, 2013, at 12:22 PM, Benoit Mathieu <bm...@deezer.com> wrote:

> Aida,
> 
> Your suggestion don't work. Pig throws the same error, because lt is a
> reserved word, so you can't use it as a field name, even in a FOREACH
> statement.
> 
> I would like to write something like:
> B = FOREACH A GENERATE `lt`;
> or
> B = FOREACH A GENERATE \lt;
> 
> so that Pig will interpret lt as a field identifier, but those statement
> are incorrect. Is there any solution ?
> 
> ++
> Benoit
> 
> 
> 
> 
> Benoit Mathieu
> Lead Software Engineer, Data
> skype: bmat06
> 
> 
> 
> 2013/12/3 Aida Mashkouri Najafi <ai...@infectiousmedia.com>
> 
>> Just rename your column locally in your code.
>> 
>> You can regenerate you column names and rename them by
>> 
>> e.g.
>> 
>> FOREACH A GENERATE It as 'new_name', the rest of columns you like to call.
>> 
>> Just rename it temporary to skip the error.
>> 
>> 
>> On 3 December 2013 16:09, Benoit Mathieu <bm...@deezer.com> wrote:
>> 
>>> Hello,
>>> 
>>> Is there any way to escape field names so that Pig do not throw an error
>>> when field name is a reserved keyword ?
>>> 
>>> Here is my script:
>>> A = LOAD 'logs.stream5' USING org.apache.hcatalog.pig.HCatLoader();
>>> B = FILTER A BY lt>120;
>>> DUMP B;
>>> 
>>> Pig throws an ERROR because "lt" is a reserved keyword.
>>> It's annoying because I can't change this field name, and I don't really
>>> want to use $15 because it will break on future schema changes.
>>> Is there any magic syntax ?
>>> 
>>> Thanks for help.
>>> 
>>> Benoit
>>> 
>> 
>> 
>> 
>> --
>> Aida Mashkouri NajafiData Scientist T. +44(0) 20 7775 5625 [image:
>> Infectious Media]3-7 Herbal Hill / London / EC1R
>> 5EJwww.infectiousmedia.com [image:
>> Infectious Media] <http://www.infectiousmedia.com/> [image:
>> Facebook]<http://www.facebook.com/infectiousmedia>[image:
>> Twitter] <https://twitter.com/infectiousmedia>[image:
>> LinkedIn]<http://www.linkedin.com/company/infectious-media-ltd>[image:
>> Youtube] <http://www.youtube.com/user/InfectiousMediaLtd>    This email
>> and
>> any attachments are confidential and may also beprivileged. If you are not
>> the intended recipient, please notify the senderimmediately, and do not
>> disclose the contents to another person, use it forany purpose, or store,
>> or copy the information in any medium. Please alsodestroy and delete the
>> message from your computer.
>>

Re: escape field names

Posted by Benoit Mathieu <bm...@deezer.com>.

Aida,

Your suggestion don't work. Pig throws the same error, because lt is a
reserved word, so you can't use it as a field name, even in a FOREACH
statement.

I would like to write something like:
B = FOREACH A GENERATE `lt`;
or
B = FOREACH A GENERATE \lt;

so that Pig will interpret lt as a field identifier, but those statement
are incorrect. Is there any solution ?

++
Benoit




Benoit Mathieu
Lead Software Engineer, Data
skype: bmat06



2013/12/3 Aida Mashkouri Najafi <ai...@infectiousmedia.com>

> Just rename your column locally in your code.
>
> You can regenerate you column names and rename them by
>
> e.g.
>
> FOREACH A GENERATE It as 'new_name', the rest of columns you like to call.
>
> Just rename it temporary to skip the error.
>
>
> On 3 December 2013 16:09, Benoit Mathieu <bm...@deezer.com> wrote:
>
> > Hello,
> >
> > Is there any way to escape field names so that Pig do not throw an error
> > when field name is a reserved keyword ?
> >
> > Here is my script:
> > A = LOAD 'logs.stream5' USING org.apache.hcatalog.pig.HCatLoader();
> > B = FILTER A BY lt>120;
> > DUMP B;
> >
> > Pig throws an ERROR because "lt" is a reserved keyword.
> > It's annoying because I can't change this field name, and I don't really
> > want to use $15 because it will break on future schema changes.
> > Is there any magic syntax ?
> >
> > Thanks for help.
> >
> > Benoit
> >
>
>
>
> --
>  Aida Mashkouri NajafiData Scientist T. +44(0) 20 7775 5625 [image:
> Infectious Media]3-7 Herbal Hill / London / EC1R
> 5EJwww.infectiousmedia.com [image:
> Infectious Media] <http://www.infectiousmedia.com/> [image:
> Facebook]<http://www.facebook.com/infectiousmedia>[image:
> Twitter] <https://twitter.com/infectiousmedia>[image:
> LinkedIn]<http://www.linkedin.com/company/infectious-media-ltd>[image:
> Youtube] <http://www.youtube.com/user/InfectiousMediaLtd>    This email
> and
> any attachments are confidential and may also beprivileged. If you are not
> the intended recipient, please notify the senderimmediately, and do not
> disclose the contents to another person, use it forany purpose, or store,
> or copy the information in any medium. Please alsodestroy and delete the
> message from your computer.
>

Re: escape field names

Posted by Aida Mashkouri Najafi <ai...@infectiousmedia.com>.

Just rename your column locally in your code.

You can regenerate you column names and rename them by

e.g.

FOREACH A GENERATE It as 'new_name', the rest of columns you like to call.

Just rename it temporary to skip the error.


On 3 December 2013 16:09, Benoit Mathieu <bm...@deezer.com> wrote:

> Hello,
>
> Is there any way to escape field names so that Pig do not throw an error
> when field name is a reserved keyword ?
>
> Here is my script:
> A = LOAD 'logs.stream5' USING org.apache.hcatalog.pig.HCatLoader();
> B = FILTER A BY lt>120;
> DUMP B;
>
> Pig throws an ERROR because "lt" is a reserved keyword.
> It's annoying because I can't change this field name, and I don't really
> want to use $15 because it will break on future schema changes.
> Is there any magic syntax ?
>
> Thanks for help.
>
> Benoit
>



-- 
 Aida Mashkouri NajafiData Scientist T. +44(0) 20 7775 5625 [image:
Infectious Media]3-7 Herbal Hill / London / EC1R
5EJwww.infectiousmedia.com [image:
Infectious Media] <http://www.infectiousmedia.com/> [image:
Facebook]<http://www.facebook.com/infectiousmedia>[image:
Twitter] <https://twitter.com/infectiousmedia>[image:
LinkedIn]<http://www.linkedin.com/company/infectious-media-ltd>[image:
Youtube] <http://www.youtube.com/user/InfectiousMediaLtd>    This email and
any attachments are confidential and may also beprivileged. If you are not
the intended recipient, please notify the senderimmediately, and do not
disclose the contents to another person, use it forany purpose, or store,
or copy the information in any medium. Please alsodestroy and delete the
message from your computer.