You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by rakesh sharma <ra...@hotmail.com> on 2012/03/13 22:37:52 UTC
Creating a relation on the fly
Hi All,
I have a situation where I need to create a relation by a combination of UDF and parameter values. For example, first field will be generated by UDF UUIDGenerator, second field by parameter p1, and third field by parameter p2. I am looking some way of having a relation window as expressed below:
window = generate UUIDGenerator() as uuid, '$p1' as value1, '$p2' as value2;
Any idea how it can be done.
Thanks,Rakesh
Re: Creating a relation on the fly
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
Code samples help when debugging code :)
On Tue, Mar 13, 2012 at 5:27 PM, rakesh sharma
<ra...@hotmail.com> wrote:
>
> That is what I was doing and I got the following error:
> 2012-03-13 20:38:14,404 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during parsing. Encountered " "generate" "generate "" at line 5, column 7.Was expecting one of: "define" ... "load" ... "filter" ... "foreach" ... "order" ... "distinct" ... "cogroup" ... "join" ... "cross" ... "union" ... "group" ... "stream" ... "store" ... "mapreduce" ... "limit" ... "sample" ... <IDENTIFIER> ... "(" ... <IDENTIFIER> ...
>
>> From: billgraham@gmail.com
>> Date: Tue, 13 Mar 2012 17:19:41 -0700
>> Subject: Re: Creating a relation on the fly
>> To: user@pig.apache.org
>>
>> Are you doing this verbatim:
>>
>> window = generate UUIDGenerator() as uuid, '$p1' as value1, '$p2' as value2;
>>
>> If so that's an invalid statement. GENERATE works with FOREACH.
>>
>> What is you exact pig script and error?
>>
>> On Tue, Mar 13, 2012 at 4:51 PM, rakesh sharma
>> <ra...@hotmail.com>wrote:
>>
>> >
>> > Parameter substitution works fine. I am having issues where I want to tie
>> > all three values UDF generated and parameters into a relation.
>> >
>> > > From: billgraham@gmail.com
>> > > Date: Tue, 13 Mar 2012 16:43:25 -0700
>> > > Subject: Re: Creating a relation on the fly
>> > > To: user@pig.apache.org
>> > >
>> > > Are p1 and p2 static params for the life of the script or are they
>> > dynamic?
>> > > You could do something like this if it's the former:
>> > >
>> > > pig -p p1=foo -p p2=bar -f script.pig
>> > >
>> > > and they'd be properly inserted into $p1 and $p2 in your script.
>> > >
>> > >
>> > > On Tue, Mar 13, 2012 at 2:37 PM, rakesh sharma
>> > > <ra...@hotmail.com>wrote:
>> > >
>> > > >
>> > > > Hi All,
>> > > > I have a situation where I need to create a relation by a combination
>> > of
>> > > > UDF and parameter values. For example, first field will be generated
>> > by UDF
>> > > > UUIDGenerator, second field by parameter p1, and third field by
>> > parameter
>> > > > p2. I am looking some way of having a relation window as expressed
>> > below:
>> > > > window = generate UUIDGenerator() as uuid, '$p1' as value1, '$p2' as
>> > > > value2;
>> > > > Any idea how it can be done.
>> > > > Thanks,Rakesh
>> > >
>> > >
>> >
>
RE: Creating a relation on the fly
Posted by rakesh sharma <ra...@hotmail.com>.
That is what I was doing and I got the following error:
2012-03-13 20:38:14,404 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1000: Error during parsing. Encountered " "generate" "generate "" at line 5, column 7.Was expecting one of: "define" ... "load" ... "filter" ... "foreach" ... "order" ... "distinct" ... "cogroup" ... "join" ... "cross" ... "union" ... "group" ... "stream" ... "store" ... "mapreduce" ... "limit" ... "sample" ... <IDENTIFIER> ... "(" ... <IDENTIFIER> ...
> From: billgraham@gmail.com
> Date: Tue, 13 Mar 2012 17:19:41 -0700
> Subject: Re: Creating a relation on the fly
> To: user@pig.apache.org
>
> Are you doing this verbatim:
>
> window = generate UUIDGenerator() as uuid, '$p1' as value1, '$p2' as value2;
>
> If so that's an invalid statement. GENERATE works with FOREACH.
>
> What is you exact pig script and error?
>
> On Tue, Mar 13, 2012 at 4:51 PM, rakesh sharma
> <ra...@hotmail.com>wrote:
>
> >
> > Parameter substitution works fine. I am having issues where I want to tie
> > all three values UDF generated and parameters into a relation.
> >
> > > From: billgraham@gmail.com
> > > Date: Tue, 13 Mar 2012 16:43:25 -0700
> > > Subject: Re: Creating a relation on the fly
> > > To: user@pig.apache.org
> > >
> > > Are p1 and p2 static params for the life of the script or are they
> > dynamic?
> > > You could do something like this if it's the former:
> > >
> > > pig -p p1=foo -p p2=bar -f script.pig
> > >
> > > and they'd be properly inserted into $p1 and $p2 in your script.
> > >
> > >
> > > On Tue, Mar 13, 2012 at 2:37 PM, rakesh sharma
> > > <ra...@hotmail.com>wrote:
> > >
> > > >
> > > > Hi All,
> > > > I have a situation where I need to create a relation by a combination
> > of
> > > > UDF and parameter values. For example, first field will be generated
> > by UDF
> > > > UUIDGenerator, second field by parameter p1, and third field by
> > parameter
> > > > p2. I am looking some way of having a relation window as expressed
> > below:
> > > > window = generate UUIDGenerator() as uuid, '$p1' as value1, '$p2' as
> > > > value2;
> > > > Any idea how it can be done.
> > > > Thanks,Rakesh
> > >
> > >
> >
Re: Creating a relation on the fly
Posted by Bill Graham <bi...@gmail.com>.
Are you doing this verbatim:
window = generate UUIDGenerator() as uuid, '$p1' as value1, '$p2' as value2;
If so that's an invalid statement. GENERATE works with FOREACH.
What is you exact pig script and error?
On Tue, Mar 13, 2012 at 4:51 PM, rakesh sharma
<ra...@hotmail.com>wrote:
>
> Parameter substitution works fine. I am having issues where I want to tie
> all three values UDF generated and parameters into a relation.
>
> > From: billgraham@gmail.com
> > Date: Tue, 13 Mar 2012 16:43:25 -0700
> > Subject: Re: Creating a relation on the fly
> > To: user@pig.apache.org
> >
> > Are p1 and p2 static params for the life of the script or are they
> dynamic?
> > You could do something like this if it's the former:
> >
> > pig -p p1=foo -p p2=bar -f script.pig
> >
> > and they'd be properly inserted into $p1 and $p2 in your script.
> >
> >
> > On Tue, Mar 13, 2012 at 2:37 PM, rakesh sharma
> > <ra...@hotmail.com>wrote:
> >
> > >
> > > Hi All,
> > > I have a situation where I need to create a relation by a combination
> of
> > > UDF and parameter values. For example, first field will be generated
> by UDF
> > > UUIDGenerator, second field by parameter p1, and third field by
> parameter
> > > p2. I am looking some way of having a relation window as expressed
> below:
> > > window = generate UUIDGenerator() as uuid, '$p1' as value1, '$p2' as
> > > value2;
> > > Any idea how it can be done.
> > > Thanks,Rakesh
> >
> >
>
RE: Creating a relation on the fly
Posted by rakesh sharma <ra...@hotmail.com>.
Parameter substitution works fine. I am having issues where I want to tie all three values UDF generated and parameters into a relation.
> From: billgraham@gmail.com
> Date: Tue, 13 Mar 2012 16:43:25 -0700
> Subject: Re: Creating a relation on the fly
> To: user@pig.apache.org
>
> Are p1 and p2 static params for the life of the script or are they dynamic?
> You could do something like this if it's the former:
>
> pig -p p1=foo -p p2=bar -f script.pig
>
> and they'd be properly inserted into $p1 and $p2 in your script.
>
>
> On Tue, Mar 13, 2012 at 2:37 PM, rakesh sharma
> <ra...@hotmail.com>wrote:
>
> >
> > Hi All,
> > I have a situation where I need to create a relation by a combination of
> > UDF and parameter values. For example, first field will be generated by UDF
> > UUIDGenerator, second field by parameter p1, and third field by parameter
> > p2. I am looking some way of having a relation window as expressed below:
> > window = generate UUIDGenerator() as uuid, '$p1' as value1, '$p2' as
> > value2;
> > Any idea how it can be done.
> > Thanks,Rakesh
>
>
>
>
> --
> *Note that I'm no longer using my Yahoo! email address. Please email me at
> billgraham@gmail.com going forward.*
Re: Creating a relation on the fly
Posted by Bill Graham <bi...@gmail.com>.
Are p1 and p2 static params for the life of the script or are they dynamic?
You could do something like this if it's the former:
pig -p p1=foo -p p2=bar -f script.pig
and they'd be properly inserted into $p1 and $p2 in your script.
On Tue, Mar 13, 2012 at 2:37 PM, rakesh sharma
<ra...@hotmail.com>wrote:
>
> Hi All,
> I have a situation where I need to create a relation by a combination of
> UDF and parameter values. For example, first field will be generated by UDF
> UUIDGenerator, second field by parameter p1, and third field by parameter
> p2. I am looking some way of having a relation window as expressed below:
> window = generate UUIDGenerator() as uuid, '$p1' as value1, '$p2' as
> value2;
> Any idea how it can be done.
> Thanks,Rakesh
--
*Note that I'm no longer using my Yahoo! email address. Please email me at
billgraham@gmail.com going forward.*