You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Yang <te...@gmail.com> on 2012/11/08 00:44:17 UTC
issues with using JOIN inside a MACRO?
hadoop@ip-10-245-54-191:~/top50/new$ cat a.pig
DEFINE mymacro(blah, zoo) RETURNS foo {
x = JOIN $blah BY id, $zoo BY id;
y = JOIN x BY $blah::id, $zoo BY id;
$foo = foreach y generate x::$blah::id;
};
raw = load 'a.txt' as (id:chararray);
mymacro(raw, raw);
#############################################
with the above code, I had to refer to fields within a JOIN result by the
table name, which is passed in as a MACRO param. but it seems that pig
can't recognize this syntax:
hadoop@ip-10-245-54-191:~/top50/new$ pig -x local a.pig
2012-11-07 23:40:53,524 [main] INFO org.apache.pig.Main - Logging error
messages to: /home/hadoop/top50/new/pig_1352331653520.log
2012-11-07 23:40:53,636 [main] INFO org.apache.pig.impl.util.Utils -
Default bootup file /home/hadoop/.pigbootup not found
2012-11-07 23:40:53,739 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
Connecting to hadoop file system at: file:///
<file a.pig, line 4, column 30> Unexpected character '$'
2012-11-07 23:40:54,399 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 1200: <file a.pig, line 4, column 30> Unexpected character '$'
Details at logfile: /home/hadoop/top50/new/pig_1352331653520.log
how can I work around this?
thanks
Yang
Re: issues with using JOIN inside a MACRO?
Posted by Yang <te...@gmail.com>.
I ended up doing a manual creation of a temp var:
blah0 = foreach $blah generate *;
then substitute $blah with blah0 below
On Thu, Nov 8, 2012 at 7:36 AM, David LaBarbera <
davidlabarbera@localresponse.com> wrote:
> Have you tried using a positional reference ($0)?
>
> David
>
> On Nov 7, 2012, at 6:44 PM, Yang <te...@gmail.com> wrote:
>
> > hadoop@ip-10-245-54-191:~/top50/new$ cat a.pig
> > DEFINE mymacro(blah, zoo) RETURNS foo {
> > x = JOIN $blah BY id, $zoo BY id;
> > y = JOIN x BY $blah::id, $zoo BY id;
> > $foo = foreach y generate x::$blah::id;
> > };
> >
> >
> > raw = load 'a.txt' as (id:chararray);
> > mymacro(raw, raw);
> > #############################################
> >
> > with the above code, I had to refer to fields within a JOIN result by the
> > table name, which is passed in as a MACRO param. but it seems that pig
> > can't recognize this syntax:
> >
> > hadoop@ip-10-245-54-191:~/top50/new$ pig -x local a.pig
> > 2012-11-07 23:40:53,524 [main] INFO org.apache.pig.Main - Logging error
> > messages to: /home/hadoop/top50/new/pig_1352331653520.log
> > 2012-11-07 23:40:53,636 [main] INFO org.apache.pig.impl.util.Utils -
> > Default bootup file /home/hadoop/.pigbootup not found
> > 2012-11-07 23:40:53,739 [main] INFO
> > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> > Connecting to hadoop file system at: file:///
> > <file a.pig, line 4, column 30> Unexpected character '$'
> > 2012-11-07 23:40:54,399 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> > ERROR 1200: <file a.pig, line 4, column 30> Unexpected character '$'
> > Details at logfile: /home/hadoop/top50/new/pig_1352331653520.log
> >
> >
> > how can I work around this?
> >
> > thanks
> > Yang
>
>
Re: issues with using JOIN inside a MACRO?
Posted by David LaBarbera <da...@localresponse.com>.
Have you tried using a positional reference ($0)?
David
On Nov 7, 2012, at 6:44 PM, Yang <te...@gmail.com> wrote:
> hadoop@ip-10-245-54-191:~/top50/new$ cat a.pig
> DEFINE mymacro(blah, zoo) RETURNS foo {
> x = JOIN $blah BY id, $zoo BY id;
> y = JOIN x BY $blah::id, $zoo BY id;
> $foo = foreach y generate x::$blah::id;
> };
>
>
> raw = load 'a.txt' as (id:chararray);
> mymacro(raw, raw);
> #############################################
>
> with the above code, I had to refer to fields within a JOIN result by the
> table name, which is passed in as a MACRO param. but it seems that pig
> can't recognize this syntax:
>
> hadoop@ip-10-245-54-191:~/top50/new$ pig -x local a.pig
> 2012-11-07 23:40:53,524 [main] INFO org.apache.pig.Main - Logging error
> messages to: /home/hadoop/top50/new/pig_1352331653520.log
> 2012-11-07 23:40:53,636 [main] INFO org.apache.pig.impl.util.Utils -
> Default bootup file /home/hadoop/.pigbootup not found
> 2012-11-07 23:40:53,739 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> Connecting to hadoop file system at: file:///
> <file a.pig, line 4, column 30> Unexpected character '$'
> 2012-11-07 23:40:54,399 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 1200: <file a.pig, line 4, column 30> Unexpected character '$'
> Details at logfile: /home/hadoop/top50/new/pig_1352331653520.log
>
>
> how can I work around this?
>
> thanks
> Yang