You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Ayman Farahat <ay...@yahoo.com> on 2012/04/11 17:36:19 UTC
Problem with Piggybank on local machine
Hello;
I am using pig 0.9.2
It compiles fine and runs. I can also build my own UDFs.
When i try using Piggybank i get an error :
Any help much appreciated
Pig Stack Trace
---------------
ERROR 1070: Could not resolve org.apache.pig.piggybank.evaluation.string.LOWER using imports: [, org.apache.pig.builtin., org.apache.pig.impl.builtin.]
Failed to parse: Pig script failed to parse:
<line 2, column 26> Failed to generate logical plan. Nested exception: java.lang.RuntimeException: Cannot instantiate: org.apache.pig.piggybank.evaluation.string.LOWER
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
at org.apache.pig.Main.run(Main.java:495)
at org.apache.pig.Main.main(Main.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by:
<line 2, column 26> Failed to generate logical plan. Nested exception: java.lang.RuntimeException: Cannot instantiate: org.apache.pig.piggybank.evaluation.string.LOWER
at org.apache.pig.parser.LogicalPlanBuilder.buildUDF(LogicalPlanBuilder.java:912)
________________________________
From: James Newhaven <ja...@gmail.com>
To: user@pig.apache.org
Sent: Wednesday, April 11, 2012 8:15 AM
Subject: Re: Strange behaviour when using FLATTEN
Thanks Pablo,
That link was really useful. The reason the fieldnames contain NULL, is
that I used FLATTEN with TOP, which currently discards the bagname. This is
going to be fixed in 0.11 as per
https://issues.apache.org/jira/browse/PIG-2365
On Wed, Apr 11, 2012 at 2:55 PM, pablomar
<pa...@gmail.com>wrote:
> this is a very good explanation of flatten:
>
>
> http://ofps.oreilly.com/titles/9781449302641/advanced_pig_latin.html#foreach_flatten
>
>
>
> On Wed, Apr 11, 2012 at 9:46 AM, James Newhaven <james.newhaven@gmail.com
> >wrote:
>
> > Hi,
> >
> > Sorry, this is a PIG newbie question...
> >
> > When I use FLATTEN, I don't understand the structure of the relation that
> > is returned.
> >
> > For example, the following relation A is the result of using flatten:
> >
> > DESCRIBE A;
> >
> > A {null::group: chararray,null::tagcount: long}
> >
> > I'm not really sure why there is "null" prefixing group and tagcount. Can
> > anyone help explain what this means?
> >
> > Thanks,
> > James
> >
>
Re: Extracting only the first tuple out of a bag
Posted by James Newhaven <ja...@gmail.com>.
You could also try the Pig DataFu Library FirstTupleFromBag -
FirstTupleFromBag<http://sna-projects.com/datafu/javadoc/0.0.4/datafu/pig/bags/FirstTupleFromBag.html>
James
On Thu, Apr 12, 2012 at 8:01 AM, keeyong han <ke...@hotmail.com> wrote:
>
> I have a bag consisting of tuples and I would like to extract only the
> first tuple out of bag. I don't know whether this is supposed to be hard
> but I can't think of how. Only way I can think of is to write a UDF ...
> Please help :)
> Thanks in advance!-Keeyong
Re: Extracting only the first tuple out of a bag
Posted by Gianmarco De Francisci Morales <gd...@apache.org>.
Hi,
TOP should do the trick
http://pig.apache.org/docs/r0.9.2/func.html#topx
Cheers,
--
Gianmarco
On Thu, Apr 12, 2012 at 09:01, keeyong han <ke...@hotmail.com> wrote:
>
> I have a bag consisting of tuples and I would like to extract only the
> first tuple out of bag. I don't know whether this is supposed to be hard
> but I can't think of how. Only way I can think of is to write a UDF ...
> Please help :)
> Thanks in advance!-Keeyong
Extracting only the first tuple out of a bag
Posted by keeyong han <ke...@hotmail.com>.
I have a bag consisting of tuples and I would like to extract only the first tuple out of bag. I don't know whether this is supposed to be hard but I can't think of how. Only way I can think of is to write a UDF ...
Please help :)
Thanks in advance!-Keeyong
Re: Problem with Piggybank on local machine
Posted by Ayman Farahat <ay...@yahoo.com>.
I did that; once i have grunt jar i register the piggybank.jar
Thanks
Ayman
________________________________
From: Jonathan Coveney <jc...@gmail.com>
To: Ayman Farahat <ay...@yahoo.com>
Cc: "user@pig.apache.org" <us...@pig.apache.org>
Sent: Wednesday, April 11, 2012 10:26 AM
Subject: Re: Problem with Piggybank on local machine
same thing holds. before entering the reference to the piggybank function,
you need to register the jar that contains it.
2012/4/11 Ayman Farahat <ay...@yahoo.com>
> I did this from within grunt
>
>
> ------------------------------
> *From:* Jonathan Coveney <jc...@gmail.com>
> *To:* user@pig.apache.org; Ayman Farahat <ay...@yahoo.com>
> *Sent:* Wednesday, April 11, 2012 10:17 AM
> *Subject:* Re: Problem with Piggybank on local machine
>
> add "register /path/to/piggybank.jar" to the top of your script
>
> 2012/4/11 Ayman Farahat <ay...@yahoo.com>
>
>
>
> Hello;
> I am using pig 0.9.2
> It compiles fine and runs. I can also build my own UDFs.
> When i try using Piggybank i get an error :
>
> Any help much appreciated
>
>
> Pig Stack Trace
> ---------------
> ERROR 1070: Could not resolve
> org.apache.pig.piggybank.evaluation.string.LOWER using imports: [,
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>
> Failed to parse: Pig script failed to parse:
> <line 2, column 26> Failed to generate logical plan. Nested exception:
> java.lang.RuntimeException: Cannot instantiate:
> org.apache.pig.piggybank.evaluation.string.LOWER
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at
> org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
> at
> org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
> at org.apache.pig.Main.run(Main.java:495)
> at org.apache.pig.Main.main(Main.java:111)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by:
> <line 2, column 26> Failed to generate logical plan. Nested exception:
> java.lang.RuntimeException: Cannot instantiate:
> org.apache.pig.piggybank.evaluation.string.LOWER
> at
> org.apache.pig.parser.LogicalPlanBuilder.buildUDF(LogicalPlanBuilder.java:912)
>
>
>
> ________________________________
> From: James Newhaven <ja...@gmail.com>
> To: user@pig.apache.org
> Sent: Wednesday, April 11, 2012 8:15 AM
> Subject: Re: Strange behaviour when using FLATTEN
>
> Thanks Pablo,
>
> That link was really useful. The reason the fieldnames contain NULL, is
> that I used FLATTEN with TOP, which currently discards the bagname. This is
> going to be fixed in 0.11 as per
> https://issues.apache.org/jira/browse/PIG-2365
>
>
>
> On Wed, Apr 11, 2012 at 2:55 PM, pablomar
> <pa...@gmail.com>wrote:
>
> > this is a very good explanation of flatten:
> >
> >
> >
> http://ofps.oreilly.com/titles/9781449302641/advanced_pig_latin.html#foreach_flatten
> >
> >
> >
> > On Wed, Apr 11, 2012 at 9:46 AM, James Newhaven <
> james.newhaven@gmail.com
> > >wrote:
> >
> > > Hi,
> > >
> > > Sorry, this is a PIG newbie question...
> > >
> > > When I use FLATTEN, I don't understand the structure of the relation
> that
> > > is returned.
> > >
> > > For example, the following relation A is the result of using flatten:
> > >
> > > DESCRIBE A;
> > >
> > > A {null::group: chararray,null::tagcount: long}
> > >
> > > I'm not really sure why there is "null" prefixing group and tagcount.
> Can
> > > anyone help explain what this means?
> > >
> > > Thanks,
> > > James
> > >
> >
>
>
>
>
>
Re: Problem with Piggybank on local machine
Posted by Jonathan Coveney <jc...@gmail.com>.
same thing holds. before entering the reference to the piggybank function,
you need to register the jar that contains it.
2012/4/11 Ayman Farahat <ay...@yahoo.com>
> I did this from within grunt
>
>
> ------------------------------
> *From:* Jonathan Coveney <jc...@gmail.com>
> *To:* user@pig.apache.org; Ayman Farahat <ay...@yahoo.com>
> *Sent:* Wednesday, April 11, 2012 10:17 AM
> *Subject:* Re: Problem with Piggybank on local machine
>
> add "register /path/to/piggybank.jar" to the top of your script
>
> 2012/4/11 Ayman Farahat <ay...@yahoo.com>
>
>
>
> Hello;
> I am using pig 0.9.2
> It compiles fine and runs. I can also build my own UDFs.
> When i try using Piggybank i get an error :
>
> Any help much appreciated
>
>
> Pig Stack Trace
> ---------------
> ERROR 1070: Could not resolve
> org.apache.pig.piggybank.evaluation.string.LOWER using imports: [,
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>
> Failed to parse: Pig script failed to parse:
> <line 2, column 26> Failed to generate logical plan. Nested exception:
> java.lang.RuntimeException: Cannot instantiate:
> org.apache.pig.piggybank.evaluation.string.LOWER
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at
> org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
> at
> org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
> at org.apache.pig.Main.run(Main.java:495)
> at org.apache.pig.Main.main(Main.java:111)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by:
> <line 2, column 26> Failed to generate logical plan. Nested exception:
> java.lang.RuntimeException: Cannot instantiate:
> org.apache.pig.piggybank.evaluation.string.LOWER
> at
> org.apache.pig.parser.LogicalPlanBuilder.buildUDF(LogicalPlanBuilder.java:912)
>
>
>
> ________________________________
> From: James Newhaven <ja...@gmail.com>
> To: user@pig.apache.org
> Sent: Wednesday, April 11, 2012 8:15 AM
> Subject: Re: Strange behaviour when using FLATTEN
>
> Thanks Pablo,
>
> That link was really useful. The reason the fieldnames contain NULL, is
> that I used FLATTEN with TOP, which currently discards the bagname. This is
> going to be fixed in 0.11 as per
> https://issues.apache.org/jira/browse/PIG-2365
>
>
>
> On Wed, Apr 11, 2012 at 2:55 PM, pablomar
> <pa...@gmail.com>wrote:
>
> > this is a very good explanation of flatten:
> >
> >
> >
> http://ofps.oreilly.com/titles/9781449302641/advanced_pig_latin.html#foreach_flatten
> >
> >
> >
> > On Wed, Apr 11, 2012 at 9:46 AM, James Newhaven <
> james.newhaven@gmail.com
> > >wrote:
> >
> > > Hi,
> > >
> > > Sorry, this is a PIG newbie question...
> > >
> > > When I use FLATTEN, I don't understand the structure of the relation
> that
> > > is returned.
> > >
> > > For example, the following relation A is the result of using flatten:
> > >
> > > DESCRIBE A;
> > >
> > > A {null::group: chararray,null::tagcount: long}
> > >
> > > I'm not really sure why there is "null" prefixing group and tagcount.
> Can
> > > anyone help explain what this means?
> > >
> > > Thanks,
> > > James
> > >
> >
>
>
>
>
>
Re: Problem with Piggybank on local machine
Posted by Ayman Farahat <ay...@yahoo.com>.
I did this from within grunt
________________________________
From: Jonathan Coveney <jc...@gmail.com>
To: user@pig.apache.org; Ayman Farahat <ay...@yahoo.com>
Sent: Wednesday, April 11, 2012 10:17 AM
Subject: Re: Problem with Piggybank on local machine
add "register /path/to/piggybank.jar" to the top of your script
2012/4/11 Ayman Farahat <ay...@yahoo.com>
>
>Hello;
>I am using pig 0.9.2
>It compiles fine and runs. I can also build my own UDFs.
>When i try using Piggybank i get an error :
>
>Any help much appreciated
>
>
>Pig Stack Trace
>---------------
>ERROR 1070: Could not resolve org.apache.pig.piggybank.evaluation.string.LOWER using imports: [, org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>
>Failed to parse: Pig script failed to parse:
><line 2, column 26> Failed to generate logical plan. Nested exception: java.lang.RuntimeException: Cannot instantiate: org.apache.pig.piggybank.evaluation.string.LOWER
> at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
> at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942)
> at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
> at org.apache.pig.Main.run(Main.java:495)
> at org.apache.pig.Main.main(Main.java:111)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>Caused by:
><line 2, column 26> Failed to generate logical plan. Nested exception: java.lang.RuntimeException: Cannot instantiate: org.apache.pig.piggybank.evaluation.string.LOWER
> at org.apache.pig.parser.LogicalPlanBuilder.buildUDF(LogicalPlanBuilder.java:912)
>
>
>
>________________________________
> From: James Newhaven <ja...@gmail.com>
>To: user@pig.apache.org
>Sent: Wednesday, April 11, 2012 8:15 AM
>Subject: Re: Strange behaviour when using FLATTEN
>
>Thanks Pablo,
>
>That link was really useful. The reason the fieldnames contain NULL, is
>that I used FLATTEN with TOP, which currently discards the bagname. This is
>going to be fixed in 0.11 as per
>https://issues.apache.org/jira/browse/PIG-2365
>
>
>
>On Wed, Apr 11, 2012 at 2:55 PM, pablomar
><pa...@gmail.com>wrote:
>
>> this is a very good explanation of flatten:
>>
>>
>> http://ofps.oreilly.com/titles/9781449302641/advanced_pig_latin.html#foreach_flatten
>>
>>
>>
>> On Wed, Apr 11, 2012 at 9:46 AM, James Newhaven <james.newhaven@gmail.com
>> >wrote:
>>
>> > Hi,
>> >
>> > Sorry, this is a PIG newbie question...
>> >
>> > When I use FLATTEN, I don't understand the structure of the relation that
>> > is returned.
>> >
>> > For example, the following relation A is the result of using flatten:
>> >
>> > DESCRIBE A;
>> >
>> > A {null::group: chararray,null::tagcount: long}
>> >
>> > I'm not really sure why there is "null" prefixing group and tagcount. Can
>> > anyone help explain what this means?
>> >
>> > Thanks,
>> > James
>> >
>>
Re: Problem with Piggybank on local machine
Posted by Jonathan Coveney <jc...@gmail.com>.
add "register /path/to/piggybank.jar" to the top of your script
2012/4/11 Ayman Farahat <ay...@yahoo.com>
>
>
> Hello;
> I am using pig 0.9.2
> It compiles fine and runs. I can also build my own UDFs.
> When i try using Piggybank i get an error :
>
> Any help much appreciated
>
>
> Pig Stack Trace
> ---------------
> ERROR 1070: Could not resolve
> org.apache.pig.piggybank.evaluation.string.LOWER using imports: [,
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>
> Failed to parse: Pig script failed to parse:
> <line 2, column 26> Failed to generate logical plan. Nested exception:
> java.lang.RuntimeException: Cannot instantiate:
> org.apache.pig.piggybank.evaluation.string.LOWER
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
> at
> org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
> at
> org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
> at org.apache.pig.Main.run(Main.java:495)
> at org.apache.pig.Main.main(Main.java:111)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by:
> <line 2, column 26> Failed to generate logical plan. Nested exception:
> java.lang.RuntimeException: Cannot instantiate:
> org.apache.pig.piggybank.evaluation.string.LOWER
> at
> org.apache.pig.parser.LogicalPlanBuilder.buildUDF(LogicalPlanBuilder.java:912)
>
>
>
> ________________________________
> From: James Newhaven <ja...@gmail.com>
> To: user@pig.apache.org
> Sent: Wednesday, April 11, 2012 8:15 AM
> Subject: Re: Strange behaviour when using FLATTEN
>
> Thanks Pablo,
>
> That link was really useful. The reason the fieldnames contain NULL, is
> that I used FLATTEN with TOP, which currently discards the bagname. This is
> going to be fixed in 0.11 as per
> https://issues.apache.org/jira/browse/PIG-2365
>
>
>
> On Wed, Apr 11, 2012 at 2:55 PM, pablomar
> <pa...@gmail.com>wrote:
>
> > this is a very good explanation of flatten:
> >
> >
> >
> http://ofps.oreilly.com/titles/9781449302641/advanced_pig_latin.html#foreach_flatten
> >
> >
> >
> > On Wed, Apr 11, 2012 at 9:46 AM, James Newhaven <
> james.newhaven@gmail.com
> > >wrote:
> >
> > > Hi,
> > >
> > > Sorry, this is a PIG newbie question...
> > >
> > > When I use FLATTEN, I don't understand the structure of the relation
> that
> > > is returned.
> > >
> > > For example, the following relation A is the result of using flatten:
> > >
> > > DESCRIBE A;
> > >
> > > A {null::group: chararray,null::tagcount: long}
> > >
> > > I'm not really sure why there is "null" prefixing group and tagcount.
> Can
> > > anyone help explain what this means?
> > >
> > > Thanks,
> > > James
> > >
> >