You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Chandramohan <mc...@gmail.com> on 2014/04/14 14:15:46 UTC
Fwd: Writing groovy UDFs
Could any one please provide a sample code of pig with groovy UDF, along
with the process to deplo and run it?
I tried on my own which is not working
I wrote a groovy UDF as
import org.apache.pig.builtin.OutputSchema;
class GroovyUDFs {
@OutputSchema('x:long')
long square(long x) {
return x*x;
}
}
I have written a simple pig as
--Register 'test.groovy' using groovy as myfuncs;
register 'test.groovy' using
org.apache.pig.scripting.groovy.GroovyScriptEngine as myfuncs;
--A = LOAD 'emp.csv' As(id:int, name:chararray, salary:long);
--B = FOR EACH A myfuncs.GroovyUDFs.square(A.id);
--DUMP A;
Only register command is kept uncommented and ran pig in local mode
I am getting below error
Pig Stack Trace
---------------
ERROR 2998: Unhandled internal error. groovy/util/ResourceException
java.lang.NoClassDefFoundError: groovy/util/ResourceException
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at org.apache.pig.scripting.ScriptEngine.getInstance(ScriptEngine.java:275)
at org.apache.pig.PigServer.registerCode(PigServer.java:502)
at
org.apache.pig.tools.grunt.GruntParser.processRegister(GruntParser.java:436)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:445)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
at org.apache.pig.Main.run(Main.java:604)
at org.apache.pig.Main.main(Main.java:157)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: java.lang.ClassNotFoundException: groovy.util.ResourceException
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 16 more
================================================================================
Re: Writing groovy UDFs
Posted by Chandramohan <mc...@gmail.com>.
Thanks Cheolsoo, that worked well.
Is there any sample on how to do exception handling and logging in groovy
UDFs, so that pig can know about exceptions.
On Tue, Apr 15, 2014 at 11:58 PM, Cheolsoo Park <pi...@gmail.com>wrote:
> Did you try to drop groovy-all*.jar in the Pig/lib directory?
>
> Btw, this issue should be fixed by PIG-3737-
> https://issues.apache.org/jira/browse/PIG-3737
>
>
> On Mon, Apr 14, 2014 at 5:18 AM, Chandramohan <mchandramohan@gmail.com
> >wrote:
>
> > I tried running the same with python which worked fine.
> > I also added groovy to pig_classpath, I set groovy in $PATH
> > Please find attached code sample which has python and groovy code along
> > with their pig scripts.
> >
> > Could you please help here?
> >
> >
> > On Mon, Apr 14, 2014 at 5:45 PM, Chandramohan <mchandramohan@gmail.com
> >wrote:
> >
> >> Could any one please provide a sample code of pig with groovy UDF, along
> >> with the process to deplo and run it?
> >> I tried on my own which is not working
> >>
> >> I wrote a groovy UDF as
> >>
> >> import org.apache.pig.builtin.OutputSchema;
> >>
> >> class GroovyUDFs {
> >> @OutputSchema('x:long')
> >> long square(long x) {
> >> return x*x;
> >> }
> >> }
> >>
> >> I have written a simple pig as
> >>
> >> --Register 'test.groovy' using groovy as myfuncs;
> >> register 'test.groovy' using
> >> org.apache.pig.scripting.groovy.GroovyScriptEngine as myfuncs;
> >>
> >> --A = LOAD 'emp.csv' As(id:int, name:chararray, salary:long);
> >> --B = FOR EACH A myfuncs.GroovyUDFs.square(A.id);
> >> --DUMP A;
> >>
> >>
> >> Only register command is kept uncommented and ran pig in local mode
> >> I am getting below error
> >>
> >> Pig Stack Trace
> >> ---------------
> >> ERROR 2998: Unhandled internal error. groovy/util/ResourceException
> >>
> >> java.lang.NoClassDefFoundError: groovy/util/ResourceException
> >> at java.lang.Class.forName0(Native Method)
> >> at java.lang.Class.forName(Class.java:169)
> >> at
> >> org.apache.pig.scripting.ScriptEngine.getInstance(ScriptEngine.java:275)
> >> at org.apache.pig.PigServer.registerCode(PigServer.java:502)
> >> at
> >>
> org.apache.pig.tools.grunt.GruntParser.processRegister(GruntParser.java:436)
> >> at
> >>
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:445)
> >> at
> >>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> >> at
> >>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
> >> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
> >> at org.apache.pig.Main.run(Main.java:604)
> >> at org.apache.pig.Main.main(Main.java:157)
> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> at
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >> at
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >> at java.lang.reflect.Method.invoke(Method.java:597)
> >> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> >> Caused by: java.lang.ClassNotFoundException:
> groovy.util.ResourceException
> >> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> >> at java.security.AccessController.doPrivileged(Native Method)
> >> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> >> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> >> ... 16 more
> >>
> >>
> ================================================================================
> >>
> >>
> >>
> >
>
Re: Writing groovy UDFs
Posted by Cheolsoo Park <pi...@gmail.com>.
Did you try to drop groovy-all*.jar in the Pig/lib directory?
Btw, this issue should be fixed by PIG-3737-
https://issues.apache.org/jira/browse/PIG-3737
On Mon, Apr 14, 2014 at 5:18 AM, Chandramohan <mc...@gmail.com>wrote:
> I tried running the same with python which worked fine.
> I also added groovy to pig_classpath, I set groovy in $PATH
> Please find attached code sample which has python and groovy code along
> with their pig scripts.
>
> Could you please help here?
>
>
> On Mon, Apr 14, 2014 at 5:45 PM, Chandramohan <mc...@gmail.com>wrote:
>
>> Could any one please provide a sample code of pig with groovy UDF, along
>> with the process to deplo and run it?
>> I tried on my own which is not working
>>
>> I wrote a groovy UDF as
>>
>> import org.apache.pig.builtin.OutputSchema;
>>
>> class GroovyUDFs {
>> @OutputSchema('x:long')
>> long square(long x) {
>> return x*x;
>> }
>> }
>>
>> I have written a simple pig as
>>
>> --Register 'test.groovy' using groovy as myfuncs;
>> register 'test.groovy' using
>> org.apache.pig.scripting.groovy.GroovyScriptEngine as myfuncs;
>>
>> --A = LOAD 'emp.csv' As(id:int, name:chararray, salary:long);
>> --B = FOR EACH A myfuncs.GroovyUDFs.square(A.id);
>> --DUMP A;
>>
>>
>> Only register command is kept uncommented and ran pig in local mode
>> I am getting below error
>>
>> Pig Stack Trace
>> ---------------
>> ERROR 2998: Unhandled internal error. groovy/util/ResourceException
>>
>> java.lang.NoClassDefFoundError: groovy/util/ResourceException
>> at java.lang.Class.forName0(Native Method)
>> at java.lang.Class.forName(Class.java:169)
>> at
>> org.apache.pig.scripting.ScriptEngine.getInstance(ScriptEngine.java:275)
>> at org.apache.pig.PigServer.registerCode(PigServer.java:502)
>> at
>> org.apache.pig.tools.grunt.GruntParser.processRegister(GruntParser.java:436)
>> at
>> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:445)
>> at
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
>> at
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
>> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
>> at org.apache.pig.Main.run(Main.java:604)
>> at org.apache.pig.Main.main(Main.java:157)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>> Caused by: java.lang.ClassNotFoundException: groovy.util.ResourceException
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>> ... 16 more
>>
>> ================================================================================
>>
>>
>>
>
Re: Writing groovy UDFs
Posted by Chandramohan <mc...@gmail.com>.
I tried running the same with python which worked fine.
I also added groovy to pig_classpath, I set groovy in $PATH
Please find attached code sample which has python and groovy code along
with their pig scripts.
Could you please help here?
On Mon, Apr 14, 2014 at 5:45 PM, Chandramohan <mc...@gmail.com>wrote:
> Could any one please provide a sample code of pig with groovy UDF, along
> with the process to deplo and run it?
> I tried on my own which is not working
>
> I wrote a groovy UDF as
>
> import org.apache.pig.builtin.OutputSchema;
>
> class GroovyUDFs {
> @OutputSchema('x:long')
> long square(long x) {
> return x*x;
> }
> }
>
> I have written a simple pig as
>
> --Register 'test.groovy' using groovy as myfuncs;
> register 'test.groovy' using
> org.apache.pig.scripting.groovy.GroovyScriptEngine as myfuncs;
>
> --A = LOAD 'emp.csv' As(id:int, name:chararray, salary:long);
> --B = FOR EACH A myfuncs.GroovyUDFs.square(A.id);
> --DUMP A;
>
>
> Only register command is kept uncommented and ran pig in local mode
> I am getting below error
>
> Pig Stack Trace
> ---------------
> ERROR 2998: Unhandled internal error. groovy/util/ResourceException
>
> java.lang.NoClassDefFoundError: groovy/util/ResourceException
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at org.apache.pig.scripting.ScriptEngine.getInstance(ScriptEngine.java:275)
> at org.apache.pig.PigServer.registerCode(PigServer.java:502)
> at
> org.apache.pig.tools.grunt.GruntParser.processRegister(GruntParser.java:436)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:445)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
> at org.apache.pig.Main.run(Main.java:604)
> at org.apache.pig.Main.main(Main.java:157)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> Caused by: java.lang.ClassNotFoundException: groovy.util.ResourceException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> ... 16 more
>
> ================================================================================
>
>
>