You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Daniel Eklund <do...@gmail.com> on 2011/06/03 00:56:22 UTC
jython not working in cluster mode
Does anyone have followup to this problem?
http://www.mail-archive.com/user@pig.apache.org/msg01502.html
I am getting:
Caused by: java.io.IOException: Deserialization error: could not instantiate
'org.apache.pig.scripting.jython.JythonFunction' with arguments
'[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelev
ant]'
at
org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.setup(PigMapReduce.java:299)
... 8 more
Caused by: java.lang.RuntimeException: could not instantiate
'org.apache.pig.scripting.jython.JythonFunction' with arguments
'[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelevant]'
at
org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:502)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.instantiateFunc(POUserFunc.java:109)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.readObject(POUserFunc.java:451)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
at
java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
Re: jython not working in cluster mode
Posted by Alan Gates <ga...@yahoo-inc.com>.
https://issues.apache.org/jira/browse/PIG-2101
Alan.
On Jun 6, 2011, at 6:06 PM, Mridul Muralidharan wrote:
>
> You might want to raise a JIRA on this - both abs and rel paths should
> be supported ...
>
>
> Regards,
> Mridul
>
> On Friday 03 June 2011 11:15 PM, Daniel Eklund wrote:
>> Shawn... excellent!.. thank you. it worked.
>>
>> interestingly, I remember having to use the absolute path in local
>> mode....
>>
>> daniel
>>
>> On Fri, Jun 3, 2011 at 12:01 PM, Xiaomeng Wan<sh...@gmail.com>
>> wrote:
>>
>>> Hi Daniel,
>>>
>>> We found it was the absolute path which didn't work. Try relative
>>> path
>>> when register your travelocity-udfs.py.
>>>
>>> Shawn
>>>
>>> On Thu, Jun 2, 2011 at 10:49 PM, Daniel
>>> Eklund<do...@gmail.com> wrote:
>>>> I am tracing through the code, and on the backend, the pig
>>>> implementation
>>> of
>>>> map PigBaseMap (or PigMapReduce) is deserializing the pig.mapPlan
>>>> or
>>>> pig.reducePlan.
>>>>
>>>> As a shot in the dark I installed jython.jar in the lib
>>>> directories of
>>> all
>>>> the nodes in the cluster, but nothing happened. I am using the
>>> jython-2.5.0
>>>> version.
>>>>
>>>> As I understand, the Register command should be shipping out the
>>>> udfs (in
>>> py
>>>> form) into the distributed cache on the cluster. Is this true?
>>>> Where
>>> would
>>>> I see this in HDFS?
>>>>
>>>> thanks,
>>>> daniel
>>>>
>>>> On Thu, Jun 2, 2011 at 5:56 PM, Daniel Eklund<do...@gmail.com>
>>> wrote:
>>>>
>>>>> Does anyone have followup to this problem?
>>>>> http://www.mail-archive.com/user@pig.apache.org/msg01502.html
>>>>>
>>>>> I am getting:
>>>>> Caused by: java.io.IOException: Deserialization error: could not
>>>>> instantiate 'org.apache.pig.scripting.jython.JythonFunction' with
>>> arguments
>>>>> '[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelev
>>>>> ant]'
>>>>> at
>>>>>
>>> org
>>> .apache
>>> .pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:
>>> 55)
>>>>> at
>>>>>
>>> org
>>> .apache
>>> .pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce
>>> $Reduce.setup(PigMapReduce.java:299)
>>>>> ... 8 more
>>>>> Caused by: java.lang.RuntimeException: could not instantiate
>>>>> 'org.apache.pig.scripting.jython.JythonFunction' with arguments
>>>>> '[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelevant]'
>>>>> at
>>>>>
>>> org
>>> .apache
>>> .pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:502)
>>>>> at
>>>>>
>>> org
>>> .apache
>>> .pig
>>> .backend
>>> .hadoop
>>> .executionengine
>>> .physicalLayer
>>> .expressionOperators.POUserFunc.instantiateFunc(POUserFunc.java:109)
>>>>> at
>>>>>
>>> org
>>> .apache
>>> .pig
>>> .backend
>>> .hadoop
>>> .executionengine
>>> .physicalLayer
>>> .expressionOperators.POUserFunc.readObject(POUserFunc.java:451)
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>>> Method)
>>>>> at
>>>>>
>>> sun
>>> .reflect
>>> .NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>> at
>>>>>
>>> sun
>>> .reflect
>>> .DelegatingMethodAccessorImpl
>>> .invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at
>>>>> java
>>>>> .io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
>>>>> at
>>>>> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:
>>>>> 1848)
>>>>> at
>>>>>
>>> java
>>> .io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:
>>> 1752)
>>>>>
>>>>>
>>>>
>>>
>
Re: jython not working in cluster mode
Posted by Alan Gates <ga...@yahoo-inc.com>.
https://issues.apache.org/jira/browse/PIG-2101
Alan.
On Jun 6, 2011, at 6:06 PM, Mridul Muralidharan wrote:
>
> You might want to raise a JIRA on this - both abs and rel paths should
> be supported ...
>
>
> Regards,
> Mridul
>
> On Friday 03 June 2011 11:15 PM, Daniel Eklund wrote:
>> Shawn... excellent!.. thank you. it worked.
>>
>> interestingly, I remember having to use the absolute path in local
>> mode....
>>
>> daniel
>>
>> On Fri, Jun 3, 2011 at 12:01 PM, Xiaomeng Wan<sh...@gmail.com>
>> wrote:
>>
>>> Hi Daniel,
>>>
>>> We found it was the absolute path which didn't work. Try relative
>>> path
>>> when register your travelocity-udfs.py.
>>>
>>> Shawn
>>>
>>> On Thu, Jun 2, 2011 at 10:49 PM, Daniel
>>> Eklund<do...@gmail.com> wrote:
>>>> I am tracing through the code, and on the backend, the pig
>>>> implementation
>>> of
>>>> map PigBaseMap (or PigMapReduce) is deserializing the pig.mapPlan
>>>> or
>>>> pig.reducePlan.
>>>>
>>>> As a shot in the dark I installed jython.jar in the lib
>>>> directories of
>>> all
>>>> the nodes in the cluster, but nothing happened. I am using the
>>> jython-2.5.0
>>>> version.
>>>>
>>>> As I understand, the Register command should be shipping out the
>>>> udfs (in
>>> py
>>>> form) into the distributed cache on the cluster. Is this true?
>>>> Where
>>> would
>>>> I see this in HDFS?
>>>>
>>>> thanks,
>>>> daniel
>>>>
>>>> On Thu, Jun 2, 2011 at 5:56 PM, Daniel Eklund<do...@gmail.com>
>>> wrote:
>>>>
>>>>> Does anyone have followup to this problem?
>>>>> http://www.mail-archive.com/user@pig.apache.org/msg01502.html
>>>>>
>>>>> I am getting:
>>>>> Caused by: java.io.IOException: Deserialization error: could not
>>>>> instantiate 'org.apache.pig.scripting.jython.JythonFunction' with
>>> arguments
>>>>> '[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelev
>>>>> ant]'
>>>>> at
>>>>>
>>> org
>>> .apache
>>> .pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:
>>> 55)
>>>>> at
>>>>>
>>> org
>>> .apache
>>> .pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce
>>> $Reduce.setup(PigMapReduce.java:299)
>>>>> ... 8 more
>>>>> Caused by: java.lang.RuntimeException: could not instantiate
>>>>> 'org.apache.pig.scripting.jython.JythonFunction' with arguments
>>>>> '[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelevant]'
>>>>> at
>>>>>
>>> org
>>> .apache
>>> .pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:502)
>>>>> at
>>>>>
>>> org
>>> .apache
>>> .pig
>>> .backend
>>> .hadoop
>>> .executionengine
>>> .physicalLayer
>>> .expressionOperators.POUserFunc.instantiateFunc(POUserFunc.java:109)
>>>>> at
>>>>>
>>> org
>>> .apache
>>> .pig
>>> .backend
>>> .hadoop
>>> .executionengine
>>> .physicalLayer
>>> .expressionOperators.POUserFunc.readObject(POUserFunc.java:451)
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>>>> Method)
>>>>> at
>>>>>
>>> sun
>>> .reflect
>>> .NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>> at
>>>>>
>>> sun
>>> .reflect
>>> .DelegatingMethodAccessorImpl
>>> .invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at
>>>>> java
>>>>> .io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
>>>>> at
>>>>> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:
>>>>> 1848)
>>>>> at
>>>>>
>>> java
>>> .io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:
>>> 1752)
>>>>>
>>>>>
>>>>
>>>
>
Re: jython not working in cluster mode
Posted by Mridul Muralidharan <mr...@yahoo-inc.com>.
You might want to raise a JIRA on this - both abs and rel paths should
be supported ...
Regards,
Mridul
On Friday 03 June 2011 11:15 PM, Daniel Eklund wrote:
> Shawn... excellent!.. thank you. it worked.
>
> interestingly, I remember having to use the absolute path in local mode....
>
> daniel
>
> On Fri, Jun 3, 2011 at 12:01 PM, Xiaomeng Wan<sh...@gmail.com> wrote:
>
>> Hi Daniel,
>>
>> We found it was the absolute path which didn't work. Try relative path
>> when register your travelocity-udfs.py.
>>
>> Shawn
>>
>> On Thu, Jun 2, 2011 at 10:49 PM, Daniel Eklund<do...@gmail.com> wrote:
>>> I am tracing through the code, and on the backend, the pig implementation
>> of
>>> map PigBaseMap (or PigMapReduce) is deserializing the pig.mapPlan or
>>> pig.reducePlan.
>>>
>>> As a shot in the dark I installed jython.jar in the lib directories of
>> all
>>> the nodes in the cluster, but nothing happened. I am using the
>> jython-2.5.0
>>> version.
>>>
>>> As I understand, the Register command should be shipping out the udfs (in
>> py
>>> form) into the distributed cache on the cluster. Is this true? Where
>> would
>>> I see this in HDFS?
>>>
>>> thanks,
>>> daniel
>>>
>>> On Thu, Jun 2, 2011 at 5:56 PM, Daniel Eklund<do...@gmail.com>
>> wrote:
>>>
>>>> Does anyone have followup to this problem?
>>>> http://www.mail-archive.com/user@pig.apache.org/msg01502.html
>>>>
>>>> I am getting:
>>>> Caused by: java.io.IOException: Deserialization error: could not
>>>> instantiate 'org.apache.pig.scripting.jython.JythonFunction' with
>> arguments
>>>> '[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelev
>>>> ant]'
>>>> at
>>>>
>> org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55)
>>>> at
>>>>
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.setup(PigMapReduce.java:299)
>>>> ... 8 more
>>>> Caused by: java.lang.RuntimeException: could not instantiate
>>>> 'org.apache.pig.scripting.jython.JythonFunction' with arguments
>>>> '[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelevant]'
>>>> at
>>>>
>> org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:502)
>>>> at
>>>>
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.instantiateFunc(POUserFunc.java:109)
>>>> at
>>>>
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.readObject(POUserFunc.java:451)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>> at
>>>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at
>>>> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
>>>> at
>>>> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
>>>> at
>>>>
>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
>>>>
>>>>
>>>
>>
Re: jython not working in cluster mode
Posted by Daniel Eklund <do...@gmail.com>.
Shawn... excellent!.. thank you. it worked.
interestingly, I remember having to use the absolute path in local mode....
daniel
On Fri, Jun 3, 2011 at 12:01 PM, Xiaomeng Wan <sh...@gmail.com> wrote:
> Hi Daniel,
>
> We found it was the absolute path which didn't work. Try relative path
> when register your travelocity-udfs.py.
>
> Shawn
>
> On Thu, Jun 2, 2011 at 10:49 PM, Daniel Eklund <do...@gmail.com> wrote:
> > I am tracing through the code, and on the backend, the pig implementation
> of
> > map PigBaseMap (or PigMapReduce) is deserializing the pig.mapPlan or
> > pig.reducePlan.
> >
> > As a shot in the dark I installed jython.jar in the lib directories of
> all
> > the nodes in the cluster, but nothing happened. I am using the
> jython-2.5.0
> > version.
> >
> > As I understand, the Register command should be shipping out the udfs (in
> py
> > form) into the distributed cache on the cluster. Is this true? Where
> would
> > I see this in HDFS?
> >
> > thanks,
> > daniel
> >
> > On Thu, Jun 2, 2011 at 5:56 PM, Daniel Eklund <do...@gmail.com>
> wrote:
> >
> >> Does anyone have followup to this problem?
> >> http://www.mail-archive.com/user@pig.apache.org/msg01502.html
> >>
> >> I am getting:
> >> Caused by: java.io.IOException: Deserialization error: could not
> >> instantiate 'org.apache.pig.scripting.jython.JythonFunction' with
> arguments
> >> '[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelev
> >> ant]'
> >> at
> >>
> org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55)
> >> at
> >>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.setup(PigMapReduce.java:299)
> >> ... 8 more
> >> Caused by: java.lang.RuntimeException: could not instantiate
> >> 'org.apache.pig.scripting.jython.JythonFunction' with arguments
> >> '[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelevant]'
> >> at
> >>
> org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:502)
> >> at
> >>
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.instantiateFunc(POUserFunc.java:109)
> >> at
> >>
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.readObject(POUserFunc.java:451)
> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> at
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >> at
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >> at java.lang.reflect.Method.invoke(Method.java:597)
> >> at
> >> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
> >> at
> >> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
> >> at
> >>
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
> >>
> >>
> >
>
Re: jython not working in cluster mode
Posted by Xiaomeng Wan <sh...@gmail.com>.
Hi Daniel,
We found it was the absolute path which didn't work. Try relative path
when register your travelocity-udfs.py.
Shawn
On Thu, Jun 2, 2011 at 10:49 PM, Daniel Eklund <do...@gmail.com> wrote:
> I am tracing through the code, and on the backend, the pig implementation of
> map PigBaseMap (or PigMapReduce) is deserializing the pig.mapPlan or
> pig.reducePlan.
>
> As a shot in the dark I installed jython.jar in the lib directories of all
> the nodes in the cluster, but nothing happened. I am using the jython-2.5.0
> version.
>
> As I understand, the Register command should be shipping out the udfs (in py
> form) into the distributed cache on the cluster. Is this true? Where would
> I see this in HDFS?
>
> thanks,
> daniel
>
> On Thu, Jun 2, 2011 at 5:56 PM, Daniel Eklund <do...@gmail.com> wrote:
>
>> Does anyone have followup to this problem?
>> http://www.mail-archive.com/user@pig.apache.org/msg01502.html
>>
>> I am getting:
>> Caused by: java.io.IOException: Deserialization error: could not
>> instantiate 'org.apache.pig.scripting.jython.JythonFunction' with arguments
>> '[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelev
>> ant]'
>> at
>> org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55)
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.setup(PigMapReduce.java:299)
>> ... 8 more
>> Caused by: java.lang.RuntimeException: could not instantiate
>> 'org.apache.pig.scripting.jython.JythonFunction' with arguments
>> '[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelevant]'
>> at
>> org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:502)
>> at
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.instantiateFunc(POUserFunc.java:109)
>> at
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.readObject(POUserFunc.java:451)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
>> at
>> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
>> at
>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
>>
>>
>
Re: jython not working in cluster mode
Posted by Daniel Eklund <do...@gmail.com>.
I am tracing through the code, and on the backend, the pig implementation of
map PigBaseMap (or PigMapReduce) is deserializing the pig.mapPlan or
pig.reducePlan.
As a shot in the dark I installed jython.jar in the lib directories of all
the nodes in the cluster, but nothing happened. I am using the jython-2.5.0
version.
As I understand, the Register command should be shipping out the udfs (in py
form) into the distributed cache on the cluster. Is this true? Where would
I see this in HDFS?
thanks,
daniel
On Thu, Jun 2, 2011 at 5:56 PM, Daniel Eklund <do...@gmail.com> wrote:
> Does anyone have followup to this problem?
> http://www.mail-archive.com/user@pig.apache.org/msg01502.html
>
> I am getting:
> Caused by: java.io.IOException: Deserialization error: could not
> instantiate 'org.apache.pig.scripting.jython.JythonFunction' with arguments
> '[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelev
> ant]'
> at
> org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.setup(PigMapReduce.java:299)
> ... 8 more
> Caused by: java.lang.RuntimeException: could not instantiate
> 'org.apache.pig.scripting.jython.JythonFunction' with arguments
> '[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelevant]'
> at
> org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:502)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.instantiateFunc(POUserFunc.java:109)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.readObject(POUserFunc.java:451)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
>
>
Re: jython not working in cluster mode
Posted by Daniel Eklund <do...@gmail.com>.
I can actually still get it to work with a relative directory not involving
'..'
For instance
Register 'test/simple.py' as myNamespace;
where test is a subdir in the working directory.
On Fri, Jun 3, 2011 at 1:37 AM, Alan Gates <al...@gmail.com> wrote:
> I ran into this and found that if my python script was in my working
> directory on the front end it worked. See PIG-2101.
>
> Alan.
>
> On Jun 2, 2011, at 15:56, Daniel Eklund <do...@gmail.com> wrote:
>
> > Does anyone have followup to this problem?
> > http://www.mail-archive.com/user@pig.apache.org/msg01502.html
> >
> > I am getting:
> > Caused by: java.io.IOException: Deserialization error: could not
> instantiate
> > 'org.apache.pig.scripting.jython.JythonFunction' with arguments
> > '[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelev
> > ant]'
> > at
> >
> org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55)
> > at
> >
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.setup(PigMapReduce.java:299)
> > ... 8 more
> > Caused by: java.lang.RuntimeException: could not instantiate
> > 'org.apache.pig.scripting.jython.JythonFunction' with arguments
> > '[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelevant]'
> > at
> >
> org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:502)
> > at
> >
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.instantiateFunc(POUserFunc.java:109)
> > at
> >
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.readObject(POUserFunc.java:451)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at
> > java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
> > at
> > java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
> > at
> > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)
>
Re: jython not working in cluster mode
Posted by Alan Gates <al...@gmail.com>.
I ran into this and found that if my python script was in my working directory on the front end it worked. See PIG-2101.
Alan.
On Jun 2, 2011, at 15:56, Daniel Eklund <do...@gmail.com> wrote:
> Does anyone have followup to this problem?
> http://www.mail-archive.com/user@pig.apache.org/msg01502.html
>
> I am getting:
> Caused by: java.io.IOException: Deserialization error: could not instantiate
> 'org.apache.pig.scripting.jython.JythonFunction' with arguments
> '[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelev
> ant]'
> at
> org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:55)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Reduce.setup(PigMapReduce.java:299)
> ... 8 more
> Caused by: java.lang.RuntimeException: could not instantiate
> 'org.apache.pig.scripting.jython.JythonFunction' with arguments
> '[/tank/deklund/tvpig/lib/travelocity-udfs.py, dateRelevant]'
> at
> org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:502)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.instantiateFunc(POUserFunc.java:109)
> at
> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POUserFunc.readObject(POUserFunc.java:451)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
> at
> java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1848)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1752)