You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by agateaaa <ag...@gmail.com> on 2012/11/03 08:17:04 UTC
pig 0.9.2, embedded pig in python and Script context is not set error
Hi
I am using pig 0.9.2. Trying to run embedded pig in python but get this
error
I have all hadoop lib jars, pig-0.9.2-withouthadoop.jar and
jython-2.5.2.jar in class path
CLASSPATH=
/etc/hadoop
/usr/lib/jvm/java-6-sun/lib/tools.jar
/usr/share/hadoop/lib/*.jar
/etc/pig
/etc/hadoop
/usr/share/pig/lib/jython-2.5.2.jar <--replaced this jar with jython-2.5.2
standalone as per https://issues.apache.org/jira/browse/PIG-2665
/etc/hadoop
/usr/share/pig/pig-0.9.2-withouthadoop.jar
java -cp $CLASSPATH org.python.util.jython
/usr/share/MAB/PaasApp/testembed.py
Traceback (most recent call last):
File "~/testembed.py", line 12, in <module>
P = Pig.compileFromFile("""~/testembed.pig""")
at org.apache.pig.scripting.Pig.getScriptContext(Pig.java:373)
at org.apache.pig.scripting.Pig.compile(Pig.java:158)
at org.apache.pig.scripting.Pig.compileFromFile(Pig.java:191)
at org.apache.pig.scripting.Pig.compileFromFile(Pig.java:175)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
java.io.IOException: java.io.IOException: Script context is not set
On the other hand running the same script using
pig testembed.py
runs fine
What am I missing?
Thanks in advance
- Agateaaa
Re: pig 0.9.2, embedded pig in python and Script context is not set error
Posted by agateaaa <ag...@gmail.com>.
Thanks for looking at this.
I am writing control flow framework in python (jython) to process data and
I thought it would be better to call embedded pig from python directly.
I guess I can work around to make a process call to "pig embed.py" from my
python code - but doesnt seem efficient since its spawing a new
(jython?) process.
http://pig.apache.org/docs/r0.9.1/cont.html#embed-python seems to suggest
it can be done so I was curious if its limitation of the version of pig
(0.9.2) or something I have completely missed.
Thanks
Agateaaa
On Sat, Nov 3, 2012 at 5:07 PM, Cheolsoo Park <ch...@cloudera.com> wrote:
> Hi,
>
> I am able to reproduce your error. Unfortunately, I don't think that you
> can get it working without modifying source code.
>
> The difference between running "pig embed.py" and "java
> org.python.util.jython embed.py" is that some initialization happens in the
> former while it doesn't in the latter. In particular, ScriptPigContext is
> always initialized by Pig before Pig.compile() is called:
>
> ScriptPigContext.set(pigContext, this);
>
> But if the script is executed directly via jython, this step is skipped, so
> the exception is thrown:
>
> java.io.IOException: java.io.IOException: Script context is not set
>
> Is there any reason why you run your script directly via jython not via
> pig?
>
> Thanks,
> Cheolsoo
>
>
> On Sat, Nov 3, 2012 at 12:17 AM, agateaaa <ag...@gmail.com> wrote:
>
> > Hi
> >
> > I am using pig 0.9.2. Trying to run embedded pig in python but get this
> > error
> >
> > I have all hadoop lib jars, pig-0.9.2-withouthadoop.jar and
> > jython-2.5.2.jar in class path
> >
> > CLASSPATH=
> > /etc/hadoop
> > /usr/lib/jvm/java-6-sun/lib/tools.jar
> > /usr/share/hadoop/lib/*.jar
> > /etc/pig
> > /etc/hadoop
> > /usr/share/pig/lib/jython-2.5.2.jar <--replaced this jar with
> jython-2.5.2
> > standalone as per https://issues.apache.org/jira/browse/PIG-2665
> > /etc/hadoop
> > /usr/share/pig/pig-0.9.2-withouthadoop.jar
> >
> > java -cp $CLASSPATH org.python.util.jython
> > /usr/share/MAB/PaasApp/testembed.py
> > Traceback (most recent call last):
> > File "~/testembed.py", line 12, in <module>
> > P = Pig.compileFromFile("""~/testembed.pig""")
> > at org.apache.pig.scripting.Pig.getScriptContext(Pig.java:373)
> > at org.apache.pig.scripting.Pig.compile(Pig.java:158)
> > at org.apache.pig.scripting.Pig.compileFromFile(Pig.java:191)
> > at org.apache.pig.scripting.Pig.compileFromFile(Pig.java:175)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> >
> > java.io.IOException: java.io.IOException: Script context is not set
> >
> > On the other hand running the same script using
> >
> > pig testembed.py
> >
> > runs fine
> >
> > What am I missing?
> >
> > Thanks in advance
> >
> > - Agateaaa
> >
>
Re: pig 0.9.2, embedded pig in python and Script context is not set error
Posted by Cheolsoo Park <ch...@cloudera.com>.
Hi,
I am able to reproduce your error. Unfortunately, I don't think that you
can get it working without modifying source code.
The difference between running "pig embed.py" and "java
org.python.util.jython embed.py" is that some initialization happens in the
former while it doesn't in the latter. In particular, ScriptPigContext is
always initialized by Pig before Pig.compile() is called:
ScriptPigContext.set(pigContext, this);
But if the script is executed directly via jython, this step is skipped, so
the exception is thrown:
java.io.IOException: java.io.IOException: Script context is not set
Is there any reason why you run your script directly via jython not via pig?
Thanks,
Cheolsoo
On Sat, Nov 3, 2012 at 12:17 AM, agateaaa <ag...@gmail.com> wrote:
> Hi
>
> I am using pig 0.9.2. Trying to run embedded pig in python but get this
> error
>
> I have all hadoop lib jars, pig-0.9.2-withouthadoop.jar and
> jython-2.5.2.jar in class path
>
> CLASSPATH=
> /etc/hadoop
> /usr/lib/jvm/java-6-sun/lib/tools.jar
> /usr/share/hadoop/lib/*.jar
> /etc/pig
> /etc/hadoop
> /usr/share/pig/lib/jython-2.5.2.jar <--replaced this jar with jython-2.5.2
> standalone as per https://issues.apache.org/jira/browse/PIG-2665
> /etc/hadoop
> /usr/share/pig/pig-0.9.2-withouthadoop.jar
>
> java -cp $CLASSPATH org.python.util.jython
> /usr/share/MAB/PaasApp/testembed.py
> Traceback (most recent call last):
> File "~/testembed.py", line 12, in <module>
> P = Pig.compileFromFile("""~/testembed.pig""")
> at org.apache.pig.scripting.Pig.getScriptContext(Pig.java:373)
> at org.apache.pig.scripting.Pig.compile(Pig.java:158)
> at org.apache.pig.scripting.Pig.compileFromFile(Pig.java:191)
> at org.apache.pig.scripting.Pig.compileFromFile(Pig.java:175)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
>
> java.io.IOException: java.io.IOException: Script context is not set
>
> On the other hand running the same script using
>
> pig testembed.py
>
> runs fine
>
> What am I missing?
>
> Thanks in advance
>
> - Agateaaa
>