You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Dhaval Shah <pr...@yahoo.co.in> on 2012/10/10 00:29:03 UTC
Error with Pig (CDH4.0.0)
Hi everyone.. We just upgraded to CDH4.0.0 and are seeing a very weird issue with Pig.. Everytime I try to run a LOAD command, it dies with the following exception:
ERROR 2998: Unhandled internal error. name
java.lang.NoSuchFieldError: name
at org.apache.pig.parser.QueryParserStringStream.<init>(QueryParserStringStream.java:32)
at org.apache.pig.parser.QueryParserDriver.tokenize(QueryParserDriver.java:194)
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:162)
at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:967)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
at org.apache.pig.Main.run(Main.java:495)
at org.apache.pig.Main.main(Main.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
The same script/command worked fine with CDH3U3.. Did any APIs change? Or is this a bug?
Regards,
Dhaval
Re: Error with Pig (CDH4.0.0)
Posted by Cheolsoo Park <ch...@cloudera.com>.
Typo: should NOT be visible.
On Tue, Oct 9, 2012 at 4:31 PM, Cheolsoo Park <ch...@cloudera.com> wrote:
> Please note that there was an internal change in the Pig query parser from
> 0.8 to 0.9, but this change should be visible to the end user:
> https://issues.apache.org/jira/browse/PIG-1618
>
> Given your call stack, my best guess is that the old version of pig jar is
> present in classpath and causes trouble. Can you verify that PIG_HOME
> points to where the CDH4.0.0 pig is installed, and the old pig is not
> present in classpath?
>
> Thanks,
> Cheolsoo
>
> On Tue, Oct 9, 2012 at 4:15 PM, Cheolsoo Park <ch...@cloudera.com>wrote:
>
>> Hi Dhaval,
>>
>> That certainly works for me.
>>
>> How did you upgrade CDH? Did you install it via RPMs? Did you completely
>> uninstall CDH3u3 before installing CDH4?
>>
>> It sounds to me like a CDH-upgrade issue rather than Pig issue. Can you
>> please provide steps that you took to upgrade CDH?
>>
>> Thanks,
>> Cheolsoo
>>
>> On Tue, Oct 9, 2012 at 4:02 PM, Dhaval Shah <pr...@yahoo.co.in>wrote:
>>
>>> Thanks for getting back Cheolsoo..
>>>
>>> All I am trying to do is run the pig shell/grunt and do this:
>>> p = LOAD 'file_name';
>>>
>>> It comes back with the exception mentioned below..
>>>
>>> Regards,
>>> Dhaval
>>>
>>>
>>> ----- Original Message -----
>>> From: Cheolsoo Park <ch...@cloudera.com>
>>> To: user@pig.apache.org; Dhaval Shah <pr...@yahoo.co.in>
>>> Cc:
>>> Sent: Tuesday, 9 October 2012 6:57 PM
>>> Subject: Re: Error with Pig (CDH4.0.0)
>>>
>>> Hi Dhaval,
>>>
>>> CDH3u3 includes Pig 0.8, and CDH4.0.0 includes Pig 0.9. There were
>>> some incompatibilities introduced between two version.
>>>
>>>
>>> https://cwiki.apache.org/confluence/display/PIG/Pig+0.9+Backward+Compatibility
>>>
>>> To pin down the exact cause, I'd like to reproduce your error. Would
>>> mind providing an example script that generates the exception?
>>>
>>> Thanks,
>>> Cheolsoo
>>>
>>> On Tue, Oct 9, 2012 at 3:29 PM, Dhaval Shah <prince_mithibai@yahoo.co.in
>>> >wrote:
>>>
>>> > Hi everyone.. We just upgraded to CDH4.0.0 and are seeing a very weird
>>> > issue with Pig.. Everytime I try to run a LOAD command, it dies with
>>> the
>>> > following exception:
>>> >
>>> > ERROR 2998: Unhandled internal error. name
>>> >
>>> > java.lang.NoSuchFieldError: name
>>> > at
>>> >
>>> org.apache.pig.parser.QueryParserStringStream.<init>(QueryParserStringStream.java:32)
>>> > at
>>> >
>>> org.apache.pig.parser.QueryParserDriver.tokenize(QueryParserDriver.java:194)
>>> > at
>>> >
>>> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:162)
>>> > at
>>> > org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
>>> > at
>>> > org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
>>> > at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
>>> > at
>>> > org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:967)
>>> > at
>>> >
>>> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
>>> > at
>>> >
>>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
>>> > at
>>> >
>>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>>> > at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
>>> > at org.apache.pig.Main.run(Main.java:495)
>>> > at org.apache.pig.Main.main(Main.java:111)
>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> > at
>>> >
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> > at
>>> >
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> > at java.lang.reflect.Method.invoke(Method.java:597)
>>> > at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>>> >
>>> > The same script/command worked fine with CDH3U3.. Did any APIs change?
>>> Or
>>> > is this a bug?
>>> > Regards,
>>> > Dhaval
>>> >
>>>
>>>
>>
>
Re: Error with Pig (CDH4.0.0)
Posted by Cheolsoo Park <ch...@cloudera.com>.
Please note that there was an internal change in the Pig query parser from
0.8 to 0.9, but this change should be visible to the end user:
https://issues.apache.org/jira/browse/PIG-1618
Given your call stack, my best guess is that the old version of pig jar is
present in classpath and causes trouble. Can you verify that PIG_HOME
points to where the CDH4.0.0 pig is installed, and the old pig is not
present in classpath?
Thanks,
Cheolsoo
On Tue, Oct 9, 2012 at 4:15 PM, Cheolsoo Park <ch...@cloudera.com> wrote:
> Hi Dhaval,
>
> That certainly works for me.
>
> How did you upgrade CDH? Did you install it via RPMs? Did you completely
> uninstall CDH3u3 before installing CDH4?
>
> It sounds to me like a CDH-upgrade issue rather than Pig issue. Can you
> please provide steps that you took to upgrade CDH?
>
> Thanks,
> Cheolsoo
>
> On Tue, Oct 9, 2012 at 4:02 PM, Dhaval Shah <pr...@yahoo.co.in>wrote:
>
>> Thanks for getting back Cheolsoo..
>>
>> All I am trying to do is run the pig shell/grunt and do this:
>> p = LOAD 'file_name';
>>
>> It comes back with the exception mentioned below..
>>
>> Regards,
>> Dhaval
>>
>>
>> ----- Original Message -----
>> From: Cheolsoo Park <ch...@cloudera.com>
>> To: user@pig.apache.org; Dhaval Shah <pr...@yahoo.co.in>
>> Cc:
>> Sent: Tuesday, 9 October 2012 6:57 PM
>> Subject: Re: Error with Pig (CDH4.0.0)
>>
>> Hi Dhaval,
>>
>> CDH3u3 includes Pig 0.8, and CDH4.0.0 includes Pig 0.9. There were
>> some incompatibilities introduced between two version.
>>
>>
>> https://cwiki.apache.org/confluence/display/PIG/Pig+0.9+Backward+Compatibility
>>
>> To pin down the exact cause, I'd like to reproduce your error. Would
>> mind providing an example script that generates the exception?
>>
>> Thanks,
>> Cheolsoo
>>
>> On Tue, Oct 9, 2012 at 3:29 PM, Dhaval Shah <prince_mithibai@yahoo.co.in
>> >wrote:
>>
>> > Hi everyone.. We just upgraded to CDH4.0.0 and are seeing a very weird
>> > issue with Pig.. Everytime I try to run a LOAD command, it dies with the
>> > following exception:
>> >
>> > ERROR 2998: Unhandled internal error. name
>> >
>> > java.lang.NoSuchFieldError: name
>> > at
>> >
>> org.apache.pig.parser.QueryParserStringStream.<init>(QueryParserStringStream.java:32)
>> > at
>> >
>> org.apache.pig.parser.QueryParserDriver.tokenize(QueryParserDriver.java:194)
>> > at
>> >
>> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:162)
>> > at
>> > org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
>> > at
>> > org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
>> > at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
>> > at
>> > org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:967)
>> > at
>> >
>> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
>> > at
>> >
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
>> > at
>> >
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>> > at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
>> > at org.apache.pig.Main.run(Main.java:495)
>> > at org.apache.pig.Main.main(Main.java:111)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > at
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> > at
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> > at java.lang.reflect.Method.invoke(Method.java:597)
>> > at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>> >
>> > The same script/command worked fine with CDH3U3.. Did any APIs change?
>> Or
>> > is this a bug?
>> > Regards,
>> > Dhaval
>> >
>>
>>
>
Re: Error with Pig (CDH4.0.0)
Posted by Russell Jurney <ru...@gmail.com>.
Cheolsoo, these questions aren't spam. Pig lives in different forests
:) I've just always wondered why more CDH pig users don't upgrade via
download, so I finally asked. Your answer makes sense.
Everyone loves BBQ, pork for everyone! :D
Russell Jurney http://datasyndrome.com
On Oct 9, 2012, at 4:50 PM, Cheolsoo Park <ch...@cloudera.com> wrote:
> Hi Russell,
>
> I am sorry if this was a spam to other Pig users. I will try to redirect
> CDH-specific questions to cdh-user if that's preferred. Please let me know.
>
> Regarding your question, I guess that the main complexity with upgrading
> from CDH3 to CDH4 come from that CDH4 includes both MR1 and MR2. We provide
> RPMs so that many details can be automatically taken care of, but it can
> sometimes cause subtle problems if the instructions on the Cloudera web
> page are not followed depending on environment.
>
> Thanks,
> Cheolsoo
>
> On Tue, Oct 9, 2012 at 4:34 PM, Russell Jurney <ru...@gmail.com>wrote:
>
>> I've often wondered - as we see a lot of these legacy pig issues in
>> CDH on this list - is it hard to upgrade Pig on CDH by downloading the
>> latest stable release of pig, unpacking the .tgz and running it? Is
>> upgrading Pig on CDH as simple as wget, like on Apache Hadoop and
>> others, or is it somehow more complex? Pig is after all a client-side
>> tool.
>>
>> Russell Jurney http://datasyndrome.com
>>
>> On Oct 9, 2012, at 4:15 PM, Cheolsoo Park <ch...@cloudera.com> wrote:
>>
>>> Hi Dhaval,
>>>
>>> That certainly works for me.
>>>
>>> How did you upgrade CDH? Did you install it via RPMs? Did you completely
>>> uninstall CDH3u3 before installing CDH4?
>>>
>>> It sounds to me like a CDH-upgrade issue rather than Pig issue. Can you
>>> please provide steps that you took to upgrade CDH?
>>>
>>> Thanks,
>>> Cheolsoo
>>>
>>> On Tue, Oct 9, 2012 at 4:02 PM, Dhaval Shah <prince_mithibai@yahoo.co.in
>>> wrote:
>>>
>>>> Thanks for getting back Cheolsoo..
>>>>
>>>> All I am trying to do is run the pig shell/grunt and do this:
>>>> p = LOAD 'file_name';
>>>>
>>>> It comes back with the exception mentioned below..
>>>>
>>>> Regards,
>>>> Dhaval
>>>>
>>>>
>>>> ----- Original Message -----
>>>> From: Cheolsoo Park <ch...@cloudera.com>
>>>> To: user@pig.apache.org; Dhaval Shah <pr...@yahoo.co.in>
>>>> Cc:
>>>> Sent: Tuesday, 9 October 2012 6:57 PM
>>>> Subject: Re: Error with Pig (CDH4.0.0)
>>>>
>>>> Hi Dhaval,
>>>>
>>>> CDH3u3 includes Pig 0.8, and CDH4.0.0 includes Pig 0.9. There were
>>>> some incompatibilities introduced between two version.
>>>>
>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/PIG/Pig+0.9+Backward+Compatibility
>>>>
>>>> To pin down the exact cause, I'd like to reproduce your error. Would
>>>> mind providing an example script that generates the exception?
>>>>
>>>> Thanks,
>>>> Cheolsoo
>>>>
>>>> On Tue, Oct 9, 2012 at 3:29 PM, Dhaval Shah <
>> prince_mithibai@yahoo.co.in
>>>>> wrote:
>>>>
>>>>> Hi everyone.. We just upgraded to CDH4.0.0 and are seeing a very weird
>>>>> issue with Pig.. Everytime I try to run a LOAD command, it dies with
>> the
>>>>> following exception:
>>>>>
>>>>> ERROR 2998: Unhandled internal error. name
>>>>>
>>>>> java.lang.NoSuchFieldError: name
>>>>> at
>>>>>
>>>>
>> org.apache.pig.parser.QueryParserStringStream.<init>(QueryParserStringStream.java:32)
>>>>> at
>>>>>
>>>>
>> org.apache.pig.parser.QueryParserDriver.tokenize(QueryParserDriver.java:194)
>>>>> at
>>>>>
>> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:162)
>>>>> at
>>>>> org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
>>>>> at
>>>>> org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
>>>>> at
>>>>> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:967)
>>>>> at
>>>>>
>>>>
>> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
>>>>> at
>>>>>
>>>>
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
>>>>> at
>>>>>
>>>>
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>>>>> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
>>>>> at org.apache.pig.Main.run(Main.java:495)
>>>>> at org.apache.pig.Main.main(Main.java:111)
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at
>>>>>
>>>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>> at
>>>>>
>>>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>>>>>
>>>>> The same script/command worked fine with CDH3U3.. Did any APIs change?
>> Or
>>>>> is this a bug?
>>>>> Regards,
>>>>> Dhaval
>>>>>
>>>>
>>>>
>>
Re: Error with Pig (CDH4.0.0)
Posted by Cheolsoo Park <ch...@cloudera.com>.
Hi Russell,
I am sorry if this was a spam to other Pig users. I will try to redirect
CDH-specific questions to cdh-user if that's preferred. Please let me know.
Regarding your question, I guess that the main complexity with upgrading
from CDH3 to CDH4 come from that CDH4 includes both MR1 and MR2. We provide
RPMs so that many details can be automatically taken care of, but it can
sometimes cause subtle problems if the instructions on the Cloudera web
page are not followed depending on environment.
Thanks,
Cheolsoo
On Tue, Oct 9, 2012 at 4:34 PM, Russell Jurney <ru...@gmail.com>wrote:
> I've often wondered - as we see a lot of these legacy pig issues in
> CDH on this list - is it hard to upgrade Pig on CDH by downloading the
> latest stable release of pig, unpacking the .tgz and running it? Is
> upgrading Pig on CDH as simple as wget, like on Apache Hadoop and
> others, or is it somehow more complex? Pig is after all a client-side
> tool.
>
> Russell Jurney http://datasyndrome.com
>
> On Oct 9, 2012, at 4:15 PM, Cheolsoo Park <ch...@cloudera.com> wrote:
>
> > Hi Dhaval,
> >
> > That certainly works for me.
> >
> > How did you upgrade CDH? Did you install it via RPMs? Did you completely
> > uninstall CDH3u3 before installing CDH4?
> >
> > It sounds to me like a CDH-upgrade issue rather than Pig issue. Can you
> > please provide steps that you took to upgrade CDH?
> >
> > Thanks,
> > Cheolsoo
> >
> > On Tue, Oct 9, 2012 at 4:02 PM, Dhaval Shah <prince_mithibai@yahoo.co.in
> >wrote:
> >
> >> Thanks for getting back Cheolsoo..
> >>
> >> All I am trying to do is run the pig shell/grunt and do this:
> >> p = LOAD 'file_name';
> >>
> >> It comes back with the exception mentioned below..
> >>
> >> Regards,
> >> Dhaval
> >>
> >>
> >> ----- Original Message -----
> >> From: Cheolsoo Park <ch...@cloudera.com>
> >> To: user@pig.apache.org; Dhaval Shah <pr...@yahoo.co.in>
> >> Cc:
> >> Sent: Tuesday, 9 October 2012 6:57 PM
> >> Subject: Re: Error with Pig (CDH4.0.0)
> >>
> >> Hi Dhaval,
> >>
> >> CDH3u3 includes Pig 0.8, and CDH4.0.0 includes Pig 0.9. There were
> >> some incompatibilities introduced between two version.
> >>
> >>
> >>
> https://cwiki.apache.org/confluence/display/PIG/Pig+0.9+Backward+Compatibility
> >>
> >> To pin down the exact cause, I'd like to reproduce your error. Would
> >> mind providing an example script that generates the exception?
> >>
> >> Thanks,
> >> Cheolsoo
> >>
> >> On Tue, Oct 9, 2012 at 3:29 PM, Dhaval Shah <
> prince_mithibai@yahoo.co.in
> >>> wrote:
> >>
> >>> Hi everyone.. We just upgraded to CDH4.0.0 and are seeing a very weird
> >>> issue with Pig.. Everytime I try to run a LOAD command, it dies with
> the
> >>> following exception:
> >>>
> >>> ERROR 2998: Unhandled internal error. name
> >>>
> >>> java.lang.NoSuchFieldError: name
> >>> at
> >>>
> >>
> org.apache.pig.parser.QueryParserStringStream.<init>(QueryParserStringStream.java:32)
> >>> at
> >>>
> >>
> org.apache.pig.parser.QueryParserDriver.tokenize(QueryParserDriver.java:194)
> >>> at
> >>>
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:162)
> >>> at
> >>> org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
> >>> at
> >>> org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
> >>> at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
> >>> at
> >>> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:967)
> >>> at
> >>>
> >>
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
> >>> at
> >>>
> >>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
> >>> at
> >>>
> >>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> >>> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
> >>> at org.apache.pig.Main.run(Main.java:495)
> >>> at org.apache.pig.Main.main(Main.java:111)
> >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >>> at
> >>>
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >>> at
> >>>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>> at java.lang.reflect.Method.invoke(Method.java:597)
> >>> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> >>>
> >>> The same script/command worked fine with CDH3U3.. Did any APIs change?
> Or
> >>> is this a bug?
> >>> Regards,
> >>> Dhaval
> >>>
> >>
> >>
>
Re: Error with Pig (CDH4.0.0)
Posted by Dhaval Shah <pr...@yahoo.co.in>.
Reply inline!
Regards,
Dhaval
----- Original Message -----
From: Cheolsoo Park <ch...@cloudera.com>
To: user@pig.apache.org; Dhaval Shah <pr...@yahoo.co.in>
Cc:
Sent: Wednesday, 10 October 2012 4:55 PM
Subject: Re: Error with Pig (CDH4.0.0)
Hi Dhaval,
Thank you very much for sharing your analysis! Your explanation definitely
provides more insights. :-)
If you don't mind, I'd like to clarify a couple of things. I am just trying
to see if there's something to be fixed in terms of Pig packaging:
>> the root issue was that the pig libraries needed to be present in the
HADOOP_CLASSPATH and it needs to be specifically set in hadoop_env.sh..
I am not sure if missing dependency libraries is the root cause. Given the
following error in your call stack, I believe that you have a different
version of antlr in classpath.
---Dhaval -> Yes I did have Antlr 2.7.7 and 3.0.1 in the classpath from Mahout 0.6.. However, I still do have it in my classpath (and it appears earlier in the classpath than Pig) and it still works for me.. Not sure what the actual issue is but something to think about.. Also, I saw that pig jars have antlr bundled.. So technically those should take precedence anyways unless its being called in a different way..
java.lang.NoSuchFieldError: name
> at org.apache.pig.parser.QueryParserStringStream.<init>
> (QueryParserStringStream.java:32)
If you look at QueryParserStringStream.java, it extends ANTLRStringStream,
and Pig 0.9 is compiled against antlr 3.4. Now if antlr is missing, you
should get ClassNotFoundError not NoSuchFiledError. Only possible reason
that I can think of is that there is a different version of antlr in
classpath at run-time.
In fact, there was a similar discussion on this mailing list a while ago
(Please note that he is reporting the same call stack as yours):
http://mail-archives.apache.org/mod_mbox/pig-user/201206.mbox/%3CCAOkn+GjPCgY1i+-zfNWF9cKz+m8feRj=6Rk1shSS293oqECbMg@mail.gmail.com%3E
Antlr is a common tool, so it's possible that it's installed by other tools
such as Hive.
>> I tried setting PIG_CLASSPATH and HADOOP_CLASSPATH to have those
libraries in the shell and then start pig but that did not help either..
Assuming that a wrong version of antlr is present in classpath, this makes
perfect sense because PIG_CLASSPATH adds libraries to the end of CLASPATH:
---Dhaval --> I think PIG_CLASSPATH libraries should be added to the front to take precedence since we are calling pig and expressing that we want to use pig and not anything else
# add user-specified CLASSPATH
if [ "$PIG_CLASSPATH" != "" ]; then
CLASSPATH=${CLASSPATH}:${PIG_CLASSPATH}
fi
That is, as long as the wrong antlr is present before the correct one, the
wrong one be picked up at run-time.
>> 2. The error message is not super helpful.. If libraries are missing,
the pig shell/grunt should not open at all.. However, in my case, it did
start up and then the error message was in no ways intuitive or pointing to
the root issue..
I agree with you that the error message is not very intuitive here. But
errors caused by dependency libraries can be only caught when Pig makes
calls to methods of those libraries at run-time. Furthermore, if there are
two different versions of the same library in classpath, the root cause can
be more subtle.
The challenge is that a Hadoop distribution such as CDH bundles many
sub-projects, and they often depend on different versions of the same
libraries. We do our best to harmonize all the versions of dependency
libraries across the platform, but I admit that it is not always perfect.
---Dhaval --> True I understand that issue and I guess the only thing we could do is update the pig executable here to exclude the known conflicts which in my opinion is a reasonable thing to do
Thanks,
Cheolsoo
On Wed, Oct 10, 2012 at 7:57 AM, Dhaval Shah <pr...@yahoo.co.in>wrote:
> Alright I was eventually able to get the issue resolved.. For everyone's
> benefit, the root issue was that the pig libraries needed to be present in
> the HADOOP_CLASSPATH and it needs to be specifically set in hadoop_env.sh..
> I tried setting PIG_CLASSPATH and HADOOP_CLASSPATH to have those libraries
> in the shell and then start pig but that did not help either.. Setting that
> explicitly in hadoop_env.sh did the trick..
>
> As I see there are 2 issues here:
> 1. Somehow the pig executable seems broken.. As in setting
> HADOOP_CLASSPATH on the shell and then running pig should have the same
> effect as setting it in hadoop_env.sh.. Also having a * in the
> PIG_CLASSPATH/HADOOP_CLASSPATH (when set in the shell) makes the pig
> executable set HADOOP_CLASSPATH to empty before it actually runs the hadoop
> jar command inside the pig script.. In CDH3, I never needed to set the
> classpath to include pig libraries.. The pig executable did that for
> me..(and I am never overriding HADOOP_CLASSPATH.. I only append to it)
> 2. The error message is not super helpful.. If libraries are missing, the
> pig shell/grunt should not open at all.. However, in my case, it did start
> up and then the error message was in no ways intuitive or pointing to the
> root issue..
>
> Also, I don't think this is a legacy pig issue as we tried the newer
> version 0.10 and it came up with the same issue.. A second issue that did
> pop up after putting the pig libraries in the classpath was with mapreduce
> framework being selected as classic and it turned out that the MRv1
> libraries needed to be explicitly specified in the HADOOP_CLASSPATH
> (without that entry Sqoop, Java MR jobs work fine but Pig did not for
> whatever reason)
>
> Anyways thanks everyone for the help
>
> Regards,
> Dhaval
>
>
> ----- Original Message -----
> From: Russell Jurney <ru...@gmail.com>
> To: "user@pig.apache.org" <us...@pig.apache.org>
> Cc:
> Sent: Tuesday, 9 October 2012 7:34 PM
> Subject: Re: Error with Pig (CDH4.0.0)
>
> I've often wondered - as we see a lot of these legacy pig issues in
> CDH on this list - is it hard to upgrade Pig on CDH by downloading the
> latest stable release of pig, unpacking the .tgz and running it? Is
> upgrading Pig on CDH as simple as wget, like on Apache Hadoop and
> others, or is it somehow more complex? Pig is after all a client-side
> tool.
>
> Russell Jurney http://datasyndrome.com
>
> On Oct 9, 2012, at 4:15 PM, Cheolsoo Park <ch...@cloudera.com> wrote:
>
> > Hi Dhaval,
> >
> > That certainly works for me.
> >
> > How did you upgrade CDH? Did you install it via RPMs? Did you completely
> > uninstall CDH3u3 before installing CDH4?
> >
> > It sounds to me like a CDH-upgrade issue rather than Pig issue. Can you
> > please provide steps that you took to upgrade CDH?
> >
> > Thanks,
> > Cheolsoo
> >
> > On Tue, Oct 9, 2012 at 4:02 PM, Dhaval Shah <prince_mithibai@yahoo.co.in
> >wrote:
> >
> >> Thanks for getting back Cheolsoo..
> >>
> >> All I am trying to do is run the pig shell/grunt and do this:
> >> p = LOAD 'file_name';
> >>
> >> It comes back with the exception mentioned below..
> >>
> >> Regards,
> >> Dhaval
> >>
> >>
> >> ----- Original Message -----
> >> From: Cheolsoo Park <ch...@cloudera.com>
> >> To: user@pig.apache.org; Dhaval Shah <pr...@yahoo.co.in>
> >> Cc:
> >> Sent: Tuesday, 9 October 2012 6:57 PM
> >> Subject: Re: Error with Pig (CDH4.0.0)
> >>
> >> Hi Dhaval,
> >>
> >> CDH3u3 includes Pig 0.8, and CDH4.0.0 includes Pig 0.9. There were
> >> some incompatibilities introduced between two version.
> >>
> >>
> >>
> https://cwiki.apache.org/confluence/display/PIG/Pig+0.9+Backward+Compatibility
> >>
> >> To pin down the exact cause, I'd like to reproduce your error. Would
> >> mind providing an example script that generates the exception?
> >>
> >> Thanks,
> >> Cheolsoo
> >>
> >> On Tue, Oct 9, 2012 at 3:29 PM, Dhaval Shah <
> prince_mithibai@yahoo.co.in
> >>> wrote:
> >>
> >>> Hi everyone.. We just upgraded to CDH4.0.0 and are seeing a very weird
> >>> issue with Pig.. Everytime I try to run a LOAD command, it dies with
> the
> >>> following exception:
> >>>
> >>> ERROR 2998: Unhandled internal error. name
> >>>
> >>> java.lang.NoSuchFieldError: name
> >>> at
> >>>
> >>
> org.apache.pig.parser.QueryParserStringStream.<init>(QueryParserStringStream.java:32)
> >>> at
> >>>
> >>
> org.apache.pig.parser.QueryParserDriver.tokenize(QueryParserDriver.java:194)
> >>> at
> >>>
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:162)
> >>> at
> >>> org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
> >>> at
> >>> org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
> >>> at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
> >>> at
> >>> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:967)
> >>> at
> >>>
> >>
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
> >>> at
> >>>
> >>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
> >>> at
> >>>
> >>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> >>> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
> >>> at org.apache.pig.Main.run(Main.java:495)
> >>> at org.apache.pig.Main.main(Main.java:111)
> >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >>> at
> >>>
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >>> at
> >>>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>> at java.lang.reflect.Method.invoke(Method.java:597)
> >>> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> >>>
> >>> The same script/command worked fine with CDH3U3.. Did any APIs change?
> Or
> >>> is this a bug?
> >>> Regards,
> >>> Dhaval
> >>>
> >>
> >>
>
>
Re: Error with Pig (CDH4.0.0)
Posted by Cheolsoo Park <ch...@cloudera.com>.
Hi Dhaval,
Thank you very much for sharing your analysis! Your explanation definitely
provides more insights. :-)
If you don't mind, I'd like to clarify a couple of things. I am just trying
to see if there's something to be fixed in terms of Pig packaging:
>> the root issue was that the pig libraries needed to be present in the
HADOOP_CLASSPATH and it needs to be specifically set in hadoop_env.sh..
I am not sure if missing dependency libraries is the root cause. Given the
following error in your call stack, I believe that you have a different
version of antlr in classpath.
java.lang.NoSuchFieldError: name
> at org.apache.pig.parser.QueryParserStringStream.<init>
> (QueryParserStringStream.java:32)
If you look at QueryParserStringStream.java, it extends ANTLRStringStream,
and Pig 0.9 is compiled against antlr 3.4. Now if antlr is missing, you
should get ClassNotFoundError not NoSuchFiledError. Only possible reason
that I can think of is that there is a different version of antlr in
classpath at run-time.
In fact, there was a similar discussion on this mailing list a while ago
(Please note that he is reporting the same call stack as yours):
http://mail-archives.apache.org/mod_mbox/pig-user/201206.mbox/%3CCAOkn+GjPCgY1i+-zfNWF9cKz+m8feRj=6Rk1shSS293oqECbMg@mail.gmail.com%3E
Antlr is a common tool, so it's possible that it's installed by other tools
such as Hive.
>> I tried setting PIG_CLASSPATH and HADOOP_CLASSPATH to have those
libraries in the shell and then start pig but that did not help either..
Assuming that a wrong version of antlr is present in classpath, this makes
perfect sense because PIG_CLASSPATH adds libraries to the end of CLASPATH:
# add user-specified CLASSPATH
if [ "$PIG_CLASSPATH" != "" ]; then
CLASSPATH=${CLASSPATH}:${PIG_CLASSPATH}
fi
That is, as long as the wrong antlr is present before the correct one, the
wrong one be picked up at run-time.
>> 2. The error message is not super helpful.. If libraries are missing,
the pig shell/grunt should not open at all.. However, in my case, it did
start up and then the error message was in no ways intuitive or pointing to
the root issue..
I agree with you that the error message is not very intuitive here. But
errors caused by dependency libraries can be only caught when Pig makes
calls to methods of those libraries at run-time. Furthermore, if there are
two different versions of the same library in classpath, the root cause can
be more subtle.
The challenge is that a Hadoop distribution such as CDH bundles many
sub-projects, and they often depend on different versions of the same
libraries. We do our best to harmonize all the versions of dependency
libraries across the platform, but I admit that it is not always perfect.
Thanks,
Cheolsoo
On Wed, Oct 10, 2012 at 7:57 AM, Dhaval Shah <pr...@yahoo.co.in>wrote:
> Alright I was eventually able to get the issue resolved.. For everyone's
> benefit, the root issue was that the pig libraries needed to be present in
> the HADOOP_CLASSPATH and it needs to be specifically set in hadoop_env.sh..
> I tried setting PIG_CLASSPATH and HADOOP_CLASSPATH to have those libraries
> in the shell and then start pig but that did not help either.. Setting that
> explicitly in hadoop_env.sh did the trick..
>
> As I see there are 2 issues here:
> 1. Somehow the pig executable seems broken.. As in setting
> HADOOP_CLASSPATH on the shell and then running pig should have the same
> effect as setting it in hadoop_env.sh.. Also having a * in the
> PIG_CLASSPATH/HADOOP_CLASSPATH (when set in the shell) makes the pig
> executable set HADOOP_CLASSPATH to empty before it actually runs the hadoop
> jar command inside the pig script.. In CDH3, I never needed to set the
> classpath to include pig libraries.. The pig executable did that for
> me..(and I am never overriding HADOOP_CLASSPATH.. I only append to it)
> 2. The error message is not super helpful.. If libraries are missing, the
> pig shell/grunt should not open at all.. However, in my case, it did start
> up and then the error message was in no ways intuitive or pointing to the
> root issue..
>
> Also, I don't think this is a legacy pig issue as we tried the newer
> version 0.10 and it came up with the same issue.. A second issue that did
> pop up after putting the pig libraries in the classpath was with mapreduce
> framework being selected as classic and it turned out that the MRv1
> libraries needed to be explicitly specified in the HADOOP_CLASSPATH
> (without that entry Sqoop, Java MR jobs work fine but Pig did not for
> whatever reason)
>
> Anyways thanks everyone for the help
>
> Regards,
> Dhaval
>
>
> ----- Original Message -----
> From: Russell Jurney <ru...@gmail.com>
> To: "user@pig.apache.org" <us...@pig.apache.org>
> Cc:
> Sent: Tuesday, 9 October 2012 7:34 PM
> Subject: Re: Error with Pig (CDH4.0.0)
>
> I've often wondered - as we see a lot of these legacy pig issues in
> CDH on this list - is it hard to upgrade Pig on CDH by downloading the
> latest stable release of pig, unpacking the .tgz and running it? Is
> upgrading Pig on CDH as simple as wget, like on Apache Hadoop and
> others, or is it somehow more complex? Pig is after all a client-side
> tool.
>
> Russell Jurney http://datasyndrome.com
>
> On Oct 9, 2012, at 4:15 PM, Cheolsoo Park <ch...@cloudera.com> wrote:
>
> > Hi Dhaval,
> >
> > That certainly works for me.
> >
> > How did you upgrade CDH? Did you install it via RPMs? Did you completely
> > uninstall CDH3u3 before installing CDH4?
> >
> > It sounds to me like a CDH-upgrade issue rather than Pig issue. Can you
> > please provide steps that you took to upgrade CDH?
> >
> > Thanks,
> > Cheolsoo
> >
> > On Tue, Oct 9, 2012 at 4:02 PM, Dhaval Shah <prince_mithibai@yahoo.co.in
> >wrote:
> >
> >> Thanks for getting back Cheolsoo..
> >>
> >> All I am trying to do is run the pig shell/grunt and do this:
> >> p = LOAD 'file_name';
> >>
> >> It comes back with the exception mentioned below..
> >>
> >> Regards,
> >> Dhaval
> >>
> >>
> >> ----- Original Message -----
> >> From: Cheolsoo Park <ch...@cloudera.com>
> >> To: user@pig.apache.org; Dhaval Shah <pr...@yahoo.co.in>
> >> Cc:
> >> Sent: Tuesday, 9 October 2012 6:57 PM
> >> Subject: Re: Error with Pig (CDH4.0.0)
> >>
> >> Hi Dhaval,
> >>
> >> CDH3u3 includes Pig 0.8, and CDH4.0.0 includes Pig 0.9. There were
> >> some incompatibilities introduced between two version.
> >>
> >>
> >>
> https://cwiki.apache.org/confluence/display/PIG/Pig+0.9+Backward+Compatibility
> >>
> >> To pin down the exact cause, I'd like to reproduce your error. Would
> >> mind providing an example script that generates the exception?
> >>
> >> Thanks,
> >> Cheolsoo
> >>
> >> On Tue, Oct 9, 2012 at 3:29 PM, Dhaval Shah <
> prince_mithibai@yahoo.co.in
> >>> wrote:
> >>
> >>> Hi everyone.. We just upgraded to CDH4.0.0 and are seeing a very weird
> >>> issue with Pig.. Everytime I try to run a LOAD command, it dies with
> the
> >>> following exception:
> >>>
> >>> ERROR 2998: Unhandled internal error. name
> >>>
> >>> java.lang.NoSuchFieldError: name
> >>> at
> >>>
> >>
> org.apache.pig.parser.QueryParserStringStream.<init>(QueryParserStringStream.java:32)
> >>> at
> >>>
> >>
> org.apache.pig.parser.QueryParserDriver.tokenize(QueryParserDriver.java:194)
> >>> at
> >>>
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:162)
> >>> at
> >>> org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
> >>> at
> >>> org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
> >>> at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
> >>> at
> >>> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:967)
> >>> at
> >>>
> >>
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
> >>> at
> >>>
> >>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
> >>> at
> >>>
> >>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> >>> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
> >>> at org.apache.pig.Main.run(Main.java:495)
> >>> at org.apache.pig.Main.main(Main.java:111)
> >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >>> at
> >>>
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >>> at
> >>>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>> at java.lang.reflect.Method.invoke(Method.java:597)
> >>> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> >>>
> >>> The same script/command worked fine with CDH3U3.. Did any APIs change?
> Or
> >>> is this a bug?
> >>> Regards,
> >>> Dhaval
> >>>
> >>
> >>
>
>
Re: Error with Pig (CDH4.0.0)
Posted by Dhaval Shah <pr...@yahoo.co.in>.
Alright I was eventually able to get the issue resolved.. For everyone's benefit, the root issue was that the pig libraries needed to be present in the HADOOP_CLASSPATH and it needs to be specifically set in hadoop_env.sh.. I tried setting PIG_CLASSPATH and HADOOP_CLASSPATH to have those libraries in the shell and then start pig but that did not help either.. Setting that explicitly in hadoop_env.sh did the trick..
As I see there are 2 issues here:
1. Somehow the pig executable seems broken.. As in setting HADOOP_CLASSPATH on the shell and then running pig should have the same effect as setting it in hadoop_env.sh.. Also having a * in the PIG_CLASSPATH/HADOOP_CLASSPATH (when set in the shell) makes the pig executable set HADOOP_CLASSPATH to empty before it actually runs the hadoop jar command inside the pig script.. In CDH3, I never needed to set the classpath to include pig libraries.. The pig executable did that for me..(and I am never overriding HADOOP_CLASSPATH.. I only append to it)
2. The error message is not super helpful.. If libraries are missing, the pig shell/grunt should not open at all.. However, in my case, it did start up and then the error message was in no ways intuitive or pointing to the root issue..
Also, I don't think this is a legacy pig issue as we tried the newer version 0.10 and it came up with the same issue.. A second issue that did pop up after putting the pig libraries in the classpath was with mapreduce framework being selected as classic and it turned out that the MRv1 libraries needed to be explicitly specified in the HADOOP_CLASSPATH (without that entry Sqoop, Java MR jobs work fine but Pig did not for whatever reason)
Anyways thanks everyone for the help
Regards,
Dhaval
----- Original Message -----
From: Russell Jurney <ru...@gmail.com>
To: "user@pig.apache.org" <us...@pig.apache.org>
Cc:
Sent: Tuesday, 9 October 2012 7:34 PM
Subject: Re: Error with Pig (CDH4.0.0)
I've often wondered - as we see a lot of these legacy pig issues in
CDH on this list - is it hard to upgrade Pig on CDH by downloading the
latest stable release of pig, unpacking the .tgz and running it? Is
upgrading Pig on CDH as simple as wget, like on Apache Hadoop and
others, or is it somehow more complex? Pig is after all a client-side
tool.
Russell Jurney http://datasyndrome.com
On Oct 9, 2012, at 4:15 PM, Cheolsoo Park <ch...@cloudera.com> wrote:
> Hi Dhaval,
>
> That certainly works for me.
>
> How did you upgrade CDH? Did you install it via RPMs? Did you completely
> uninstall CDH3u3 before installing CDH4?
>
> It sounds to me like a CDH-upgrade issue rather than Pig issue. Can you
> please provide steps that you took to upgrade CDH?
>
> Thanks,
> Cheolsoo
>
> On Tue, Oct 9, 2012 at 4:02 PM, Dhaval Shah <pr...@yahoo.co.in>wrote:
>
>> Thanks for getting back Cheolsoo..
>>
>> All I am trying to do is run the pig shell/grunt and do this:
>> p = LOAD 'file_name';
>>
>> It comes back with the exception mentioned below..
>>
>> Regards,
>> Dhaval
>>
>>
>> ----- Original Message -----
>> From: Cheolsoo Park <ch...@cloudera.com>
>> To: user@pig.apache.org; Dhaval Shah <pr...@yahoo.co.in>
>> Cc:
>> Sent: Tuesday, 9 October 2012 6:57 PM
>> Subject: Re: Error with Pig (CDH4.0.0)
>>
>> Hi Dhaval,
>>
>> CDH3u3 includes Pig 0.8, and CDH4.0.0 includes Pig 0.9. There were
>> some incompatibilities introduced between two version.
>>
>>
>> https://cwiki.apache.org/confluence/display/PIG/Pig+0.9+Backward+Compatibility
>>
>> To pin down the exact cause, I'd like to reproduce your error. Would
>> mind providing an example script that generates the exception?
>>
>> Thanks,
>> Cheolsoo
>>
>> On Tue, Oct 9, 2012 at 3:29 PM, Dhaval Shah <prince_mithibai@yahoo.co.in
>>> wrote:
>>
>>> Hi everyone.. We just upgraded to CDH4.0.0 and are seeing a very weird
>>> issue with Pig.. Everytime I try to run a LOAD command, it dies with the
>>> following exception:
>>>
>>> ERROR 2998: Unhandled internal error. name
>>>
>>> java.lang.NoSuchFieldError: name
>>> at
>>>
>> org.apache.pig.parser.QueryParserStringStream.<init>(QueryParserStringStream.java:32)
>>> at
>>>
>> org.apache.pig.parser.QueryParserDriver.tokenize(QueryParserDriver.java:194)
>>> at
>>> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:162)
>>> at
>>> org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
>>> at
>>> org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
>>> at
>>> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:967)
>>> at
>>>
>> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
>>> at
>>>
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
>>> at
>>>
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>>> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
>>> at org.apache.pig.Main.run(Main.java:495)
>>> at org.apache.pig.Main.main(Main.java:111)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> at
>>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>>>
>>> The same script/command worked fine with CDH3U3.. Did any APIs change? Or
>>> is this a bug?
>>> Regards,
>>> Dhaval
>>>
>>
>>
Re: Error with Pig (CDH4.0.0)
Posted by Russell Jurney <ru...@gmail.com>.
I've often wondered - as we see a lot of these legacy pig issues in
CDH on this list - is it hard to upgrade Pig on CDH by downloading the
latest stable release of pig, unpacking the .tgz and running it? Is
upgrading Pig on CDH as simple as wget, like on Apache Hadoop and
others, or is it somehow more complex? Pig is after all a client-side
tool.
Russell Jurney http://datasyndrome.com
On Oct 9, 2012, at 4:15 PM, Cheolsoo Park <ch...@cloudera.com> wrote:
> Hi Dhaval,
>
> That certainly works for me.
>
> How did you upgrade CDH? Did you install it via RPMs? Did you completely
> uninstall CDH3u3 before installing CDH4?
>
> It sounds to me like a CDH-upgrade issue rather than Pig issue. Can you
> please provide steps that you took to upgrade CDH?
>
> Thanks,
> Cheolsoo
>
> On Tue, Oct 9, 2012 at 4:02 PM, Dhaval Shah <pr...@yahoo.co.in>wrote:
>
>> Thanks for getting back Cheolsoo..
>>
>> All I am trying to do is run the pig shell/grunt and do this:
>> p = LOAD 'file_name';
>>
>> It comes back with the exception mentioned below..
>>
>> Regards,
>> Dhaval
>>
>>
>> ----- Original Message -----
>> From: Cheolsoo Park <ch...@cloudera.com>
>> To: user@pig.apache.org; Dhaval Shah <pr...@yahoo.co.in>
>> Cc:
>> Sent: Tuesday, 9 October 2012 6:57 PM
>> Subject: Re: Error with Pig (CDH4.0.0)
>>
>> Hi Dhaval,
>>
>> CDH3u3 includes Pig 0.8, and CDH4.0.0 includes Pig 0.9. There were
>> some incompatibilities introduced between two version.
>>
>>
>> https://cwiki.apache.org/confluence/display/PIG/Pig+0.9+Backward+Compatibility
>>
>> To pin down the exact cause, I'd like to reproduce your error. Would
>> mind providing an example script that generates the exception?
>>
>> Thanks,
>> Cheolsoo
>>
>> On Tue, Oct 9, 2012 at 3:29 PM, Dhaval Shah <prince_mithibai@yahoo.co.in
>>> wrote:
>>
>>> Hi everyone.. We just upgraded to CDH4.0.0 and are seeing a very weird
>>> issue with Pig.. Everytime I try to run a LOAD command, it dies with the
>>> following exception:
>>>
>>> ERROR 2998: Unhandled internal error. name
>>>
>>> java.lang.NoSuchFieldError: name
>>> at
>>>
>> org.apache.pig.parser.QueryParserStringStream.<init>(QueryParserStringStream.java:32)
>>> at
>>>
>> org.apache.pig.parser.QueryParserDriver.tokenize(QueryParserDriver.java:194)
>>> at
>>> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:162)
>>> at
>>> org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
>>> at
>>> org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
>>> at
>>> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:967)
>>> at
>>>
>> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
>>> at
>>>
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
>>> at
>>>
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>>> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
>>> at org.apache.pig.Main.run(Main.java:495)
>>> at org.apache.pig.Main.main(Main.java:111)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> at
>>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>>>
>>> The same script/command worked fine with CDH3U3.. Did any APIs change? Or
>>> is this a bug?
>>> Regards,
>>> Dhaval
>>>
>>
>>
Re: Error with Pig (CDH4.0.0)
Posted by Cheolsoo Park <ch...@cloudera.com>.
Hi Dhaval,
That certainly works for me.
How did you upgrade CDH? Did you install it via RPMs? Did you completely
uninstall CDH3u3 before installing CDH4?
It sounds to me like a CDH-upgrade issue rather than Pig issue. Can you
please provide steps that you took to upgrade CDH?
Thanks,
Cheolsoo
On Tue, Oct 9, 2012 at 4:02 PM, Dhaval Shah <pr...@yahoo.co.in>wrote:
> Thanks for getting back Cheolsoo..
>
> All I am trying to do is run the pig shell/grunt and do this:
> p = LOAD 'file_name';
>
> It comes back with the exception mentioned below..
>
> Regards,
> Dhaval
>
>
> ----- Original Message -----
> From: Cheolsoo Park <ch...@cloudera.com>
> To: user@pig.apache.org; Dhaval Shah <pr...@yahoo.co.in>
> Cc:
> Sent: Tuesday, 9 October 2012 6:57 PM
> Subject: Re: Error with Pig (CDH4.0.0)
>
> Hi Dhaval,
>
> CDH3u3 includes Pig 0.8, and CDH4.0.0 includes Pig 0.9. There were
> some incompatibilities introduced between two version.
>
>
> https://cwiki.apache.org/confluence/display/PIG/Pig+0.9+Backward+Compatibility
>
> To pin down the exact cause, I'd like to reproduce your error. Would
> mind providing an example script that generates the exception?
>
> Thanks,
> Cheolsoo
>
> On Tue, Oct 9, 2012 at 3:29 PM, Dhaval Shah <prince_mithibai@yahoo.co.in
> >wrote:
>
> > Hi everyone.. We just upgraded to CDH4.0.0 and are seeing a very weird
> > issue with Pig.. Everytime I try to run a LOAD command, it dies with the
> > following exception:
> >
> > ERROR 2998: Unhandled internal error. name
> >
> > java.lang.NoSuchFieldError: name
> > at
> >
> org.apache.pig.parser.QueryParserStringStream.<init>(QueryParserStringStream.java:32)
> > at
> >
> org.apache.pig.parser.QueryParserDriver.tokenize(QueryParserDriver.java:194)
> > at
> > org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:162)
> > at
> > org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
> > at
> > org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
> > at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
> > at
> > org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:967)
> > at
> >
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
> > at
> >
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
> > at
> >
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> > at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
> > at org.apache.pig.Main.run(Main.java:495)
> > at org.apache.pig.Main.main(Main.java:111)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> >
> > The same script/command worked fine with CDH3U3.. Did any APIs change? Or
> > is this a bug?
> > Regards,
> > Dhaval
> >
>
>
Re: Error with Pig (CDH4.0.0)
Posted by Dhaval Shah <pr...@yahoo.co.in>.
Thanks for getting back Cheolsoo..
All I am trying to do is run the pig shell/grunt and do this:
p = LOAD 'file_name';
It comes back with the exception mentioned below..
Regards,
Dhaval
----- Original Message -----
From: Cheolsoo Park <ch...@cloudera.com>
To: user@pig.apache.org; Dhaval Shah <pr...@yahoo.co.in>
Cc:
Sent: Tuesday, 9 October 2012 6:57 PM
Subject: Re: Error with Pig (CDH4.0.0)
Hi Dhaval,
CDH3u3 includes Pig 0.8, and CDH4.0.0 includes Pig 0.9. There were
some incompatibilities introduced between two version.
https://cwiki.apache.org/confluence/display/PIG/Pig+0.9+Backward+Compatibility
To pin down the exact cause, I'd like to reproduce your error. Would
mind providing an example script that generates the exception?
Thanks,
Cheolsoo
On Tue, Oct 9, 2012 at 3:29 PM, Dhaval Shah <pr...@yahoo.co.in>wrote:
> Hi everyone.. We just upgraded to CDH4.0.0 and are seeing a very weird
> issue with Pig.. Everytime I try to run a LOAD command, it dies with the
> following exception:
>
> ERROR 2998: Unhandled internal error. name
>
> java.lang.NoSuchFieldError: name
> at
> org.apache.pig.parser.QueryParserStringStream.<init>(QueryParserStringStream.java:32)
> at
> org.apache.pig.parser.QueryParserDriver.tokenize(QueryParserDriver.java:194)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:162)
> at
> org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
> at
> org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:967)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
> at org.apache.pig.Main.run(Main.java:495)
> at org.apache.pig.Main.main(Main.java:111)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>
> The same script/command worked fine with CDH3U3.. Did any APIs change? Or
> is this a bug?
> Regards,
> Dhaval
>
Re: Error with Pig (CDH4.0.0)
Posted by Cheolsoo Park <ch...@cloudera.com>.
Hi Dhaval,
CDH3u3 includes Pig 0.8, and CDH4.0.0 includes Pig 0.9. There were
some incompatibilities introduced between two version.
https://cwiki.apache.org/confluence/display/PIG/Pig+0.9+Backward+Compatibility
To pin down the exact cause, I'd like to reproduce your error. Would
mind providing an example script that generates the exception?
Thanks,
Cheolsoo
On Tue, Oct 9, 2012 at 3:29 PM, Dhaval Shah <pr...@yahoo.co.in>wrote:
> Hi everyone.. We just upgraded to CDH4.0.0 and are seeing a very weird
> issue with Pig.. Everytime I try to run a LOAD command, it dies with the
> following exception:
>
> ERROR 2998: Unhandled internal error. name
>
> java.lang.NoSuchFieldError: name
> at
> org.apache.pig.parser.QueryParserStringStream.<init>(QueryParserStringStream.java:32)
> at
> org.apache.pig.parser.QueryParserDriver.tokenize(QueryParserDriver.java:194)
> at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:162)
> at
> org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
> at
> org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
> at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:967)
> at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
> at org.apache.pig.Main.run(Main.java:495)
> at org.apache.pig.Main.main(Main.java:111)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>
> The same script/command worked fine with CDH3U3.. Did any APIs change? Or
> is this a bug?
> Regards,
> Dhaval
>