You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Kuo Liu <ku...@seagate.com> on 2014/07/30 17:54:33 UTC
"MAX" command in pig-0.12.1 and pig-0.13.0
Hello,
I've recently switched using pig-0.9.2.15 to pig-0.12.1 and pig-0.13.0 and
I've encountered some issues.
Here's the detail:
http://stackoverflow.com/questions/25026992/pig-max-command-for-pig-0-12-1-and-pig-0-13-0-with-hadoop-2-4-0
Any help is appreciated.
Best,
-Kuo Liu
Re: "MAX" command in pig-0.12.1 and pig-0.13.0
Posted by Kuo Liu <ku...@seagate.com>.
Hi Asit,
Thank you for your suggestion.
I ran the test again using pig-0.13.0.
I've echo-ed $HADOOP_BIN and $PIG_JAR inside of the script:
$PIG_HOME/bin/pig, right before
"exec "$HADOOP_BIN" jar "$PIG_JAR" "${remaining[@]}" and it shows $PIG_JAR
is pig-0.13.0-withouthadoop-h2.jar
[root@namenode hadoop2]# /usr/lib/pig/pig-0.13.0/bin/pig
/usr/bin/hadoop
/usr/lib/pig/pig-0.13.0/bin/../pig-0.13.0-withouthadoop-h2.jar
I also used the -secrectDebugCmd flag, the last line of the following also
shows I'm using pig-0.13.0-withouthadoop-h2.jar
[root@namenode hadoop2]# /usr/lib/pig/pig-0.13.0/bin/pig -secretDebugCmd
Find hadoop at /usr/bin/hadoop
dry run:
HADOOP_CLASSPATH:
/usr/lib/pig/pig-0.13.0/bin/../conf:/usr/java/default/lib/tools.jar:/etc/hadoop/conf:/usr/lib/pig/pig-0.13.0/bin/../lib/accumulo-core-1.5.0.jar:/usr/lib/pig/pig-0.13.0/bin/../lib/accumulo-fate-1.5.0.jar:/usr/lib/pig/pig-0.13.0/bin/../lib/accumulo-server-1.5.0.jar:/usr/lib/pig/pig-0.13.0/bin/../lib/accumulo-start-1.5.0.jar:/usr/lib/pig/pig-0.13.0/bin/../lib/accumulo-trace-1.5.0.jar:/usr/lib/pig/pig-0.13.0/bin/../lib/avro-1.7.5.jar:/usr/lib/pig/pig-0.13.0/bin/../lib/avro-mapred-1.7.5.jar:/usr/lib/pig/pig-0.13.0/bin/../lib/avro-tools-1.7.5-nodeps.jar:/usr/lib/pig/pig-0.13.0/bin/../lib/groovy-all-1.8.6.jar:/usr/lib/pig/pig-0.13.0/bin/../lib/hbase-0.94.1.jar:/usr/lib/pig/pig-0.13.0/bin/../lib/jruby-complete-1.6.7.jar:/usr/lib/pig/pig-0.13.0/bin/../lib/js-1.7R2.jar:/usr/lib/pig/pig-0.13.0/bin/../lib/json-simple-1.1.jar:/usr/lib/pig/pig-0.13.0/bin/../lib/jython-standalone-2.5.3.jar:/usr/lib/pig/pig-0.13.0/bin/../lib/piggybank.jar:/usr/lib/pig/pig-0.13.0/bin/../lib/protobuf-java-2.4.0a.jar:/usr/lib/pig/pig-0.13.0/bin/../lib/zookeeper-3.4.5.jar:/usr/lib/pig/pig-0.13.0/bin/../pig-0.13.0-withouthadoop-h2.jar:
HADOOP_OPTS: -Xmx1000m -Dpig.log.dir=/usr/lib/pig/pig-0.13.0/bin/../logs
-Dpig.log.file=pig.log -Dpig.home.dir=/usr/lib/pig/pig-0.13.0/bin/..
HADOOP_CLIENT_OPTS: -Xmx1000m
-Dpig.log.dir=/usr/lib/pig/pig-0.13.0/bin/../logs -Dpig.log.file=pig.log
-Dpig.home.dir=/usr/lib/pig/pig-0.13.0/bin/..
/usr/bin/hadoop jar
/usr/lib/pig/pig-0.13.0/bin/../pig-0.13.0-withouthadoop-h2.jar
It looks like the version of jar file I''m using is correct -- since I'm
using Hadoop-2.4.0.
Any other suggestion what might cause this?
Thank you,
-Kuo
On Wed, Jul 30, 2014 at 4:52 PM, Asit Parija <as...@sigmoidanalytics.com>
wrote:
> Hi Kuo ,
> Looks like there is a PIG compilation issue with Hadoop .Kindly check
> that you are using the right PIG jars .
>
>
> https://urldefense.proofpoint.com/v1/url?u=https://issues.apache.org/jira/browse/PIG-3892&k=2a4Akkj3oY%2FOkjwft1MTMw%3D%3D%0A&r=2N0ueFInU9%2FScBPiE%2FMJ%2B0KCwDYrtB9sT1XH%2FV4UvUw%3D%0A&m=Kjma0EXVqTZyQKKHkUuBWRI3iQOtWzkk%2B%2FeS7Xohzy8%3D%0A&s=69e90ba500c42ac3b357dd79849b29224b94b41e42b1a968c89a8ddb6e1ef352
>
> Thanks
> Asit
>
>
> On Wed, Jul 30, 2014 at 11:54 AM, Kuo Liu <ku...@seagate.com> wrote:
>
> > Hello,
> >
> > I've recently switched using pig-0.9.2.15 to pig-0.12.1 and pig-0.13.0
> and
> > I've encountered some issues.
> >
> > Here's the detail:
> >
> >
> https://urldefense.proofpoint.com/v1/url?u=http://stackoverflow.com/questions/25026992/pig-max-command-for-pig-0-12-1-and-pig-0-13-0-with-hadoop-2-4-0&k=2a4Akkj3oY%2FOkjwft1MTMw%3D%3D%0A&r=2N0ueFInU9%2FScBPiE%2FMJ%2B0KCwDYrtB9sT1XH%2FV4UvUw%3D%0A&m=Kjma0EXVqTZyQKKHkUuBWRI3iQOtWzkk%2B%2FeS7Xohzy8%3D%0A&s=89c7815c317a547b5ac79aaa2d5891cba25761971d3fed5d772f03b79e80f670
> >
> >
> > Any help is appreciated.
> > Best,
> > -Kuo Liu
> >
>
--
*Kuo Liu*, Ph.D.
Sr Engineer
Cloud Modeling and Data Analytics
Seagate Technology
T: 720.684.2545
kuo.liu@seagate.com <ch...@seagate.com>
389 Disc Drive • Longmont, CO 80503 • USA
Re: "MAX" command in pig-0.12.1 and pig-0.13.0
Posted by Asit Parija <as...@sigmoidanalytics.com>.
Hi Kuo ,
Looks like there is a PIG compilation issue with Hadoop .Kindly check
that you are using the right PIG jars .
https://issues.apache.org/jira/browse/PIG-3892
Thanks
Asit
On Wed, Jul 30, 2014 at 11:54 AM, Kuo Liu <ku...@seagate.com> wrote:
> Hello,
>
> I've recently switched using pig-0.9.2.15 to pig-0.12.1 and pig-0.13.0 and
> I've encountered some issues.
>
> Here's the detail:
>
> http://stackoverflow.com/questions/25026992/pig-max-command-for-pig-0-12-1-and-pig-0-13-0-with-hadoop-2-4-0
>
>
> Any help is appreciated.
> Best,
> -Kuo Liu
>