You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by cs...@orange.com on 2013/05/03 11:44:53 UTC
Different behaviour with same Pig Version
Hi,
I have 2 differents behaviour for the same Pig version (with hadoop 1.1.2) on different servers. If anyone can tell me why with the same versions and the same parameters I don't have the same behaviour. I'm trying to set the builtin import list but it's not used with one hadoop cluster whereas it's used when I use it on a single node cluster :
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log'
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/..'
+ '[' '' == true ']'
+ '[' -n /usr/local/bin/hadoop ']'
+ '[' false == true ']'
+ '[' -f /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-withouthadoop.jar ']'
++ echo /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
+ PIG_JAR=/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
+ '[' -z /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
+ '[' -n /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
+ CLASSPATH='/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar'
+ export 'HADOOP_CLASSPATH=/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
+ HADOOP_CLASSPATH='/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
+ export 'HADOOP_OPTS=-Xmx1000m -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/opt/apache-cassandra-1.2.3/lib/apache-cassandra-thrift-1.2.3.jar:/opt/apache-cassandra-1.2.3/lib/libthrift-0.7.0.jar -4 log4j.properties -Dpig.log.dir=/opt/pig-0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/opt/pig-0.11.1/bin/.. -Djava.net.preferIPv4Stack=true '
+ HADOOP_OPTS='-Xmx1000m -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/opt/apache-cassandra-1.2.3/lib/apache-cassandra-thrift-1.2.3.jar:/opt/apache-cassandra-1.2.3/lib/libthrift-0.7.0.jar -4 log4j.properties -Dpig.log.dir=/opt/pig-0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/opt/pig-0.11.1/bin/.. -Djava.net.preferIPv4Stack=true '
+ '[' false == true ']'
+ exec /usr/bin/hadoop jar /opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar ./hadoop-sum-value_1.pig
2013-05-03 07:57:55,140 [main] INFO org.apache.pig.Main - Apache Pig version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
2013-05-03 07:57:55,141 [main] INFO org.apache.pig.Main - Logging error messages to: /home/ubuntu/pig_1367567875107.log
2013-05-03 07:57:57,502 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/ubuntu/.pigbootup not found
2013-05-03 07:57:58,461 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://hadoop-cass-1:9100
2013-05-03 07:57:59,663 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: hadoop-cass-1:8021
2013-05-03 07:58:01,387 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve CassandraStorage2 using imports: [, org.apache.pig.builtin., org.apache.pig.impl.builtin.]
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log'
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/..'
+ '[' '' == true ']'
+ '[' -n /usr/local/bin/hadoop ']'
+ '[' false == true ']'
+ '[' -f /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-withouthadoop.jar ']'
++ echo /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
+ PIG_JAR=/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
+ '[' -z /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
+ '[' -n /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
+ CLASSPATH='/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar'
+ export 'HADOOP_CLASSPATH=/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
+ HADOOP_CLASSPATH='/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
+ export 'HADOOP_OPTS=-Xmx1000m -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/.. '
+ HADOOP_OPTS='-Xmx1000m -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/.. '
+ '[' false == true ']'
+ exec /usr/local/bin/hadoop jar /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ./hadoop-sum-value_1.pig
2013-05-03 09:57:56,810 [main] INFO org.apache.pig.Main - Apache Pig version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
2013-05-03 09:57:56,811 [main] INFO org.apache.pig.Main - Logging error messages to: /private/tmp/pig_1367567876808.log
2013-05-03 09:57:56.980 java[81053:1203] Unable to load realm info from SCDynamicStore
2013-05-03 09:57:57,190 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /Users/cyril/.pigbootup not found
2013-05-03 09:57:57,333 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://localhost:9000
2013-05-03 09:57:57,513 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: localhost:9001
2013-05-03 09:57:57,944 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve CassandraStorage2 using imports: [org.apache.cassandra.hadoop.pig., , org.apache.pig.builtin., org.apache.pig.impl.builtin.]
I'm using an undefined function just to get the error which indicates the import list which is not the same as you can see even if I use the same options :
PIG_OPTS = ' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
If anyone has an explanation ?
thanks
--
Cyril SCETBON
_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified.
Thank you.
Re: Different behaviour with same Pig Version
Posted by cs...@orange.com.
I think it's a bug in the default hadoop-env.sh :
# Extra Java runtime options. Empty by default.
export HADOOP_OPTS="-Djava.net.preferIPv4Stack=true $HADOOP_CLIENT_OPTS"
we should be able to overload this in the console without having to go into hadoop-env.sh to change it. So I replaced this line by :
# Extra Java runtime options. Empty by default.
export HADOOP_OPTS="-Djava.net.preferIPv4Stack=true $HADOOP_CLIENT_OPTS $HADOOP_OPTS"
Regards
--
Cyril SCETBON
On May 6, 2013, at 7:35 PM, Prashant Kommireddi <pr...@gmail.com>> wrote:
This blog post describes some of this behavior
http://hadoopified.wordpress.com/2013/04/07/pig-startup-script-behavior/
On Mon, May 6, 2013 at 1:07 AM, <cs...@orange.com> wrote:
forget it ! It came from hadoop-env.sh provided by the package I installed
on one cluster :(
Regards
--
Cyril SCETBON
On May 3, 2013, at 11:44 AM, cscetbon.ext@orange.com<mailto:
cscetbon.ext@orange.com> wrote:
Hi,
I have 2 differents behaviour for the same Pig version (with hadoop 1.1.2)
on different servers. If anyone can tell me why with the same versions and
the same parameters I don't have the same behaviour. I'm trying to set the
builtin import list but it's not used with one hadoop cluster whereas it's
used when I use it on a single node cluster :
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
-Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
-Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
-Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
-Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
-Dpig.log.file=pig.log'
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
-Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
-Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
-Dpig.log.file=pig.log
-Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/..'
+ '[' '' == true ']'
+ '[' -n /usr/local/bin/hadoop ']'
+ '[' false == true ']'
+ '[' -f /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-withouthadoop.jar
']'
++ echo
/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
+
PIG_JAR=/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
+ '[' -z
/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
+ '[' -n
/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
+
CLASSPATH='/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar'
+ export
'HADOOP_CLASSPATH=/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
+
HADOOP_CLASSPATH='/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
+ export 'HADOOP_OPTS=-Xmx1000m
-Dudf.import.list=org.apache.cassandra.hadoop.pig
-Dpig.additional.jars=/opt/apache-cassandra-1.2.3/lib/apache-cassandra-thrift-1.2.3.jar:/opt/apache-cassandra-1.2.3/lib/libthrift-0.7.0.jar
-4 log4j.properties -Dpig.log.dir=/opt/pig-0.11.1/bin/../logs
-Dpig.log.file=pig.log -Dpig.home.dir=/opt/pig-0.11.1/bin/..
-Djava.net.preferIPv4Stack=true '
+ HADOOP_OPTS='-Xmx1000m
-Dudf.import.list=org.apache.cassandra.hadoop.pig
-Dpig.additional.jars=/opt/apache-cassandra-1.2.3/lib/apache-cassandra-thrift-1.2.3.jar:/opt/apache-cassandra-1.2.3/lib/libthrift-0.7.0.jar
-4 log4j.properties -Dpig.log.dir=/opt/pig-0.11.1/bin/../logs
-Dpig.log.file=pig.log -Dpig.home.dir=/opt/pig-0.11.1/bin/..
-Djava.net.preferIPv4Stack=true '
+ '[' false == true ']'
+ exec /usr/bin/hadoop jar
/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar ./hadoop-sum-value_1.pig
2013-05-03 07:57:55,140 [main] INFO org.apache.pig.Main - Apache Pig
version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
2013-05-03 07:57:55,141 [main] INFO org.apache.pig.Main - Logging error
messages to: /home/ubuntu/pig_1367567875107.log
2013-05-03 07:57:57,502 [main] INFO org.apache.pig.impl.util.Utils -
Default bootup file /home/ubuntu/.pigbootup not found
2013-05-03 07:57:58,461 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
Connecting to hadoop file system at: hdfs://hadoop-cass-1:9100
2013-05-03 07:57:59,663 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
Connecting to map-reduce job tracker at: hadoop-cass-1:8021
2013-05-03 07:58:01,387 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 1070: Could not resolve CassandraStorage2 using imports: [,
org.apache.pig.builtin., org.apache.pig.impl.builtin.]
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
-Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
-Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
-Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
-Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
-Dpig.log.file=pig.log'
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
-Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
-Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
-Dpig.log.file=pig.log
-Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/..'
+ '[' '' == true ']'
+ '[' -n /usr/local/bin/hadoop ']'
+ '[' false == true ']'
+ '[' -f /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-withouthadoop.jar
']'
++ echo
/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
+
PIG_JAR=/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
+ '[' -z
/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
+ '[' -n
/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
+
CLASSPATH='/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar'
+ export
'HADOOP_CLASSPATH=/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
+
HADOOP_CLASSPATH='/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
+ export 'HADOOP_OPTS=-Xmx1000m
-Dudf.import.list=org.apache.cassandra.hadoop.pig
-Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
-Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
-Dpig.log.file=pig.log
-Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/.. '
+ HADOOP_OPTS='-Xmx1000m
-Dudf.import.list=org.apache.cassandra.hadoop.pig
-Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
-Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
-Dpig.log.file=pig.log
-Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/.. '
+ '[' false == true ']'
+ exec /usr/local/bin/hadoop jar
/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
./hadoop-sum-value_1.pig
2013-05-03 09:57:56,810 [main] INFO org.apache.pig.Main - Apache Pig
version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
2013-05-03 09:57:56,811 [main] INFO org.apache.pig.Main - Logging error
messages to: /private/tmp/pig_1367567876808.log
2013-05-03 09:57:56.980 java[81053:1203] Unable to load realm info from
SCDynamicStore
2013-05-03 09:57:57,190 [main] INFO org.apache.pig.impl.util.Utils -
Default bootup file /Users/cyril/.pigbootup not found
2013-05-03 09:57:57,333 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
Connecting to hadoop file system at: hdfs://localhost:9000
2013-05-03 09:57:57,513 [main] INFO
org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
Connecting to map-reduce job tracker at: localhost:9001
2013-05-03 09:57:57,944 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 1070: Could not resolve CassandraStorage2 using imports:
[org.apache.cassandra.hadoop.pig., , org.apache.pig.builtin.,
org.apache.pig.impl.builtin.]
I'm using an undefined function just to get the error which indicates the
import list which is not the same as you can see even if I use the same
options :
PIG_OPTS = ' -Dudf.import.list=org.apache.cassandra.hadoop.pig
-Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
-Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
If anyone has an explanation ?
thanks
--
Cyril SCETBON
_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
electroniques etant susceptibles d'alteration,
France Telecom - Orange decline toute responsabilite si ce message a ete
altere, deforme ou falsifie. Merci.
This message and its attachments may contain confidential or privileged
information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and
delete this message and its attachments.
As emails may be altered, France Telecom - Orange is not liable for
messages that have been modified, changed or falsified.
Thank you.
_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
electroniques etant susceptibles d'alteration,
France Telecom - Orange decline toute responsabilite si ce message a ete
altere, deforme ou falsifie. Merci.
This message and its attachments may contain confidential or privileged
information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and
delete this message and its attachments.
As emails may be altered, France Telecom - Orange is not liable for
messages that have been modified, changed or falsified.
Thank you.
_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified.
Thank you.
Re: Different behaviour with same Pig Version
Posted by Prashant Kommireddi <pr...@gmail.com>.
This blog post describes some of this behavior
http://hadoopified.wordpress.com/2013/04/07/pig-startup-script-behavior/
On Mon, May 6, 2013 at 1:07 AM, <cs...@orange.com> wrote:
> forget it ! It came from hadoop-env.sh provided by the package I installed
> on one cluster :(
>
> Regards
> --
> Cyril SCETBON
>
> On May 3, 2013, at 11:44 AM, cscetbon.ext@orange.com<mailto:
> cscetbon.ext@orange.com> wrote:
>
> Hi,
>
> I have 2 differents behaviour for the same Pig version (with hadoop 1.1.2)
> on different servers. If anyone can tell me why with the same versions and
> the same parameters I don't have the same behaviour. I'm trying to set the
> builtin import list but it's not used with one hadoop cluster whereas it's
> used when I use it on a single node cluster :
>
> + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
> -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
> -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
> + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
> -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
> -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
> -Dpig.log.file=pig.log'
> + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
> -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
> -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
> -Dpig.log.file=pig.log
> -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/..'
> + '[' '' == true ']'
> + '[' -n /usr/local/bin/hadoop ']'
> + '[' false == true ']'
> + '[' -f /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-withouthadoop.jar
> ']'
> ++ echo
> /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
> +
> PIG_JAR=/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
> + '[' -z
> /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
> + '[' -n
> /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
> +
> CLASSPATH='/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar'
> + export
> 'HADOOP_CLASSPATH=/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
> +
> HADOOP_CLASSPATH='/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
> + export 'HADOOP_OPTS=-Xmx1000m
> -Dudf.import.list=org.apache.cassandra.hadoop.pig
> -Dpig.additional.jars=/opt/apache-cassandra-1.2.3/lib/apache-cassandra-thrift-1.2.3.jar:/opt/apache-cassandra-1.2.3/lib/libthrift-0.7.0.jar
> -4 log4j.properties -Dpig.log.dir=/opt/pig-0.11.1/bin/../logs
> -Dpig.log.file=pig.log -Dpig.home.dir=/opt/pig-0.11.1/bin/..
> -Djava.net.preferIPv4Stack=true '
> + HADOOP_OPTS='-Xmx1000m
> -Dudf.import.list=org.apache.cassandra.hadoop.pig
> -Dpig.additional.jars=/opt/apache-cassandra-1.2.3/lib/apache-cassandra-thrift-1.2.3.jar:/opt/apache-cassandra-1.2.3/lib/libthrift-0.7.0.jar
> -4 log4j.properties -Dpig.log.dir=/opt/pig-0.11.1/bin/../logs
> -Dpig.log.file=pig.log -Dpig.home.dir=/opt/pig-0.11.1/bin/..
> -Djava.net.preferIPv4Stack=true '
> + '[' false == true ']'
> + exec /usr/bin/hadoop jar
> /opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar ./hadoop-sum-value_1.pig
> 2013-05-03 07:57:55,140 [main] INFO org.apache.pig.Main - Apache Pig
> version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
> 2013-05-03 07:57:55,141 [main] INFO org.apache.pig.Main - Logging error
> messages to: /home/ubuntu/pig_1367567875107.log
> 2013-05-03 07:57:57,502 [main] INFO org.apache.pig.impl.util.Utils -
> Default bootup file /home/ubuntu/.pigbootup not found
> 2013-05-03 07:57:58,461 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> Connecting to hadoop file system at: hdfs://hadoop-cass-1:9100
> 2013-05-03 07:57:59,663 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> Connecting to map-reduce job tracker at: hadoop-cass-1:8021
> 2013-05-03 07:58:01,387 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 1070: Could not resolve CassandraStorage2 using imports: [,
> org.apache.pig.builtin., org.apache.pig.impl.builtin.]
>
> + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
> -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
> -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
> + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
> -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
> -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
> -Dpig.log.file=pig.log'
> + PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig
> -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
> -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
> -Dpig.log.file=pig.log
> -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/..'
> + '[' '' == true ']'
> + '[' -n /usr/local/bin/hadoop ']'
> + '[' false == true ']'
> + '[' -f /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-withouthadoop.jar
> ']'
> ++ echo
> /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
> +
> PIG_JAR=/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
> + '[' -z
> /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
> + '[' -n
> /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
> +
> CLASSPATH='/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar'
> + export
> 'HADOOP_CLASSPATH=/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
> +
> HADOOP_CLASSPATH='/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
> + export 'HADOOP_OPTS=-Xmx1000m
> -Dudf.import.list=org.apache.cassandra.hadoop.pig
> -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
> -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
> -Dpig.log.file=pig.log
> -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/.. '
> + HADOOP_OPTS='-Xmx1000m
> -Dudf.import.list=org.apache.cassandra.hadoop.pig
> -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
> -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs
> -Dpig.log.file=pig.log
> -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/.. '
> + '[' false == true ']'
> + exec /usr/local/bin/hadoop jar
> /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
> ./hadoop-sum-value_1.pig
> 2013-05-03 09:57:56,810 [main] INFO org.apache.pig.Main - Apache Pig
> version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
> 2013-05-03 09:57:56,811 [main] INFO org.apache.pig.Main - Logging error
> messages to: /private/tmp/pig_1367567876808.log
> 2013-05-03 09:57:56.980 java[81053:1203] Unable to load realm info from
> SCDynamicStore
> 2013-05-03 09:57:57,190 [main] INFO org.apache.pig.impl.util.Utils -
> Default bootup file /Users/cyril/.pigbootup not found
> 2013-05-03 09:57:57,333 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> Connecting to hadoop file system at: hdfs://localhost:9000
> 2013-05-03 09:57:57,513 [main] INFO
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> Connecting to map-reduce job tracker at: localhost:9001
> 2013-05-03 09:57:57,944 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 1070: Could not resolve CassandraStorage2 using imports:
> [org.apache.cassandra.hadoop.pig., , org.apache.pig.builtin.,
> org.apache.pig.impl.builtin.]
>
>
> I'm using an undefined function just to get the error which indicates the
> import list which is not the same as you can see even if I use the same
> options :
> PIG_OPTS = ' -Dudf.import.list=org.apache.cassandra.hadoop.pig
> -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar
> -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
>
> If anyone has an explanation ?
>
> thanks
> --
> Cyril SCETBON
>
>
>
> _________________________________________________________________________________________________________________________
>
> Ce message et ses pieces jointes peuvent contenir des informations
> confidentielles ou privilegiees et ne doivent donc
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
> recu ce message par erreur, veuillez le signaler
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
> electroniques etant susceptibles d'alteration,
> France Telecom - Orange decline toute responsabilite si ce message a ete
> altere, deforme ou falsifie. Merci.
>
> This message and its attachments may contain confidential or privileged
> information that may be protected by law;
> they should not be distributed, used or copied without authorisation.
> If you have received this email in error, please notify the sender and
> delete this message and its attachments.
> As emails may be altered, France Telecom - Orange is not liable for
> messages that have been modified, changed or falsified.
> Thank you.
>
>
>
>
> _________________________________________________________________________________________________________________________
>
> Ce message et ses pieces jointes peuvent contenir des informations
> confidentielles ou privilegiees et ne doivent donc
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
> recu ce message par erreur, veuillez le signaler
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
> electroniques etant susceptibles d'alteration,
> France Telecom - Orange decline toute responsabilite si ce message a ete
> altere, deforme ou falsifie. Merci.
>
> This message and its attachments may contain confidential or privileged
> information that may be protected by law;
> they should not be distributed, used or copied without authorisation.
> If you have received this email in error, please notify the sender and
> delete this message and its attachments.
> As emails may be altered, France Telecom - Orange is not liable for
> messages that have been modified, changed or falsified.
> Thank you.
>
>
Re: Different behaviour with same Pig Version
Posted by cs...@orange.com.
forget it ! It came from hadoop-env.sh provided by the package I installed on one cluster :(
Regards
--
Cyril SCETBON
On May 3, 2013, at 11:44 AM, cscetbon.ext@orange.com<ma...@orange.com> wrote:
Hi,
I have 2 differents behaviour for the same Pig version (with hadoop 1.1.2) on different servers. If anyone can tell me why with the same versions and the same parameters I don't have the same behaviour. I'm trying to set the builtin import list but it's not used with one hadoop cluster whereas it's used when I use it on a single node cluster :
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log'
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/..'
+ '[' '' == true ']'
+ '[' -n /usr/local/bin/hadoop ']'
+ '[' false == true ']'
+ '[' -f /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-withouthadoop.jar ']'
++ echo /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
+ PIG_JAR=/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
+ '[' -z /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
+ '[' -n /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
+ CLASSPATH='/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar'
+ export 'HADOOP_CLASSPATH=/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
+ HADOOP_CLASSPATH='/etc/hadoop/conf:/usr/lib/jvm/java-7-oracle/lib/tools.jar:::/opt/apache-cassandra-1.2.3/lib/*:/usr/share/hadoop/lib/*:/opt/pig-0.11.1/pig-0.11.1-withouthadoop.jar:/etc/hadoop:/opt/pig-0.11.1/bin/../lib/jython-standalone-2.5.3.jar:/opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
+ export 'HADOOP_OPTS=-Xmx1000m -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/opt/apache-cassandra-1.2.3/lib/apache-cassandra-thrift-1.2.3.jar:/opt/apache-cassandra-1.2.3/lib/libthrift-0.7.0.jar -4 log4j.properties -Dpig.log.dir=/opt/pig-0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/opt/pig-0.11.1/bin/.. -Djava.net.preferIPv4Stack=true '
+ HADOOP_OPTS='-Xmx1000m -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/opt/apache-cassandra-1.2.3/lib/apache-cassandra-thrift-1.2.3.jar:/opt/apache-cassandra-1.2.3/lib/libthrift-0.7.0.jar -4 log4j.properties -Dpig.log.dir=/opt/pig-0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/opt/pig-0.11.1/bin/.. -Djava.net.preferIPv4Stack=true '
+ '[' false == true ']'
+ exec /usr/bin/hadoop jar /opt/pig-0.11.1/bin/../pig-0.11.1-withouthadoop.jar ./hadoop-sum-value_1.pig
2013-05-03 07:57:55,140 [main] INFO org.apache.pig.Main - Apache Pig version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
2013-05-03 07:57:55,141 [main] INFO org.apache.pig.Main - Logging error messages to: /home/ubuntu/pig_1367567875107.log
2013-05-03 07:57:57,502 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/ubuntu/.pigbootup not found
2013-05-03 07:57:58,461 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://hadoop-cass-1:9100
2013-05-03 07:57:59,663 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: hadoop-cass-1:8021
2013-05-03 07:58:01,387 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve CassandraStorage2 using imports: [, org.apache.pig.builtin., org.apache.pig.impl.builtin.]
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log'
+ PIG_OPTS=' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/..'
+ '[' '' == true ']'
+ '[' -n /usr/local/bin/hadoop ']'
+ '[' false == true ']'
+ '[' -f /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-withouthadoop.jar ']'
++ echo /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
+ PIG_JAR=/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar
+ '[' -z /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
+ '[' -n /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ']'
+ CLASSPATH='/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar'
+ export 'HADOOP_CLASSPATH=/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
+ HADOOP_CLASSPATH='/usr/local/Cellar/hadoop/1.1.2/libexec/conf:/Library/Java/JavaVirtualMachines/1.6.0_32-b05-420.jdk/Contents/Home/lib/tools.jar:::/usr/local/Cellar/cassandra/1.2.4/*:/usr/local/Cellar/pig/0.11.1/pig-0.11.1.jar:/usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar:'
+ export 'HADOOP_OPTS=-Xmx1000m -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/.. '
+ HADOOP_OPTS='-Xmx1000m -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs -Dpig.log.file=pig.log -Dpig.home.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/.. '
+ '[' false == true ']'
+ exec /usr/local/bin/hadoop jar /usr/local/bin/../Cellar/pig/0.11.1/bin/../pig-0.11.1-withouthadoop.jar ./hadoop-sum-value_1.pig
2013-05-03 09:57:56,810 [main] INFO org.apache.pig.Main - Apache Pig version 0.11.1 (r1459641) compiled Mar 22 2013, 02:13:53
2013-05-03 09:57:56,811 [main] INFO org.apache.pig.Main - Logging error messages to: /private/tmp/pig_1367567876808.log
2013-05-03 09:57:56.980 java[81053:1203] Unable to load realm info from SCDynamicStore
2013-05-03 09:57:57,190 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /Users/cyril/.pigbootup not found
2013-05-03 09:57:57,333 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://localhost:9000
2013-05-03 09:57:57,513 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: localhost:9001
2013-05-03 09:57:57,944 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve CassandraStorage2 using imports: [org.apache.cassandra.hadoop.pig., , org.apache.pig.builtin., org.apache.pig.impl.builtin.]
I'm using an undefined function just to get the error which indicates the import list which is not the same as you can see even if I use the same options :
PIG_OPTS = ' -Dudf.import.list=org.apache.cassandra.hadoop.pig -Dpig.additional.jars=/usr/local/Cellar/cassandra/1.2.4/apache-cassandra-thrift-1.2.4.jar:/usr/local/Cellar/cassandra/1.2.4/libthrift-0.7.0.jar -Dpig.log.dir=/usr/local/bin/../Cellar/pig/0.11.1/bin/../logs'
If anyone has an explanation ?
thanks
--
Cyril SCETBON
_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified.
Thank you.
_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.
This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified.
Thank you.