You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Praveen Rachabattuni (JIRA)" <ji...@apache.org> on 2015/02/24 04:51:44 UTC
[jira] [Updated] (PIG-4183) Fix classpath error when using pig
command with Spark
[ https://issues.apache.org/jira/browse/PIG-4183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Praveen Rachabattuni updated PIG-4183:
--------------------------------------
Fix Version/s: spark-branch
> Fix classpath error when using pig command with Spark
> -----------------------------------------------------
>
> Key: PIG-4183
> URL: https://issues.apache.org/jira/browse/PIG-4183
> Project: Pig
> Issue Type: Sub-task
> Components: spark
> Reporter: liyunzhang_intel
> Assignee: liyunzhang_intel
> Fix For: spark-branch
>
> Attachments: fix-pig-4183.patch
>
>
> You can reproduce the bug in following steps
> 1.Build spark -0.9 env and build hadoop1 env:
> 2.Compile code: ant jar
> 3.Export $PIG_CLASSPATH
> echo $PIG_CLASSPATH
> /home/zly/prj/oss/pig/build/ivy/lib/Pig/*:/home/zly/prj/oss/hadoop-1.2.1/conf
> 4.Run: cd $PIG_HOME/bin; ./pig –x spark id.spark.pig
> 5.Error message found in pig log
> 549 ERROR 2998: Unhandled internal error. org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V
> 550
> 551 java.lang.NoSuchMethodError: org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V
> 552 at org.eclipse.jetty.util.log.JettyAwareLogger.log(JettyAwareLogger.java:607)
> 553 at org.eclipse.jetty.util.log.JettyAwareLogger.warn(JettyAwareLogger.java:431)
> 554 at org.eclipse.jetty.util.log.Slf4jLog.warn(Slf4jLog.java:69)
> 555 at org.eclipse.jetty.util.component.AbstractLifeCycle.setFailed(AbstractLifeCycle.java:204)
> 556 at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:74)
> 557 at org.apache.spark.HttpServer.start(HttpServer.scala:65)
> 558 at org.apache.spark.broadcast.HttpBroadcast$.createServer(HttpBroadcast.scala:130)
> 559 at org.apache.spark.broadcast.HttpBroadcast$.initialize(HttpBroadcast.scala:101)
> 560 at org.apache.spark.broadcast.HttpBroadcastFactory.initialize(HttpBroadcast.scala:70)
> 561 at org.apache.spark.broadcast.BroadcastManager.initialize(Broadcast.scala:81)
> 562 at org.apache.spark.broadcast.BroadcastManager.<init>(Broadcast.scala:68)
> 563 at org.apache.spark.SparkEnv$.create(SparkEnv.scala:175)
> 564 at org.apache.spark.SparkContext.<init>(SparkContext.scala:139)
> 565 at org.apache.spark.SparkContext.<init>(SparkContext.scala:100)
> 566 at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:81)
> 567 at org.apache.pig.backend.hadoop.executionengine.spark.SparkLauncher.startSparkIfNeeded(SparkLauncher.java:202)
> 568 at org.apache.pig.backend.hadoop.executionengine.spark.SparkLauncher.launchPig(SparkLauncher.java:114)
> 569 at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:279)
> 570 at org.apache.pig.PigServer.launchPlan(PigServer.java:1378)
> 571 at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1363)
> 572 at org.apache.pig.PigServer.execute(PigServer.java:1352)
> 573 at org.apache.pig.PigServer.executeBatch(PigServer.java:403)
> 574 at org.apache.pig.PigServer.executeBatch(PigServer.java:386)
> 575 at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:170)
> 576 at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:233)
> 577 at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:204)
> 578 at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
> 579 at org.apache.pig.Main.run(Main.java:611)
> 580 at org.apache.pig.Main.main(Main.java:164)
> 581 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 582 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 583 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 584 at java.lang.reflect.Method.invoke(Method.java:606)
> 585 at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)