You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by yash datta <sa...@gmail.com> on 2015/09/27 14:39:37 UTC
Help regarding zeppelin compilation and spark interpreter
Hello devs,
I just started using zeppelin.
I just cloned the repo from github , and made the distribution using the
following command :
mvn package -Pspark-1.5 -Dhadoop.version=2.6.0 -Phadoop-2.6 -Pyarn
-DskipTests -P build-distr
1. I see this build failure :
[INFO]
------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 01:54 min
[INFO] Finished at: 2015-09-27T17:54:49+05:30
[INFO] Final Memory: 51M/323M
[INFO]
------------------------------------------------------------------------
[ERROR] Failed to execute goal on project zeppelin-geode: Could not resolve
dependencies for project
org.apache.zeppelin:zeppelin-geode:jar:0.6.0-incubating-SNAPSHOT: Failure
to find net.sourceforge.jline:jline:jar:1.0.S2-B in
https://repo.maven.apache.org/maven2 was cached in the local repository,
resolution will not be reattempted until the update interval of central has
elapsed or updates are forced -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions,
please read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the
command
[ERROR] mvn <goals> -rf :zeppelin-geode
On adding this to the pom.xml , it works :
<repositories>
<repository>
<id>ext-release-local</id>
<url>http://repo.springsource.org/simple/ext-release-local/</url>
</repository>
</repositories>
Is this a known problem ?
2. I was able to compile thereafter. Built the distribution package, then
extracted the package, changed the default port to 8888 and created a small
notebook
The problem occurs when I try to run just a simple command on the
interpreter:
print("Hello World")
I see this error in the UI:
org.apache.thrift.transport.TTransportException at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) at
org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
at
org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
at
org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) at
org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_interpret(RemoteInterpreterService.java:220)
at
org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.interpret(RemoteInterpreterService.java:205)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.interpret(RemoteInterpreter.java:211)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:207) at
org.apache.zeppelin.scheduler.Job.run(Job.java:170) at
org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:304)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
On going through the logs I see the problem:
15/09/27 17:43:40 WARN ReliableDeliverySupervisor: Association with remote
system [akka.tcp://sparkMaster@localhost:7077] has failed, address is now
gated for [5000] ms. Reason: [Association failed with
[akka.tcp://sparkMaster@localhost:7077]] Caused by: [Connection refused:
localhost/127.0.0.1:7077]
15/09/27 17:43:40 WARN AppClient$ClientEndpoint: Failed to connect to
master localhost:7077
akka.actor.ActorNotFound: Actor not found for:
ActorSelection[Anchor(akka.tcp://sparkMaster@localhost:7077/),
Path(/user/Master)]
at
akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:65)
at
akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:63)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
at
akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:73)
at
akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:74)
at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:120)
at
akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:73)
at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:40)
at
scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:248)
at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:266)
at akka.actor.EmptyLocalActorRef.specialHandle(ActorRef.scala:533)
at akka.actor.DeadLetterActorRef.specialHandle(ActorRef.scala:569)
at akka.actor.DeadLetterActorRef.$bang(ActorRef.scala:559)
at
akka.remote.RemoteActorRefProvider$RemoteDeadLetterActorRef.$bang(RemoteActorRefProvider.scala:87)
at akka.remote.EndpointWriter.postStop(Endpoint.scala:557)
at akka.actor.Actor$class.aroundPostStop(Actor.scala:477)
at akka.remote.EndpointActor.aroundPostStop(Endpoint.scala:411)
at
akka.actor.dungeon.FaultHandling$class.akka$actor$dungeon$FaultHandling$$finishTerminate(FaultHandling.scala:210)
at akka.actor.dungeon.FaultHandling$class.terminate(FaultHandling.scala:172)
at akka.actor.ActorCell.terminate(ActorCell.scala:369)
at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:462)
at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478)
at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:263)
at akka.dispatch.Mailbox.run(Mailbox.scala:219)
at
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at
scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
15/09/27 17:44:00 INFO AppClient$ClientEndpoint: Connecting to master
spark://localhost:7077...
15/09/27 17:44:00 WARN AppClient$ClientEndpoint: Could not connect to
localhost:7077: akka.remote.EndpointAssociationException: Association
failed with [akka.tcp://sparkMaster@localhost:7077]
I tried starting spark 1.5 in standalone mode manually and then adding
SPARK_HOME / MASTER in conf/zeppelin-env.sh as well , but I get the same
error.
Please help! I am stuck here for the moment. Any help is highly appreciated.
Thanks and Best Regards
Yash Datta
--
When events unfold with calm and ease
When the winds that blow are merely breeze
Learn from nature, from birds and bees
Live your life in love, and let joy not cease.
Re: Help regarding zeppelin compilation and spark interpreter
Posted by moon soo Lee <mo...@apache.org>.
Hi,
Could you make sure SPARK_HOME/bin/spark-shell.sh command work without any
problem in the same machine that Zeppelin installed.
Then just export SPARK_HOME env variable in conf/zeppelin-env.sh will make
Zeppelin work with your spark deployment.
Thanks,
moon
On 2015년 9월 27일 (일) at 오후 2:39 yash datta <sa...@gmail.com> wrote:
> Hello devs,
>
> I just started using zeppelin.
>
> I just cloned the repo from github , and made the distribution using the
> following command :
>
>
> mvn package -Pspark-1.5 -Dhadoop.version=2.6.0 -Phadoop-2.6 -Pyarn
> -DskipTests -P build-distr
>
>
> 1. I see this build failure :
>
>
> [INFO]
> ------------------------------------------------------------------------
>
> [INFO] BUILD FAILURE
>
> [INFO]
> ------------------------------------------------------------------------
>
> [INFO] Total time: 01:54 min
>
> [INFO] Finished at: 2015-09-27T17:54:49+05:30
>
> [INFO] Final Memory: 51M/323M
>
> [INFO]
> ------------------------------------------------------------------------
>
> [ERROR] Failed to execute goal on project zeppelin-geode: Could not resolve
> dependencies for project
> org.apache.zeppelin:zeppelin-geode:jar:0.6.0-incubating-SNAPSHOT: Failure
> to find net.sourceforge.jline:jline:jar:1.0.S2-B in
> https://repo.maven.apache.org/maven2 was cached in the local repository,
> resolution will not be reattempted until the update interval of central has
> elapsed or updates are forced -> [Help 1]
>
> [ERROR]
>
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
>
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>
> [ERROR]
>
> [ERROR] For more information about the errors and possible solutions,
> please read the following articles:
>
> [ERROR] [Help 1]
>
> http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
>
> [ERROR]
>
> [ERROR] After correcting the problems, you can resume the build with the
> command
>
> [ERROR] mvn <goals> -rf :zeppelin-geode
>
>
>
> On adding this to the pom.xml , it works :
>
>
> <repositories>
>
> <repository>
>
> <id>ext-release-local</id>
>
> <url>http://repo.springsource.org/simple/ext-release-local/</url>
>
> </repository>
>
> </repositories>
>
>
> Is this a known problem ?
>
>
> 2. I was able to compile thereafter. Built the distribution package, then
> extracted the package, changed the default port to 8888 and created a small
> notebook
>
> The problem occurs when I try to run just a simple command on the
> interpreter:
>
>
> print("Hello World")
>
>
> I see this error in the UI:
>
>
> org.apache.thrift.transport.TTransportException at
>
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) at
>
> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
> at
>
> org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
> at
>
> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69) at
>
> org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_interpret(RemoteInterpreterService.java:220)
> at
>
> org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.interpret(RemoteInterpreterService.java:205)
> at
>
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.interpret(RemoteInterpreter.java:211)
> at
>
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:207) at
> org.apache.zeppelin.scheduler.Job.run(Job.java:170) at
>
> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:304)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
>
> On going through the logs I see the problem:
>
>
>
> 15/09/27 17:43:40 WARN ReliableDeliverySupervisor: Association with remote
> system [akka.tcp://sparkMaster@localhost:7077] has failed, address is now
> gated for [5000] ms. Reason: [Association failed with
> [akka.tcp://sparkMaster@localhost:7077]] Caused by: [Connection refused:
> localhost/127.0.0.1:7077]
>
> 15/09/27 17:43:40 WARN AppClient$ClientEndpoint: Failed to connect to
> master localhost:7077
>
> akka.actor.ActorNotFound: Actor not found for:
> ActorSelection[Anchor(akka.tcp://sparkMaster@localhost:7077/),
> Path(/user/Master)]
>
> at
>
> akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:65)
>
> at
>
> akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:63)
>
> at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
>
> at
>
> akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
>
> at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:73)
>
> at
>
> akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:74)
>
> at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:120)
>
> at
>
> akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:73)
>
> at
> scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:40)
>
> at
> scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:248)
>
> at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:266)
>
> at akka.actor.EmptyLocalActorRef.specialHandle(ActorRef.scala:533)
>
> at akka.actor.DeadLetterActorRef.specialHandle(ActorRef.scala:569)
>
> at akka.actor.DeadLetterActorRef.$bang(ActorRef.scala:559)
>
> at
>
> akka.remote.RemoteActorRefProvider$RemoteDeadLetterActorRef.$bang(RemoteActorRefProvider.scala:87)
>
> at akka.remote.EndpointWriter.postStop(Endpoint.scala:557)
>
> at akka.actor.Actor$class.aroundPostStop(Actor.scala:477)
>
> at akka.remote.EndpointActor.aroundPostStop(Endpoint.scala:411)
>
> at
>
> akka.actor.dungeon.FaultHandling$class.akka$actor$dungeon$FaultHandling$$finishTerminate(FaultHandling.scala:210)
>
> at
> akka.actor.dungeon.FaultHandling$class.terminate(FaultHandling.scala:172)
>
> at akka.actor.ActorCell.terminate(ActorCell.scala:369)
>
> at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:462)
>
> at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478)
>
> at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:263)
>
> at akka.dispatch.Mailbox.run(Mailbox.scala:219)
>
> at
>
> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)
>
> at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
>
> at
>
> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
>
> at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
>
> at
>
> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
>
> 15/09/27 17:44:00 INFO AppClient$ClientEndpoint: Connecting to master
> spark://localhost:7077...
>
> 15/09/27 17:44:00 WARN AppClient$ClientEndpoint: Could not connect to
> localhost:7077: akka.remote.EndpointAssociationException: Association
> failed with [akka.tcp://sparkMaster@localhost:7077]
>
>
>
>
> I tried starting spark 1.5 in standalone mode manually and then adding
> SPARK_HOME / MASTER in conf/zeppelin-env.sh as well , but I get the same
> error.
>
> Please help! I am stuck here for the moment. Any help is highly
> appreciated.
>
>
> Thanks and Best Regards
>
> Yash Datta
>
>
> --
> When events unfold with calm and ease
> When the winds that blow are merely breeze
> Learn from nature, from birds and bees
> Live your life in love, and let joy not cease.
>