You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by Jihun choi <ji...@kongstudios.net> on 2015/05/25 07:23:56 UTC

Using Zeppelin with EC2 spark cluster

Hi,
I'm a noob user with Zeppelin.

In order to do distributed computing, I've installed Spark cluster on EC2
(with master & 5 slaves) and make Zeppelin server to refer the Spark
master.
I've tried to set master url at 'interpreter' menu in Zeppelin as following:
spark://ec2-52-24-213-147.us-west-2.compute.amazonaws.com:7077 (Spark
master url which is running on AWS with 5 slaves)
And I also set export MASTER= in 'zeppelin-env.sh' as same.

However, when I run script on the notebook, the following error is occured:
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.interpret(RemoteInterpreter.java:221)
org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:212)
org.apache.zeppelin.scheduler.Job.run(Job.java:170)
org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:296)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
java.util.concurrent.FutureTask.run(FutureTask.java:262)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
java.lang.Thread.run(Thread.java:745)

Could you give me some hints please?

Best,

Re: Using Zeppelin with EC2 spark cluster

Posted by Alexander Bezzubov <ab...@nflabs.com>.
Hi Jihun,

thanks for your interest in Zeppelin!
It looks like zeppelin server having troubles connecting to spark
interpreter process.

Could you please post logs/zeppelin-* server and interpreter logs?

Thanks.

On Mon, May 25, 2015 at 2:23 PM, Jihun choi <ji...@kongstudios.net>
wrote:

> Hi,
> I'm a noob user with Zeppelin.
>
> In order to do distributed computing, I've installed Spark cluster on EC2
> (with master & 5 slaves) and make Zeppelin server to refer the Spark
> master.
> I've tried to set master url at 'interpreter' menu in Zeppelin as
> following:
> spark://ec2-52-24-213-147.us-west-2.compute.amazonaws.com:7077 (Spark
> master url which is running on AWS with 5 slaves)
> And I also set export MASTER= in 'zeppelin-env.sh' as same.
>
> However, when I run script on the notebook, the following error is occured:
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.interpret(RemoteInterpreter.java:221)
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
> org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:212)
> org.apache.zeppelin.scheduler.Job.run(Job.java:170)
> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:296)
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> java.util.concurrent.FutureTask.run(FutureTask.java:262)
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> java.lang.Thread.run(Thread.java:745)
>
> Could you give me some hints please?
>
> Best,
>



-- 
--
Kind regards,
Alexander.