You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by Sriram Narayanamoorthy <ns...@gmail.com> on 2016/10/05 05:04:31 UTC
Zeppelin 0.6.1 not connecting to Spark 2.0 on Windows
Hi all,
Hope this is the right forum to ask about this issue I am facing. I am
trying to setup Zeppelin 0.6.1 to with a local Spark 2.0 instance on my
Windows laptop. When I trying running a simple Scala statement *val a = 1 *the
interpreter errors out and I am getting the java stack trace provided
below. It seems like Spark interpreter is not able to connect to local
spark instance.
Also, I am seeing a few warnings and a message "The input line is too long"
in the command line.
Any inputs to fix this greatly appreciated!
Best,
-Sriram
*Java trace:*
java.net.ConnectException: Connection refused: connect at
java.net.DualStackPlainSocketImpl.connect0(Native Method) at
java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source) at
java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at
java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at
java.net.AbstractPlainSocketImpl.connect(Unknown Source) at
java.net.PlainSocketImpl.connect(Unknown Source) at
java.net.SocksSocketImpl.connect(Unknown Source) at
java.net.Socket.connect(Unknown Source) at
org.apache.thrift.transport.TSocket.open(TSocket.java:182) at
org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
at
org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
at
org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
at
org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:189)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:163)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:328)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:260) at
org.apache.zeppelin.scheduler.Job.run(Job.java:176) at
org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:328)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at
java.util.concurrent.FutureTask.run(Unknown Source) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown
Source) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source) at java.lang.Thread.run(Unknown Source)
*Output on the shell:*
Oct 04, 2016 9:41:22 PM com.sun.jersey.api.core.ScanningResourceConfig init
INFO: No provider classes found.
Oct 04, 2016 9:41:22 PM
com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.13 06/29/2012 05:14
PM'
Oct 04, 2016 9:41:22 PM com.sun.jersey.spi.inject.Errors
processErrorMessages
WARNING: The following warnings have been detected with resource and/or
provider classes:
WARNING: A HTTP GET method, public javax.ws.rs.core.Response
org.apache.zeppelin.rest.InterpreterRestApi.listInterpreter(java.lang.String),
should not consume any entity.
WARNING: A sub-resource method, public javax.ws.rs.core.Response
org.apache.zeppelin.rest.NotebookRestApi.createNote(java.lang.String)
throws java.io.IOException, with URI template, "/", is treated as a
resource method
WARNING: A sub-resource method, public javax.ws.rs.core.Response
org.apache.zeppelin.rest.NotebookRestApi.getNotebookList()
throws java.io.IOException, with URI template, "/", is treated as a
resource method
The input line is too long.
--
*Sriram Narayanamoorthy*
Re: Zeppelin 0.6.1 not connecting to Spark 2.0 on Windows
Posted by VIVEK NARAYANASETTY <vi...@gmail.com>.
Hi,
I am also facing the similar issue.
I am using zeppelin 0.6.0 pre-built binary version with spark 1.6.2 on my
windows laptop and everything is working fine. After upgrading to spark
2.0, zeppelin is giving me below error in UI. Also, Please find the
attached log file for more details.
Code:
=====
%spark
var test = "Hello"
Zeppelin UI Error:
=============
java.net.ConnectException: Connection refused: connect at
java.net.DualStackPlainSocketImpl.connect0(Native Method) at
java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) at
java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at
java.net.Socket.connect(Socket.java:579) at
org.apache.thrift.transport.TSocket.open(TSocket.java:182) at
org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
at
org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
at
org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
at
org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:184)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:163)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:328)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:260) at
org.apache.zeppelin.scheduler.Job.run(Job.java:176) at
org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:328)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
On Sat, Oct 15, 2016 at 8:11 AM, Xi Shen <da...@gmail.com> wrote:
> Hi,
>
> By "local Spark 2.0 instance", did you mean a standalone cluster on your
> local? So you updated the "master", and "deploy-mode" interpreter
> configuration?
>
> I had this problem as well. But I found if you set the
> "spark.executor.memory" to "512m", and make sure your machine has
> sufficient physical memory, it is less likely to hit this problem.
>
> If this is not your case, please share your ./logs/zeppelin*spark*.log, so
> people can help you take a look.
>
>
> On Fri, Oct 14, 2016 at 11:24 PM soralee <so...@nflabs.com> wrote:
>
>> Hi
>> Thanks for your kind answers :)
>>
>> I downloaded file through your url link and I've tested.
>> This test have executed a "bin/zeppelin.cmd" command in cmd window.
>> But, I didn't find your problem.
>>
>> Did you get the other error logs?
>> This "logs" folder is under zeppelin's home.
>>
>> If not, when you execute your spark paragraph(such as "var a=1"), please
>> let
>> me know the whole error logs.
>>
>>
>> Thanks,
>> Sora
>>
>>
>>
>> --
>> View this message in context: http://apache-zeppelin-users-
>> incubating-mailing-list.75479.x6.nabble.com/Zeppelin-0-6-1-
>> not-connecting-to-Spark-2-0-on-Windows-tp4270p4359.html
>> Sent from the Apache Zeppelin Users (incubating) mailing list mailing
>> list archive at Nabble.com.
>>
> --
>
>
> Thanks,
> David S.
>
--
Thanks & Regards
*Vivek Narayanasetty*
*Go Green: Think before you print this e-mail or its attachment. You can
save paper if you do not really need to print.*
Re: Zeppelin 0.6.1 not connecting to Spark 2.0 on Windows
Posted by Xi Shen <da...@gmail.com>.
Hi,
By "local Spark 2.0 instance", did you mean a standalone cluster on your
local? So you updated the "master", and "deploy-mode" interpreter
configuration?
I had this problem as well. But I found if you set the
"spark.executor.memory" to "512m", and make sure your machine has
sufficient physical memory, it is less likely to hit this problem.
If this is not your case, please share your ./logs/zeppelin*spark*.log, so
people can help you take a look.
On Fri, Oct 14, 2016 at 11:24 PM soralee <so...@nflabs.com> wrote:
> Hi
> Thanks for your kind answers :)
>
> I downloaded file through your url link and I've tested.
> This test have executed a "bin/zeppelin.cmd" command in cmd window.
> But, I didn't find your problem.
>
> Did you get the other error logs?
> This "logs" folder is under zeppelin's home.
>
> If not, when you execute your spark paragraph(such as "var a=1"), please
> let
> me know the whole error logs.
>
>
> Thanks,
> Sora
>
>
>
> --
> View this message in context:
> http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Zeppelin-0-6-1-not-connecting-to-Spark-2-0-on-Windows-tp4270p4359.html
> Sent from the Apache Zeppelin Users (incubating) mailing list mailing list
> archive at Nabble.com.
>
--
Thanks,
David S.
Re: Zeppelin 0.6.1 not connecting to Spark 2.0 on Windows
Posted by soralee <so...@nflabs.com>.
Hi
Thanks for your kind answers :)
I downloaded file through your url link and I've tested.
This test have executed a "bin/zeppelin.cmd" command in cmd window.
But, I didn't find your problem.
Did you get the other error logs?
This "logs" folder is under zeppelin's home.
If not, when you execute your spark paragraph(such as "var a=1"), please let
me know the whole error logs.
Thanks,
Sora
--
View this message in context: http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Zeppelin-0-6-1-not-connecting-to-Spark-2-0-on-Windows-tp4270p4359.html
Sent from the Apache Zeppelin Users (incubating) mailing list mailing list archive at Nabble.com.
Re: Zeppelin 0.6.1 not connecting to Spark 2.0 on Windows
Posted by Sriram Narayanamoorthy <ns...@gmail.com>.
Hi, thanks for the response. I am using pre-built binary for windows with
all interpreters. I did not build it myself.
The link i downloaded it from is:
http://www-eu.apache.org/dist/zeppelin/zeppelin-0.6.1/zeppelin-0.6.1-bin-all.tgz
Let me know if there is any additional information that I can provide.
-Sriram
On Fri, Oct 14, 2016 at 12:18 AM soralee <so...@nflabs.com> wrote:
> Hi, Sriram Narayanamoorthy.
> Thanks for sharing that problem.
>
> could you share your used build commend?
>
> Thanks,
> Sora
>
>
>
> --
> View this message in context:
> http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Zeppelin-0-6-1-not-connecting-to-Spark-2-0-on-Windows-tp4270p4357.html
> Sent from the Apache Zeppelin Users (incubating) mailing list mailing list
> archive at Nabble.com.
>
Re: Zeppelin 0.6.1 not connecting to Spark 2.0 on Windows
Posted by soralee <so...@nflabs.com>.
Hi, Sriram Narayanamoorthy.
Thanks for sharing that problem.
could you share your used build commend?
Thanks,
Sora
--
View this message in context: http://apache-zeppelin-users-incubating-mailing-list.75479.x6.nabble.com/Zeppelin-0-6-1-not-connecting-to-Spark-2-0-on-Windows-tp4270p4357.html
Sent from the Apache Zeppelin Users (incubating) mailing list mailing list archive at Nabble.com.