You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by Bill Sparks <js...@cray.com> on 2015/11/05 17:03:44 UTC
zeppelin startup warnings
Hello.
I'm trying to run the master branch on CentOS6.5.
My compile completes and the server start successfully, in the log file I see the following warnings. Question, are these expected? Also I'm uses spark with Mesos, I've followed the wiki instruction s but get the following stack trace on a simple inspection of the spark contect variable sc. Note the spark-shell executes successfully. I'm thinking it has something to do with the thrift server not running or … not even getting to executing the spark job.
Regards
-Bill
Startup Warnings (running as root):
[root@maryann4 incubator-zeppelin]# ./bin/zeppelin.sh
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=1024m; support was removed in 8.0
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/mnt/lustre/notebooks/zeppelin/incubator-zeppelin/zeppelin-server/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/mnt/lustre/notebooks/zeppelin/incubator-zeppelin/zeppelin-zengine/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/mnt/lustre/notebooks/zeppelin/incubator-zeppelin/zeppelin-interpreter/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Nov 05, 2015 9:59:13 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.zeppelin.rest
Nov 05, 2015 9:59:13 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.zeppelin.rest.NotebookRestApi
class org.apache.zeppelin.rest.ZeppelinRestApi
class org.apache.zeppelin.rest.InterpreterRestApi
Nov 05, 2015 9:59:13 AM com.sun.jersey.api.core.ScanningResourceConfig init
INFO: No provider classes found.
Nov 05, 2015 9:59:13 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.13 06/29/2012 05:14 PM'
Nov 05, 2015 9:59:14 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A sub-resource method, public javax.ws.rs.core.Response org.apache.zeppelin.rest.NotebookRestApi.createNote(java.lang.String) throws java.io.IOException, with URI template, "/", is treated as a resource method
WARNING: A sub-resource method, public javax.ws.rs.core.Response org.apache.zeppelin.rest.NotebookRestApi.getNotebookList() throws java.io.IOException, with URI template, "/", is treated as a resource method
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.zeppelin.rest.InterpreterRestApi.listInterpreter(java.lang.String), should not consume any entity.
Error via spark interpreter …
java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.thrift.transport.TSocket.open(TSocket.java:182) at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51) at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37) at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60) at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:139) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:137) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:257) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:104) at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:197) at org.apache.zeppelin.scheduler.Job.run(Job.java:170) at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:304) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
--
Jonathan (Bill) Sparks
Software Architecture
Re: zeppelin startup warnings
Posted by moon soo Lee <mo...@apache.org>.
Do you see any other log under logs/ directory?
This also can happen when JAVA_HOME is not set to JDK7.
Thanks,
moon
On Tue, Nov 10, 2015 at 3:33 AM Bill Sparks <js...@cray.com> wrote:
> Sorry I missed your original response.
>
> I export the SPARK_HOME to the install location and set the MASTER
> environment variable to the mesos cluster. Yes we are using mesos as the
> scheduler.
>
> E.g.
>
> export SPARK_HOME=/etc/spark/spark-1.5.1
> export
> MASTER="mesos://zk://maryann2:2181,maryann3:2181,maryann4:2181/mesos"
>
> grep master /etc/spark/spark-1.5.1/conf/spark-defaults.conf
> spark.master
> mesos://zk://maryann2:2181,maryann3:2181,maryann4:2181/mesos
>
>
> Not sure about the following which is (I think) the true error. I don't
> get any interpreter logs, so I'm assuming it's not even getting started.
>
>
> INFO [2015-11-09 12:31:48,272] ({qtp1241236050-99}
> NotebookServer.java[onMessage]:101) - RECEIVE << PING
> ERROR [2015-11-09 12:31:48,648] ({Thread-14}
> RemoteInterpreterEventPoller.java[run]:63) - Can't get
> RemoteInterpreterEvent
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection refused
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
> at
> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:139)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventPoller.run(RemoteInterpreterEventPoller.java:61)
> Caused by: org.apache.thrift.transport.TTransportException:
> java.net.ConnectException: Connection refused
> at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
> ... 7 more
> Caused by: java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:579)
> at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
> ... 8 more
>
>
> --
> Jonathan (Bill) Sparks
> Software Architecture
> Cray Inc.
>
> From: Pablo Torre <pa...@gmail.com>
> Reply-To: "users@zeppelin.incubator.apache.org" <
> users@zeppelin.incubator.apache.org>
> Date: Monday, November 9, 2015 8:35 AM
> To: "users@zeppelin.incubator.apache.org" <
> users@zeppelin.incubator.apache.org>
> Subject: Re: zeppelin startup warnings
>
> Hi Moon,
>
> I have the same problem. I am doing this in a script:
>
> SPARK_PROFILE=1.4
> SPARK_VERSION=1.4.1
> HADOOP_PROFILE=2.4
> HADOOP_VERSION=2.4.0
>
>
>
> curl -sL --retry 3 \
> "
> http://mirrors.ibiblio.org/apache/spark/spark-$SPARK_VERSION/spark-$SPARK_VERSION-bin-hadoop$HADOOP_PROFILE.tgz"
> \
> | gunzip \
> | tar x -C /usr/ \
> && ln -s /usr/spark-$SPARK_VERSION-bin-hadoop$HADOOP_PROFILE /usr/spark \
> && rm -rf /usr/spark/examples \
> && rm /usr/spark/lib/spark-examples*.jar
>
>
>
> and then in my zeppelin configuration file I have this:
>
> export SPARK_HOME="/usr/spark"
>
> 2015-11-08 7:09 GMT+01:00 moon soo Lee <mo...@apache.org>:
>
>> This error may able to occur when SparkContext got some error to connect
>> to Spark cluster.
>>
>> Could you share how you configured Zeppelin?
>> Have you export SPARK_HOME and set 'master' property in 'Interpreter'
>> page?
>>
>> Thanks,
>> moon
>>
>>
>> On Fri, Nov 6, 2015 at 1:03 AM Bill Sparks <js...@cray.com> wrote:
>>
>>> Hello.
>>>
>>> I'm trying to run the master branch on CentOS6.5.
>>>
>>> My compile completes and the server start successfully, in the log file
>>> I see the following warnings. Question, are these expected? Also I'm uses
>>> spark with Mesos, I've followed the wiki instruction s but get the
>>> following stack trace on a simple inspection of the spark contect variable
>>> sc. Note the spark-shell executes successfully. I'm thinking it has
>>> something to do with the thrift server not running or … not even getting to
>>> executing the spark job.
>>>
>>> Regards
>>> -Bill
>>>
>>> Startup Warnings (running as root):
>>>
>>> [root@maryann4 incubator-zeppelin]# ./bin/zeppelin.sh
>>> OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=1024m;
>>> support was removed in 8.0
>>> SLF4J: Class path contains multiple SLF4J bindings.
>>> SLF4J: Found binding in
>>> [jar:file:/mnt/lustre/notebooks/zeppelin/incubator-zeppelin/zeppelin-server/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>>> SLF4J: Found binding in
>>> [jar:file:/mnt/lustre/notebooks/zeppelin/incubator-zeppelin/zeppelin-zengine/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>>> SLF4J: Found binding in
>>> [jar:file:/mnt/lustre/notebooks/zeppelin/incubator-zeppelin/zeppelin-interpreter/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>>> explanation.
>>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>>> Nov 05, 2015 9:59:13 AM com.sun.jersey.api.core.PackagesResourceConfig
>>> init
>>> INFO: Scanning for root resource and provider classes in the packages:
>>> org.apache.zeppelin.rest
>>> Nov 05, 2015 9:59:13 AM com.sun.jersey.api.core.ScanningResourceConfig
>>> logClasses
>>> INFO: Root resource classes found:
>>> class org.apache.zeppelin.rest.NotebookRestApi
>>> class org.apache.zeppelin.rest.ZeppelinRestApi
>>> class org.apache.zeppelin.rest.InterpreterRestApi
>>> Nov 05, 2015 9:59:13 AM com.sun.jersey.api.core.ScanningResourceConfig
>>> init
>>> INFO: No provider classes found.
>>> Nov 05, 2015 9:59:13 AM
>>> com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
>>> INFO: Initiating Jersey application, version 'Jersey: 1.13 06/29/2012
>>> 05:14 PM'
>>> Nov 05, 2015 9:59:14 AM com.sun.jersey.spi.inject.Errors
>>> processErrorMessages
>>> WARNING: The following warnings have been detected with resource and/or
>>> provider classes:
>>> WARNING: A sub-resource method, public javax.ws.rs.core.Response
>>> org.apache.zeppelin.rest.NotebookRestApi.createNote(java.lang.String)
>>> throws java.io.IOException, with URI template, "/", is treated as a
>>> resource method
>>> WARNING: A sub-resource method, public javax.ws.rs.core.Response
>>> org.apache.zeppelin.rest.NotebookRestApi.getNotebookList() throws
>>> java.io.IOException, with URI template, "/", is treated as a resource method
>>> WARNING: A HTTP GET method, public javax.ws.rs.core.Response
>>> org.apache.zeppelin.rest.InterpreterRestApi.listInterpreter(java.lang.String),
>>> should not consume any entity.
>>>
>>> Error via spark interpreter …
>>>
>>> java.net.ConnectException: Connection refused at
>>> java.net.PlainSocketImpl.socketConnect(Native Method) at
>>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
>>> at
>>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
>>> at
>>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
>>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at
>>> java.net.Socket.connect(Socket.java:589) at
>>> org.apache.thrift.transport.TSocket.open(TSocket.java:182) at
>>> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
>>> at
>>> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
>>> at
>>> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
>>> at
>>> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
>>> at
>>> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
>>> at
>>> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
>>> at
>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:139)
>>> at
>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:137)
>>> at
>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:257)
>>> at
>>> org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:104)
>>> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:197) at
>>> org.apache.zeppelin.scheduler.Job.run(Job.java:170) at
>>> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:304)
>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> at java.lang.Thread.run(Thread.java:745)
>>> --
>>> Jonathan (Bill) Sparks
>>> Software Architecture
>>>
>>>
>
>
> --
> Pablo Torre.
> Freelance software engineer and Ruby on Rails developer.
> Oleiros (Coruña)
> *Personal site <http://www.pablotorrerodriguez.com>*
> My blog <http://www.aboutechnologies.com>
>
Re: zeppelin startup warnings
Posted by Bill Sparks <js...@cray.com>.
Sorry I missed your original response.
I export the SPARK_HOME to the install location and set the MASTER environment variable to the mesos cluster. Yes we are using mesos as the scheduler.
E.g.
export SPARK_HOME=/etc/spark/spark-1.5.1
export MASTER="mesos://zk://maryann2:2181,maryann3:2181,maryann4:2181/mesos"
grep master /etc/spark/spark-1.5.1/conf/spark-defaults.conf
spark.master mesos://zk://maryann2:2181,maryann3:2181,maryann4:2181/mesos
Not sure about the following which is (I think) the true error. I don't get any interpreter logs, so I'm assuming it's not even getting started.
INFO [2015-11-09 12:31:48,272] ({qtp1241236050-99} NotebookServer.java[onMessage]:101) - RECEIVE << PING
ERROR [2015-11-09 12:31:48,648] ({Thread-14} RemoteInterpreterEventPoller.java[run]:63) - Can't get RemoteInterpreterEvent
org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:53)
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:139)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventPoller.run(RemoteInterpreterEventPoller.java:61)
Caused by: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
... 7 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
... 8 more
--
Jonathan (Bill) Sparks
Software Architecture
Cray Inc.
From: Pablo Torre <pa...@gmail.com>>
Reply-To: "users@zeppelin.incubator.apache.org<ma...@zeppelin.incubator.apache.org>" <us...@zeppelin.incubator.apache.org>>
Date: Monday, November 9, 2015 8:35 AM
To: "users@zeppelin.incubator.apache.org<ma...@zeppelin.incubator.apache.org>" <us...@zeppelin.incubator.apache.org>>
Subject: Re: zeppelin startup warnings
Hi Moon,
I have the same problem. I am doing this in a script:
SPARK_PROFILE=1.4
SPARK_VERSION=1.4.1
HADOOP_PROFILE=2.4
HADOOP_VERSION=2.4.0
curl -sL --retry 3 \
"http://mirrors.ibiblio.org/apache/spark/spark-$SPARK_VERSION/spark-$SPARK_VERSION-bin-hadoop$HADOOP_PROFILE.tgz" \
| gunzip \
| tar x -C /usr/ \
&& ln -s /usr/spark-$SPARK_VERSION-bin-hadoop$HADOOP_PROFILE /usr/spark \
&& rm -rf /usr/spark/examples \
&& rm /usr/spark/lib/spark-examples*.jar
and then in my zeppelin configuration file I have this:
export SPARK_HOME="/usr/spark"
2015-11-08 7:09 GMT+01:00 moon soo Lee <mo...@apache.org>>:
This error may able to occur when SparkContext got some error to connect to Spark cluster.
Could you share how you configured Zeppelin?
Have you export SPARK_HOME and set 'master' property in 'Interpreter' page?
Thanks,
moon
On Fri, Nov 6, 2015 at 1:03 AM Bill Sparks <js...@cray.com>> wrote:
Hello.
I'm trying to run the master branch on CentOS6.5.
My compile completes and the server start successfully, in the log file I see the following warnings. Question, are these expected? Also I'm uses spark with Mesos, I've followed the wiki instruction s but get the following stack trace on a simple inspection of the spark contect variable sc. Note the spark-shell executes successfully. I'm thinking it has something to do with the thrift server not running or … not even getting to executing the spark job.
Regards
-Bill
Startup Warnings (running as root):
[root@maryann4 incubator-zeppelin]# ./bin/zeppelin.sh
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=1024m; support was removed in 8.0
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/mnt/lustre/notebooks/zeppelin/incubator-zeppelin/zeppelin-server/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/mnt/lustre/notebooks/zeppelin/incubator-zeppelin/zeppelin-zengine/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/mnt/lustre/notebooks/zeppelin/incubator-zeppelin/zeppelin-interpreter/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Nov 05, 2015 9:59:13 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.zeppelin.rest
Nov 05, 2015 9:59:13 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.zeppelin.rest.NotebookRestApi
class org.apache.zeppelin.rest.ZeppelinRestApi
class org.apache.zeppelin.rest.InterpreterRestApi
Nov 05, 2015 9:59:13 AM com.sun.jersey.api.core.ScanningResourceConfig init
INFO: No provider classes found.
Nov 05, 2015 9:59:13 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.13 06/29/2012 05:14 PM'
Nov 05, 2015 9:59:14 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A sub-resource method, public javax.ws.rs.core.Response org.apache.zeppelin.rest.NotebookRestApi.createNote(java.lang.String) throws java.io.IOException, with URI template, "/", is treated as a resource method
WARNING: A sub-resource method, public javax.ws.rs.core.Response org.apache.zeppelin.rest.NotebookRestApi.getNotebookList() throws java.io.IOException, with URI template, "/", is treated as a resource method
WARNING: A HTTP GET method, public javax.ws.rs.core.Response org.apache.zeppelin.rest.InterpreterRestApi.listInterpreter(java.lang.String), should not consume any entity.
Error via spark interpreter …
java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.thrift.transport.TSocket.open(TSocket.java:182) at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51) at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37) at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60) at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:139) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:137) at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:257) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:104) at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:197) at org.apache.zeppelin.scheduler.Job.run(Job.java:170) at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:304) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
--
Jonathan (Bill) Sparks
Software Architecture
--
Pablo Torre.
Freelance software engineer and Ruby on Rails developer.
Oleiros (Coruña)
Personal site<http://www.pablotorrerodriguez.com>
My blog<http://www.aboutechnologies.com>
Re: zeppelin startup warnings
Posted by Pablo Torre <pa...@gmail.com>.
Hi Moon,
I have the same problem. I am doing this in a script:
SPARK_PROFILE=1.4
SPARK_VERSION=1.4.1
HADOOP_PROFILE=2.4
HADOOP_VERSION=2.4.0
curl -sL --retry 3 \
"
http://mirrors.ibiblio.org/apache/spark/spark-$SPARK_VERSION/spark-$SPARK_VERSION-bin-hadoop$HADOOP_PROFILE.tgz"
\
| gunzip \
| tar x -C /usr/ \
&& ln -s /usr/spark-$SPARK_VERSION-bin-hadoop$HADOOP_PROFILE /usr/spark \
&& rm -rf /usr/spark/examples \
&& rm /usr/spark/lib/spark-examples*.jar
and then in my zeppelin configuration file I have this:
export SPARK_HOME="/usr/spark"
2015-11-08 7:09 GMT+01:00 moon soo Lee <mo...@apache.org>:
> This error may able to occur when SparkContext got some error to connect
> to Spark cluster.
>
> Could you share how you configured Zeppelin?
> Have you export SPARK_HOME and set 'master' property in 'Interpreter' page?
>
> Thanks,
> moon
>
>
> On Fri, Nov 6, 2015 at 1:03 AM Bill Sparks <js...@cray.com> wrote:
>
>> Hello.
>>
>> I'm trying to run the master branch on CentOS6.5.
>>
>> My compile completes and the server start successfully, in the log file I
>> see the following warnings. Question, are these expected? Also I'm uses
>> spark with Mesos, I've followed the wiki instruction s but get the
>> following stack trace on a simple inspection of the spark contect variable
>> sc. Note the spark-shell executes successfully. I'm thinking it has
>> something to do with the thrift server not running or … not even getting to
>> executing the spark job.
>>
>> Regards
>> -Bill
>>
>> Startup Warnings (running as root):
>>
>> [root@maryann4 incubator-zeppelin]# ./bin/zeppelin.sh
>> OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=1024m;
>> support was removed in 8.0
>> SLF4J: Class path contains multiple SLF4J bindings.
>> SLF4J: Found binding in
>> [jar:file:/mnt/lustre/notebooks/zeppelin/incubator-zeppelin/zeppelin-server/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: Found binding in
>> [jar:file:/mnt/lustre/notebooks/zeppelin/incubator-zeppelin/zeppelin-zengine/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: Found binding in
>> [jar:file:/mnt/lustre/notebooks/zeppelin/incubator-zeppelin/zeppelin-interpreter/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
>> explanation.
>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>> Nov 05, 2015 9:59:13 AM com.sun.jersey.api.core.PackagesResourceConfig
>> init
>> INFO: Scanning for root resource and provider classes in the packages:
>> org.apache.zeppelin.rest
>> Nov 05, 2015 9:59:13 AM com.sun.jersey.api.core.ScanningResourceConfig
>> logClasses
>> INFO: Root resource classes found:
>> class org.apache.zeppelin.rest.NotebookRestApi
>> class org.apache.zeppelin.rest.ZeppelinRestApi
>> class org.apache.zeppelin.rest.InterpreterRestApi
>> Nov 05, 2015 9:59:13 AM com.sun.jersey.api.core.ScanningResourceConfig
>> init
>> INFO: No provider classes found.
>> Nov 05, 2015 9:59:13 AM
>> com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
>> INFO: Initiating Jersey application, version 'Jersey: 1.13 06/29/2012
>> 05:14 PM'
>> Nov 05, 2015 9:59:14 AM com.sun.jersey.spi.inject.Errors
>> processErrorMessages
>> WARNING: The following warnings have been detected with resource and/or
>> provider classes:
>> WARNING: A sub-resource method, public javax.ws.rs.core.Response
>> org.apache.zeppelin.rest.NotebookRestApi.createNote(java.lang.String)
>> throws java.io.IOException, with URI template, "/", is treated as a
>> resource method
>> WARNING: A sub-resource method, public javax.ws.rs.core.Response
>> org.apache.zeppelin.rest.NotebookRestApi.getNotebookList() throws
>> java.io.IOException, with URI template, "/", is treated as a resource method
>> WARNING: A HTTP GET method, public javax.ws.rs.core.Response
>> org.apache.zeppelin.rest.InterpreterRestApi.listInterpreter(java.lang.String),
>> should not consume any entity.
>>
>> Error via spark interpreter …
>>
>> java.net.ConnectException: Connection refused at
>> java.net.PlainSocketImpl.socketConnect(Native Method) at
>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
>> at
>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
>> at
>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at
>> java.net.Socket.connect(Socket.java:589) at
>> org.apache.thrift.transport.TSocket.open(TSocket.java:182) at
>> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
>> at
>> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
>> at
>> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
>> at
>> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
>> at
>> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
>> at
>> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
>> at
>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:139)
>> at
>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:137)
>> at
>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:257)
>> at
>> org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:104)
>> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:197) at
>> org.apache.zeppelin.scheduler.Job.run(Job.java:170) at
>> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:304)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> --
>> Jonathan (Bill) Sparks
>> Software Architecture
>>
>>
--
Pablo Torre.
Freelance software engineer and Ruby on Rails developer.
Oleiros (Coruña)
*Personal site <http://www.pablotorrerodriguez.com>*
My blog <http://www.aboutechnologies.com>
Re: zeppelin startup warnings
Posted by moon soo Lee <mo...@apache.org>.
This error may able to occur when SparkContext got some error to connect to
Spark cluster.
Could you share how you configured Zeppelin?
Have you export SPARK_HOME and set 'master' property in 'Interpreter' page?
Thanks,
moon
On Fri, Nov 6, 2015 at 1:03 AM Bill Sparks <js...@cray.com> wrote:
> Hello.
>
> I'm trying to run the master branch on CentOS6.5.
>
> My compile completes and the server start successfully, in the log file I
> see the following warnings. Question, are these expected? Also I'm uses
> spark with Mesos, I've followed the wiki instruction s but get the
> following stack trace on a simple inspection of the spark contect variable
> sc. Note the spark-shell executes successfully. I'm thinking it has
> something to do with the thrift server not running or … not even getting to
> executing the spark job.
>
> Regards
> -Bill
>
> Startup Warnings (running as root):
>
> [root@maryann4 incubator-zeppelin]# ./bin/zeppelin.sh
> OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=1024m;
> support was removed in 8.0
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/mnt/lustre/notebooks/zeppelin/incubator-zeppelin/zeppelin-server/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/mnt/lustre/notebooks/zeppelin/incubator-zeppelin/zeppelin-zengine/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/mnt/lustre/notebooks/zeppelin/incubator-zeppelin/zeppelin-interpreter/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> Nov 05, 2015 9:59:13 AM com.sun.jersey.api.core.PackagesResourceConfig init
> INFO: Scanning for root resource and provider classes in the packages:
> org.apache.zeppelin.rest
> Nov 05, 2015 9:59:13 AM com.sun.jersey.api.core.ScanningResourceConfig
> logClasses
> INFO: Root resource classes found:
> class org.apache.zeppelin.rest.NotebookRestApi
> class org.apache.zeppelin.rest.ZeppelinRestApi
> class org.apache.zeppelin.rest.InterpreterRestApi
> Nov 05, 2015 9:59:13 AM com.sun.jersey.api.core.ScanningResourceConfig init
> INFO: No provider classes found.
> Nov 05, 2015 9:59:13 AM
> com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
> INFO: Initiating Jersey application, version 'Jersey: 1.13 06/29/2012
> 05:14 PM'
> Nov 05, 2015 9:59:14 AM com.sun.jersey.spi.inject.Errors
> processErrorMessages
> WARNING: The following warnings have been detected with resource and/or
> provider classes:
> WARNING: A sub-resource method, public javax.ws.rs.core.Response
> org.apache.zeppelin.rest.NotebookRestApi.createNote(java.lang.String)
> throws java.io.IOException, with URI template, "/", is treated as a
> resource method
> WARNING: A sub-resource method, public javax.ws.rs.core.Response
> org.apache.zeppelin.rest.NotebookRestApi.getNotebookList() throws
> java.io.IOException, with URI template, "/", is treated as a resource method
> WARNING: A HTTP GET method, public javax.ws.rs.core.Response
> org.apache.zeppelin.rest.InterpreterRestApi.listInterpreter(java.lang.String),
> should not consume any entity.
>
> Error via spark interpreter …
>
> java.net.ConnectException: Connection refused at
> java.net.PlainSocketImpl.socketConnect(Native Method) at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at
> java.net.Socket.connect(Socket.java:589) at
> org.apache.thrift.transport.TSocket.open(TSocket.java:182) at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
> at
> org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
> at
> org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
> at
> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:139)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:137)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:257)
> at
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:104)
> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:197) at
> org.apache.zeppelin.scheduler.Job.run(Job.java:170) at
> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:304)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> --
> Jonathan (Bill) Sparks
> Software Architecture
>
>