You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by Weizhan Zeng <qg...@gmail.com> on 2017/04/05 12:30:20 UTC
multi user ConnectException
Hi, guys
My zeppelin version is 0.7 and my spark is 2.1 , And i set Run
zeppelin interpreter process as web front end user
<https://zeppelin.apache.org/docs/0.7.1/manual/userimpersonation.html> ,
then i found i cann't use spak context . Is anyone met that or can give
me some advices?
Error LOG:
INFO [2017-04-05 20:23:06,128] ({pool-2-thread-3}
RemoteInterpreterManagedProcess.java[start]:120) - Run interpreter process
[/home/zengweizhan/zeppelin-0.7.0/bin/interpreter.sh, -d,
/home/zengweizhan/zeppelin-0.7.0/interpreter/spark, -p, 28836, -u, hadoop,
-l, /home/zengweizhan/zeppelin-0.7.0/local-repo/2CF9YF8AU]
INFO [2017-04-05 20:23:07,741] ({Exec Default Executor}
RemoteInterpreterManagedProcess.java[onProcessComplete]:164) - Interpreter
process exited 0
WARN [2017-04-05 20:23:36,176] ({Thread-40} ClientFactory.java[create]:57)
- host: localhost ;port: 28836
WARN [2017-04-05 20:23:36,176] ({pool-2-thread-3}
ClientFactory.java[create]:57) - host: localhost ;port: 28836
ERROR [2017-04-05 20:23:36,176] ({Thread-40}
RemoteScheduler.java[getStatus]:256) - Can't get status information
org.apache.zeppelin.interpreter.InterpreterException:
org.apache.thrift.transport.TTransportException: java.net.ConnectException:
Connection refused
at
org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:58)
at
org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:39)
at
org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
at
org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:90)
at
org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPoller.getStatus(RemoteScheduler.java:254)
at
org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPoller.run(RemoteScheduler.java:212)
Caused by: org.apache.thrift.transport.TTransportException:
java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
at
org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:55)
... 8 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
... 9 more
ERROR [2017-04-05 20:23:36,177] ({pool-2-thread-3} Job.java[run]:188) - Job
failed
org.apache.zeppelin.interpreter.InterpreterException:
org.apache.zeppelin.interpreter.InterpreterException:
org.apache.thrift.transport.TTransportException: java.net.ConnectException:
Connection refused
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:211)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:375)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:365)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at
org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:329)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
And I also do by https://github.com/knockdata/spark-highcharts/issues/9 ,
but it still not work .
thanks .
Re: multi user ConnectException
Posted by Weizhan Zeng <qg...@gmail.com>.
I solved it , the reason is I added proxy-user and keytab together
,which cause the remote interperter can't start !
2017-04-06 17:55 GMT+08:00 Raffaele S <r....@gmail.com>:
> Did you properly set $SPARK_HOME in your environment?
>
>
> Raffaele
>
> 2017-04-05 14:30 GMT+02:00 Weizhan Zeng <qg...@gmail.com>:
>
>> Hi, guys
>> My zeppelin version is 0.7 and my spark is 2.1 , And i set Run
>> zeppelin interpreter process as web front end user
>> <https://zeppelin.apache.org/docs/0.7.1/manual/userimpersonation.html> ,
>> then i found i cann't use spak context . Is anyone met that or can give
>> me some advices?
>>
>>
>>
>> Error LOG:
>>
>>
>> INFO [2017-04-05 20:23:06,128] ({pool-2-thread-3}
>> RemoteInterpreterManagedProcess.java[start]:120) - Run interpreter
>> process [/home/zengweizhan/zeppelin-0.7.0/bin/interpreter.sh, -d,
>> /home/zengweizhan/zeppelin-0.7.0/interpreter/spark, -p, 28836, -u,
>> hadoop, -l, /home/zengweizhan/zeppelin-0.7.0/local-repo/2CF9YF8AU]
>>
>> INFO [2017-04-05 20:23:07,741] ({Exec Default Executor}
>> RemoteInterpreterManagedProcess.java[onProcessComplete]:164) -
>> Interpreter process exited 0
>>
>> WARN [2017-04-05 20:23:36,176] ({Thread-40}
>> ClientFactory.java[create]:57) - host: localhost ;port: 28836
>>
>> WARN [2017-04-05 20:23:36,176] ({pool-2-thread-3}
>> ClientFactory.java[create]:57) - host: localhost ;port: 28836
>>
>> ERROR [2017-04-05 20:23:36,176] ({Thread-40}
>> RemoteScheduler.java[getStatus]:256) - Can't get status information
>>
>> org.apache.zeppelin.interpreter.InterpreterException:
>> org.apache.thrift.transport.TTransportException:
>> java.net.ConnectException: Connection refused
>>
>> at org.apache.zeppelin.interpreter.remote.ClientFactory.create(
>> ClientFactory.java:58)
>>
>> at org.apache.zeppelin.interpreter.remote.ClientFactory.create(
>> ClientFactory.java:39)
>>
>> at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(
>> BasePooledObjectFactory.java:60)
>>
>> at org.apache.commons.pool2.impl.GenericObjectPool.create(Gener
>> icObjectPool.java:861)
>>
>> at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject
>> (GenericObjectPool.java:435)
>>
>> at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject
>> (GenericObjectPool.java:363)
>>
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProc
>> ess.getClient(RemoteInterpreterProcess.java:90)
>>
>> at org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPolle
>> r.getStatus(RemoteScheduler.java:254)
>>
>> at org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPolle
>> r.run(RemoteScheduler.java:212)
>>
>> Caused by: org.apache.thrift.transport.TTransportException:
>> java.net.ConnectException: Connection refused
>>
>> at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
>>
>> at org.apache.zeppelin.interpreter.remote.ClientFactory.create(
>> ClientFactory.java:55)
>>
>> ... 8 more
>>
>> Caused by: java.net.ConnectException: Connection refused
>>
>> at java.net.PlainSocketImpl.socketConnect(Native Method)
>>
>> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSock
>> etImpl.java:339)
>>
>> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPl
>> ainSocketImpl.java:200)
>>
>> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocket
>> Impl.java:182)
>>
>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>>
>> at java.net.Socket.connect(Socket.java:579)
>>
>> at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
>>
>> ... 9 more
>>
>> ERROR [2017-04-05 20:23:36,177] ({pool-2-thread-3} Job.java[run]:188) -
>> Job failed
>>
>> org.apache.zeppelin.interpreter.InterpreterException:
>> org.apache.zeppelin.interpreter.InterpreterException:
>> org.apache.thrift.transport.TTransportException:
>> java.net.ConnectException: Connection refused
>>
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.
>> init(RemoteInterpreter.java:211)
>>
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.
>> getFormType(RemoteInterpreter.java:375)
>>
>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormT
>> ype(LazyOpenInterpreter.java:105)
>>
>> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:365)
>>
>> at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
>>
>> at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(
>> RemoteScheduler.java:329)
>>
>> at java.util.concurrent.Executors$RunnableAdapter.call(
>> Executors.java:471)
>>
>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu
>> tureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>>
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu
>> tureTask.run(ScheduledThreadPoolExecutor.java:292)
>>
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>> Executor.java:1145)
>>
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>> lExecutor.java:615)
>>
>> at java.lang.Thread.run(Thread.java:745)
>>
>>
>> And I also do by https://github.com/knockdata/spark-highcharts/issues/9 ,
>> but it still not work .
>>
>>
>> thanks .
>>
>>
>>
>
>
Re: multi user ConnectException
Posted by Raffaele S <r....@gmail.com>.
Did you properly set $SPARK_HOME in your environment?
Raffaele
2017-04-05 14:30 GMT+02:00 Weizhan Zeng <qg...@gmail.com>:
> Hi, guys
> My zeppelin version is 0.7 and my spark is 2.1 , And i set Run
> zeppelin interpreter process as web front end user
> <https://zeppelin.apache.org/docs/0.7.1/manual/userimpersonation.html> ,
> then i found i cann't use spak context . Is anyone met that or can give
> me some advices?
>
>
>
> Error LOG:
>
>
> INFO [2017-04-05 20:23:06,128] ({pool-2-thread-3}
> RemoteInterpreterManagedProcess.java[start]:120) - Run interpreter
> process [/home/zengweizhan/zeppelin-0.7.0/bin/interpreter.sh, -d,
> /home/zengweizhan/zeppelin-0.7.0/interpreter/spark, -p, 28836, -u,
> hadoop, -l, /home/zengweizhan/zeppelin-0.7.0/local-repo/2CF9YF8AU]
>
> INFO [2017-04-05 20:23:07,741] ({Exec Default Executor}
> RemoteInterpreterManagedProcess.java[onProcessComplete]:164) -
> Interpreter process exited 0
>
> WARN [2017-04-05 20:23:36,176] ({Thread-40}
> ClientFactory.java[create]:57) - host: localhost ;port: 28836
>
> WARN [2017-04-05 20:23:36,176] ({pool-2-thread-3}
> ClientFactory.java[create]:57) - host: localhost ;port: 28836
>
> ERROR [2017-04-05 20:23:36,176] ({Thread-40} RemoteScheduler.java[getStatus]:256)
> - Can't get status information
>
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException:
> java.net.ConnectException: Connection refused
>
> at org.apache.zeppelin.interpreter.remote.ClientFactory.create(
> ClientFactory.java:58)
>
> at org.apache.zeppelin.interpreter.remote.ClientFactory.create(
> ClientFactory.java:39)
>
> at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(
> BasePooledObjectFactory.java:60)
>
> at org.apache.commons.pool2.impl.GenericObjectPool.create(
> GenericObjectPool.java:861)
>
> at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(
> GenericObjectPool.java:435)
>
> at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(
> GenericObjectPool.java:363)
>
> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.
> getClient(RemoteInterpreterProcess.java:90)
>
> at org.apache.zeppelin.scheduler.RemoteScheduler$
> JobStatusPoller.getStatus(RemoteScheduler.java:254)
>
> at org.apache.zeppelin.scheduler.RemoteScheduler$JobStatusPoller.run(
> RemoteScheduler.java:212)
>
> Caused by: org.apache.thrift.transport.TTransportException:
> java.net.ConnectException: Connection refused
>
> at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
>
> at org.apache.zeppelin.interpreter.remote.ClientFactory.create(
> ClientFactory.java:55)
>
> ... 8 more
>
> Caused by: java.net.ConnectException: Connection refused
>
> at java.net.PlainSocketImpl.socketConnect(Native Method)
>
> at java.net.AbstractPlainSocketImpl.doConnect(
> AbstractPlainSocketImpl.java:339)
>
> at java.net.AbstractPlainSocketImpl.connectToAddress(
> AbstractPlainSocketImpl.java:200)
>
> at java.net.AbstractPlainSocketImpl.connect(
> AbstractPlainSocketImpl.java:182)
>
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
>
> at java.net.Socket.connect(Socket.java:579)
>
> at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
>
> ... 9 more
>
> ERROR [2017-04-05 20:23:36,177] ({pool-2-thread-3} Job.java[run]:188) -
> Job failed
>
> org.apache.zeppelin.interpreter.InterpreterException: org.apache.zeppelin.
> interpreter.InterpreterException: org.apache.thrift.transport.TTransportException:
> java.net.ConnectException: Connection refused
>
> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(
> RemoteInterpreter.java:211)
>
> at org.apache.zeppelin.interpreter.remote.
> RemoteInterpreter.getFormType(RemoteInterpreter.java:375)
>
> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(
> LazyOpenInterpreter.java:105)
>
> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:365)
>
> at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
>
> at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(
> RemoteScheduler.java:329)
>
> at java.util.concurrent.Executors$RunnableAdapter.
> call(Executors.java:471)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>
> at java.util.concurrent.ScheduledThreadPoolExecutor$
> ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>
> at java.util.concurrent.ScheduledThreadPoolExecutor$
> ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1145)
>
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:615)
>
> at java.lang.Thread.run(Thread.java:745)
>
>
> And I also do by https://github.com/knockdata/spark-highcharts/issues/9 ,
> but it still not work .
>
>
> thanks .
>
>
>