You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by Jens Grassel <je...@wegtam.com> on 2017/11/08 13:01:23 UTC
Cannot connect to Flink cluster (TTransportException)
Hi,
we've tried to connect to a flink (1.1.3) cluster (started with
`start-cluster.sh`) from zeppelin 0.7.3 (bin-all).
The cluster is running at localhost:6123 and we configured the
interpreter using:
[X] Connect to existing process
Host : localhost
Port : 6123
But if we try to run flink code from a notebook (for example the
included flink example notebook) we get the following error message:
<---snip--->
ERROR [2017-11-08 13:21:09,289] ({pool-2-thread-2} RemoteInterpreter.java[init]:236) - Failed to create interpreter: org.apache.zeppelin.flink.FlinkInterpreter
ERROR [2017-11-08 13:21:09,290] ({pool-2-thread-2} RemoteInterpreter.java[open]:275) - Failed to initialize interpreter: org.apache.zeppelin.flink.FlinkInterpreter. Remove it from interpreterGroup
ERROR [2017-11-08 13:21:09,300] ({pool-2-thread-2} Job.java[run]:188) - Job failed
org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:450)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:111)
at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:387)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:329)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.thrift.transport.TTransportException
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_getFormType(RemoteInterpreterService.java:337)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.getFormType(RemoteInterpreterService.java:323)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:446)
... 11 more
<---snip--->
Any ideas? We haven't found anything online. :-(
Regards,
Jens
--
CTO, Wegtam GmbH, 08. Nebelung 2017, 13:53
Homepage : https://www.wegtam.com
Those who talk don't know. Those who don't talk, know.
Re: Cannot connect to Flink cluster (TTransportException)
Posted by Jens Grassel <je...@wegtam.com>.
Hi,
On Wed, 08 Nov 2017 14:47:00 +0000
Jeff Zhang <zj...@gmail.com> wrote:
JZ> I think you misunderstand the connecting existing process, it means
JZ> connecting to an existing zeppelin interpreter process. Here it is
JZ> flink interpreter process. But seems
JZ> you are configure it to connect to flink cluster. I am not expert
JZ> in flink, for how to connect to flink cluster, someone else may be
JZ> able to help you.
thanks for the enlightenment. I already guessed that but it is good to
have this confirmed.
Is there a way to cluster zeppelin like flink?
On the other hand it would be great if there are resources how to
connect to an existing flink cluster because that is what we have
already. ;-)
Regards,
Jens
--
CTO, Wegtam GmbH, 09. Nebelung 2017, 08:13
Homepage : https://www.wegtam.com
Go to a movie tonight.
Darkness becomes you.
Re: Cannot connect to Flink cluster (TTransportException)
Posted by Jeff Zhang <zj...@gmail.com>.
I think you misunderstand the connecting existing process, it means
connecting to an existing zeppelin interpreter process. Here it is flink
interpreter process. But seems
you are configure it to connect to flink cluster. I am not expert in flink,
for how to connect to flink cluster, someone else may be able to help you.
Jens Grassel <je...@wegtam.com>于2017年11月8日周三 下午10:00写道：
> On Wed, 08 Nov 2017 13:52:50 +0000
> Jeff Zhang <zj...@gmail.com> wrote:
>
> JZ> zeppelin-interpreter-flink-whatever.log is what I mean
>
> As mentioned this log file is only created by the embedded flink
> interpreter. If the "Connect to existing process" option is used the
> file is not created and even if it exist no data is written to it.
> The file you mentioned only contains data from the embedded flink which
> runs fine but no error messages or warnings regarding the connection to
> an existing flink process.
>
> Regards,
>
> Jens
>
> --
> CTO, Wegtam GmbH, 08. Nebelung 2017, 14:56
> Homepage : https://www.wegtam.com
>
> Real programmers don't write in BASIC. Actually, no programmers write
> in BASIC after reaching puberty.
>
Re: Cannot connect to Flink cluster (TTransportException)
Posted by Jens Grassel <je...@wegtam.com>.
On Wed, 08 Nov 2017 13:52:50 +0000
Jeff Zhang <zj...@gmail.com> wrote:
JZ> zeppelin-interpreter-flink-whatever.log is what I mean
As mentioned this log file is only created by the embedded flink
interpreter. If the "Connect to existing process" option is used the
file is not created and even if it exist no data is written to it.
The file you mentioned only contains data from the embedded flink which
runs fine but no error messages or warnings regarding the connection to
an existing flink process.
Regards,
Jens
--
CTO, Wegtam GmbH, 08. Nebelung 2017, 14:56
Homepage : https://www.wegtam.com
Real programmers don't write in BASIC. Actually, no programmers write
in BASIC after reaching puberty.
Re: Cannot connect to Flink cluster (TTransportException)
Posted by Jeff Zhang <zj...@gmail.com>.
zeppelin-interpreter-flink-whatever.log is what I mean
Jens Grassel <je...@wegtam.com>于2017年11月8日周三 下午9:42写道：
> Hi,
>
> On Wed, 08 Nov 2017 13:23:11 +0000
> Jeff Zhang <zj...@gmail.com> wrote:
>
> JZ> There's another log file which is interpreter log under the logs
> JZ> folder, could you check that file ?
>
> there are just two logfiles in that folder:
>
> zeppelin-whatever.log
> zeppelin-whatever.out
>
> However if we configure the interpreter to use the embedded (included)
> one then a logfile named zeppelin-interpreter-flink-whatever.log is
> created.
>
> Regards,
>
> Jens
>
> --
> CTO, Wegtam GmbH, 08. Nebelung 2017, 14:39
> Homepage : https://www.wegtam.com
>
> Johnson's First Law:
> When any mechanical contrivance fails, it will do so at the
> most inconvenient possible time.
>
Re: Cannot connect to Flink cluster (TTransportException)
Posted by Jens Grassel <je...@wegtam.com>.
Hi,
On Wed, 08 Nov 2017 13:23:11 +0000
Jeff Zhang <zj...@gmail.com> wrote:
JZ> There's another log file which is interpreter log under the logs
JZ> folder, could you check that file ?
there are just two logfiles in that folder:
zeppelin-whatever.log
zeppelin-whatever.out
However if we configure the interpreter to use the embedded (included)
one then a logfile named zeppelin-interpreter-flink-whatever.log is
created.
Regards,
Jens
--
CTO, Wegtam GmbH, 08. Nebelung 2017, 14:39
Homepage : https://www.wegtam.com
Johnson's First Law:
When any mechanical contrivance fails, it will do so at the
most inconvenient possible time.
Re: Cannot connect to Flink cluster (TTransportException)
Posted by Jeff Zhang <zj...@gmail.com>.
There's another log file which is interpreter log under the logs folder,
could you check that file ?
Jens Grassel <je...@wegtam.com>于2017年11月8日周三 下午9:01写道：
> Hi,
>
> we've tried to connect to a flink (1.1.3) cluster (started with
> `start-cluster.sh`) from zeppelin 0.7.3 (bin-all).
> The cluster is running at localhost:6123 and we configured the
> interpreter using:
>
> [X] Connect to existing process
> Host : localhost
> Port : 6123
>
> But if we try to run flink code from a notebook (for example the
> included flink example notebook) we get the following error message:
>
> <---snip--->
>
> ERROR [2017-11-08 13:21:09,289] ({pool-2-thread-2}
> RemoteInterpreter.java[init]:236) - Failed to create interpreter:
> org.apache.zeppelin.flink.FlinkInterpreter
> ERROR [2017-11-08 13:21:09,290] ({pool-2-thread-2}
> RemoteInterpreter.java[open]:275) - Failed to initialize interpreter:
> org.apache.zeppelin.flink.FlinkInterpreter. Remove it from interpreterGroup
> ERROR [2017-11-08 13:21:09,300] ({pool-2-thread-2} Job.java[run]:188) -
> Job failed
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:450)
> at
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:111)
> at
> org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:387)
> at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
> at
> org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:329)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.thrift.transport.TTransportException
> at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> at
> org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
> at
> org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
> at
> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> at
> org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_getFormType(RemoteInterpreterService.java:337)
> at
> org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.getFormType(RemoteInterpreterService.java:323)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:446)
> ... 11 more
>
> <---snip--->
>
> Any ideas? We haven't found anything online. :-(
>
> Regards,
>
> Jens
>
> --
> CTO, Wegtam GmbH, 08. Nebelung 2017, 13:53
> Homepage : https://www.wegtam.com
>
> Those who talk don't know. Those who don't talk, know.
>