You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by Abul Basar <ab...@einext.com> on 2016/11/16 09:23:25 UTC
Re: FW: Issue with Zeppelin setup on Datastax-Spark
I am facing a similar issue while using Spark R.
My environment:
- Spark 2.0.2
- Zeppelin: 0.6.2
- Java 1.8.0_111
- R: 3.3.1
SPARK_HOME is set. I am trying to run a simple paragraph.
%r
print("hello ...")
I get the following exception
*Interpreter Log*
> # getZeppelinR
> .zeppelinR = SparkR:::callJStatic("org.apache.zeppelin.spark.ZeppelinR",
"getZeppelinR", hashCode)
at org.apache.zeppelin.spark.ZeppelinR.waitForRScriptInitialized(
ZeppelinR.java:295)
at org.apache.zeppelin.spark.ZeppelinR.request(ZeppelinR.java:235)
at org.apache.zeppelin.spark.ZeppelinR.eval(ZeppelinR.java:183)
at org.apache.zeppelin.spark.ZeppelinR.open(ZeppelinR.java:172)
at org.apache.zeppelin.spark.SparkRInterpreter.open(
SparkRInterpreter.java:85)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(
LazyOpenInterpreter.java:69)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgress(
LazyOpenInterpreter.java:110)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer.
getProgress(RemoteInterpreterServer.java:404)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$
Processor$getProgress.getResult(RemoteInterpreterService.java:1509)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$
Processor$getProgress.getResult(RemoteInterpreterService.java:1494)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(
TThreadPoolServer.java:285)
at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
*Zeppelin Log*
ERROR [2016-11-16 14:42:05,664] ({Thread-377} JobProgressPoller.java[run]:54)
- Can not get or update progress
org.apache.zeppelin.interpreter.InterpreterException:
org.apache.thrift.transport.TTransportException
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getProgress(
RemoteInterpreter.java:373)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgress(
LazyOpenInterpreter.java:111)
at org.apache.zeppelin.notebook.Paragraph.progress(Paragraph.java:237)
at org.apache.zeppelin.scheduler.JobProgressPoller.run(
JobProgressPoller.java:51)
Caused by: org.apache.thrift.transport.TTransportException
at org.apache.thrift.transport.TIOStreamTransport.read(
TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(
TBinaryProtocol.java:429)
at org.apache.thrift.protocol.TBinaryProtocol.readI32(
TBinaryProtocol.java:318)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(
TBinaryProtocol.java:219)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$
Client.recv_getProgress(RemoteInterpreterService.java:296)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$
Client.getProgress(RemoteInterpreterService.java:281)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getProgress(
RemoteInterpreter.java:370)
... 3 more
When I tried run R commands using sparkR by launching
$SPARK_HOME/bin/sparkR, it works.
Would much appreciate any help.
Thanks!
Abul
On Fri, Sep 16, 2016 at 5:36 PM, Hyung Sung Shim <hs...@nflabs.com> wrote:
> Hello.
> I think that `zeppelin.interpreter.host` and `zeppelin.interpreter.port`
> values on Interpreter setting UI should be removed because Interpreter
> will automatically be launched when you run a paragraph.
>
> 2016-09-16 3:56 GMT+09:00 Arpan Saha <Ar...@mindtree.com>:
>
>> Yes. It is a zeppelin log. Not spark interpreter log. Can you please tell
>> me where can I get spark interpreter log? I don't find anything under
>> zeppelin/logs/ path.
>>
>> Thanks,
>> Arpan
>> ________________________________________
>> From: DuyHai Doan [doanduyhai@gmail.com]
>> Sent: Thursday, September 15, 2016 11:20 AM
>> To: users@zeppelin.apache.org
>> Subject: Re: FW: Issue with Zeppelin setup on Datastax-Spark
>>
>> Is it Zeppelin log or Spark interpreter log ? I guess we need also the
>> spark interpreter log to understand what is breaking Spark under the hood.
>> org.apache.thrift.transport.TTransportException is just technical
>> exception on inter-jvm communication
>>
>> On Thu, Sep 15, 2016 at 7:13 PM, Arpan Saha <Arpan.Saha@mindtree.com
>> <ma...@mindtree.com>> wrote:
>>
>> Hi,
>>
>> Please find below full stack trace.
>>
>> ERROR [2016-09-15 16:49:15,117] ({pool-1-thread-2}
>> RemoteInterpreter.java[init]:182) - Failed to create interpreter:
>> org.apache.zeppelin.spark.SparkInterpreter
>> ERROR [2016-09-15 16:49:15,118] ({pool-1-thread-2} Job.java[run]:189) -
>> Job failed
>> org.apache.zeppelin.interpreter.InterpreterException:
>> org.apache.thrift.transport.TTransportException
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.ini
>> t(RemoteInterpreter.java:183)
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>> FormType(RemoteInterpreter.java:328)
>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormT
>> ype(LazyOpenInterpreter.java:105)
>> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java
>> :260)
>> at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
>> at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(
>> RemoteScheduler.java:328)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executor
>> s.java:511)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu
>> tureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu
>> tureTask.run(ScheduledThreadPoolExecutor.java:293)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>> Executor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>> lExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: org.apache.thrift.transport.TTransportException
>> at org.apache.thrift.transport.TIOStreamTransport.read(TIOStrea
>> mTransport.java:132)
>> at org.apache.thrift.transport.TTransport.readAll(TTransport.ja
>> va:86)
>> at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryPr
>> otocol.java:429)
>> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryPr
>> otocol.java:318)
>> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(
>> TBinaryProtocol.java:219)
>> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.
>> java:69)
>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>> ice$Client.recv_createInterpreter(RemoteInterpreterService.java:184)
>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>> ice$Client.createInterpreter(RemoteInterpreterService.java:168)
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.ini
>> t(RemoteInterpreter.java:172)
>> ... 12 more
>> ERROR [2016-09-15 16:49:15,118] ({Thread-18}
>> RemoteInterpreterEventPoller.java[run]:88) - Can't get
>> RemoteInterpreterEvent
>>
>> Thanks,
>> Arpan.
>>
>> ________________________________________
>> From: DuyHai Doan [doanduyhai@gmail.com<ma...@gmail.com>]
>> Sent: Thursday, September 15, 2016 4:29 AM
>> To: users@zeppelin.apache.org<ma...@zeppelin.apache.org>
>> Subject: Re: Issue with Zeppelin setup on Datastax-Spark
>>
>> Can you provide the complete stack trace ?
>>
>> Also, worth reading my blog post about integrating Spark/Zeppelin/C* or
>> DSE : http://www.doanduyhai.com/blog/?p=2325
>>
>> On Thu, Sep 15, 2016 at 5:17 AM, Arpan Saha <Arpan.Saha@mindtree.com
>> <ma...@mindtree.com><mailto:Arpan.Saha@mindtree.com<mailto:
>> Arpan.Saha@mindtree.com>>> wrote:
>> Hi Team,
>>
>> I am trying to integrate Zeppelin 0.6.0 with DataStax 4.8.8 (which has
>> Spark 1.4.2). After I configured following properties in zeppelin-env.sh
>> when I start zeppelin daemon it started and in the browser I can see
>> zeppelin is running but when I am trying to execute spark query in the
>> notebook it is throwing below Error. Could you please help me to solve this
>> issue.
>>
>> export JAVA_HOME=<JAVA Home Path>
>> export SPARK_HOME=/etc/dse/spark
>> export HADOOP_CONF_DIR=/etc/dse/hadoop
>> export MASTER=spark://<masterhost>:7077
>>
>> Also, added/updated below properties in the Spark Interpreter screen of
>> Zeppelin UI.
>>
>> Master
>> spark.app.name<http://spark.app.name><http://spark.app.name>
>> spark.cassandra.auth.password
>> spark.cassandra.auth.username
>> spark.cassandra.connection.host
>> spark.cores.max
>> spark.executor.memory
>> zeppelin.interpreter.host
>> zeppelin.interpreter.port
>>
>> Trying to execute below statement in the notebook:
>> %spark
>> sc.version
>>
>> Below Error is getting:
>> java.net.SocketException: Connection reset at
>> java.net.SocketInputStream.read(SocketInputStream.java:209) at
>> java.net.SocketInputStream.read(SocketInputStream.java:141) at
>> java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at
>> java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at
>> java.io.BufferedInputStream.read(BufferedInputStream.java:345) at
>> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) at
>> org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
>> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
>> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
>> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>> ice$Client.recv_createInterpreter(RemoteInterpreterService.java:184) at
>> org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>> ice$Client.createInterpreter(RemoteInterpreterService.java:168) at
>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:172)
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>> FormType(RemoteInterpreter.java:328) at org.apache.zeppelin.interprete
>> r.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
>>
>> users-subscribe@zeppelin.apache.org<mailto:users-subscribe@z
>> eppelin.apache.org><mailto:users-subscribe@zeppelin.apache.org<mailto:
>> users-subscribe@zeppelin.apache.org>>
>>
>> Thanks,
>> Arpan.
>>
>> ________________________________
>> Notice: This e-mail message has been sent by an employee or contractor of
>> American Express Global Business Travel (“GBT”), a joint venture that is
>> not wholly-owned by American Express Company or any of its subsidiaries
>> (“American Express”). E-mail from GBT employees and contractors may
>> continue to utilize an American Express domain for some period; however
>> American Express is not responsible for the content of this email, which is
>> controlled by GBT. “American Express Global Business Travel”, “American
>> Express” and the American Express Logo are trademarks of American Express
>> and are used by GBT under limited license.
>> ________________________________
>> Important : Cet email a été envoyé par un employé ou un sous-traitant
>> d’American Express Global Business Travel (« GBT »), une joint-venture qui
>> n’est pas intégralement détenue par American Express Company ou l’une de
>> ses filiales (« American Express »). Les emails des employés et
>> sous-traitants de GBT peuvent continuer de comporter un nom de domaine
>> d’American Express pour une certaine période ; toutefois, American Express
>> n’est pas responsable du contenu de cet email, qui est contrôlé par GBT. «
>> American Express Global Business Travel », « American Express » et le logo
>> American Express sont des marques d’American Express et sont utilisées par
>> GBT aux termes d’une licence limitée.
>>
>> ________________________________
>>
>> http://www.mindtree.com/email/disclaimer.html
>>
>>
>>
>
Re: FW: Issue with Zeppelin setup on Datastax-Spark
Posted by Abul Basar <ab...@einext.com>.
I figured what was the issue. I did not install knitr package in R. After
installing the package in R, i restarted the zeppelin and all looks good
now.
On Wed, Nov 16, 2016 at 4:09 PM, Abul Basar <ab...@einext.com> wrote:
> Hello DuyHai,
>
> Original problem reported by Arpan Saha is related to Datastax. I am using
> Spark + Zeppelin. Below is the configuration.
>
>
> - Spark 2.0.2
> - Zeppelin: 0.6.2
> - Java 1.8.0_111
> - R: 3.3.1
>
> Thanks,
> Abul
>
> On Wed, Nov 16, 2016 at 3:44 PM, DuyHai Doan <do...@gmail.com> wrote:
>
>> I recommend downloading my pre-built Zeppelin for Datastax. Shared folder
>> link: https://drive.google.com/folderview?id=0B6wR2aj4Cb6wQ01aR3ItR0xUNms
>>
>> On Wed, Nov 16, 2016 at 11:13 AM, DuyHai Doan <do...@gmail.com>
>> wrote:
>>
>>> Ok I understand why you have issue.
>>>
>>> You are using Spark 2.0.2 and latest Datastax 5.0.3 is still using Spark
>>> version 1.6.X
>>>
>>>
>>>
>>> On Wed, Nov 16, 2016 at 10:23 AM, Abul Basar <ab...@einext.com> wrote:
>>>
>>>> I am facing a similar issue while using Spark R.
>>>>
>>>> My environment:
>>>>
>>>> - Spark 2.0.2
>>>> - Zeppelin: 0.6.2
>>>> - Java 1.8.0_111
>>>> - R: 3.3.1
>>>>
>>>>
>>>> SPARK_HOME is set. I am trying to run a simple paragraph.
>>>>
>>>> %r
>>>> print("hello ...")
>>>>
>>>> I get the following exception
>>>>
>>>> *Interpreter Log*
>>>> > # getZeppelinR
>>>>
>>>> > .zeppelinR = SparkR:::callJStatic("org.apache.zeppelin.spark.ZeppelinR",
>>>> "getZeppelinR", hashCode)
>>>>
>>>> at org.apache.zeppelin.spark.ZeppelinR.waitForRScriptInitialize
>>>> d(ZeppelinR.java:295)
>>>> at org.apache.zeppelin.spark.ZeppelinR.request(ZeppelinR.java:235)
>>>> at org.apache.zeppelin.spark.ZeppelinR.eval(ZeppelinR.java:183)
>>>> at org.apache.zeppelin.spark.ZeppelinR.open(ZeppelinR.java:172)
>>>> at org.apache.zeppelin.spark.SparkRInterpreter.open(SparkRInter
>>>> preter.java:85)
>>>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(Laz
>>>> yOpenInterpreter.java:69)
>>>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgr
>>>> ess(LazyOpenInterpreter.java:110)
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServ
>>>> er.getProgress(RemoteInterpreterServer.java:404)
>>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>>> ice$Processor$getProgress.getResult(RemoteInterpreterService.java:1509)
>>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>>> ice$Processor$getProgress.getResult(RemoteInterpreterService.java:1494)
>>>> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>>>> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>>>> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run
>>>> (TThreadPoolServer.java:285)
>>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>>>> Executor.java:1142)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>>>> lExecutor.java:617)
>>>> at java.lang.Thread.run(Thread.java:745)
>>>>
>>>>
>>>> *Zeppelin Log*
>>>> ERROR [2016-11-16 14:42:05,664] ({Thread-377}
>>>> JobProgressPoller.java[run]:54) - Can not get or update progress
>>>> org.apache.zeppelin.interpreter.InterpreterException:
>>>> org.apache.thrift.transport.TTransportException
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>>>> Progress(RemoteInterpreter.java:373)
>>>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgr
>>>> ess(LazyOpenInterpreter.java:111)
>>>> at org.apache.zeppelin.notebook.Paragraph.progress(Paragraph.java:237)
>>>> at org.apache.zeppelin.scheduler.JobProgressPoller.run(JobProgr
>>>> essPoller.java:51)
>>>> Caused by: org.apache.thrift.transport.TTransportException
>>>> at org.apache.thrift.transport.TIOStreamTransport.read(TIOStrea
>>>> mTransport.java:132)
>>>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
>>>> at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryPr
>>>> otocol.java:429)
>>>> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryPr
>>>> otocol.java:318)
>>>> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(
>>>> TBinaryProtocol.java:219)
>>>> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
>>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>>> ice$Client.recv_getProgress(RemoteInterpreterService.java:296)
>>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>>> ice$Client.getProgress(RemoteInterpreterService.java:281)
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>>>> Progress(RemoteInterpreter.java:370)
>>>> ... 3 more
>>>>
>>>>
>>>>
>>>> When I tried run R commands using sparkR by launching
>>>> $SPARK_HOME/bin/sparkR, it works.
>>>>
>>>>
>>>> Would much appreciate any help.
>>>>
>>>> Thanks!
>>>> Abul
>>>>
>>>>
>>>> On Fri, Sep 16, 2016 at 5:36 PM, Hyung Sung Shim <hs...@nflabs.com>
>>>> wrote:
>>>>
>>>>> Hello.
>>>>> I think that `zeppelin.interpreter.host` and `zeppelin.interpreter.port`
>>>>> values on Interpreter setting UI should be removed because Interpreter
>>>>> will automatically be launched when you run a paragraph.
>>>>>
>>>>> 2016-09-16 3:56 GMT+09:00 Arpan Saha <Ar...@mindtree.com>:
>>>>>
>>>>>> Yes. It is a zeppelin log. Not spark interpreter log. Can you please
>>>>>> tell me where can I get spark interpreter log? I don't find anything under
>>>>>> zeppelin/logs/ path.
>>>>>>
>>>>>> Thanks,
>>>>>> Arpan
>>>>>> ________________________________________
>>>>>> From: DuyHai Doan [doanduyhai@gmail.com]
>>>>>> Sent: Thursday, September 15, 2016 11:20 AM
>>>>>> To: users@zeppelin.apache.org
>>>>>> Subject: Re: FW: Issue with Zeppelin setup on Datastax-Spark
>>>>>>
>>>>>> Is it Zeppelin log or Spark interpreter log ? I guess we need also
>>>>>> the spark interpreter log to understand what is breaking Spark under the
>>>>>> hood. org.apache.thrift.transport.TTransportException is just
>>>>>> technical exception on inter-jvm communication
>>>>>>
>>>>>> On Thu, Sep 15, 2016 at 7:13 PM, Arpan Saha <Arpan.Saha@mindtree.com
>>>>>> <ma...@mindtree.com>> wrote:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Please find below full stack trace.
>>>>>>
>>>>>> ERROR [2016-09-15 16:49:15,117] ({pool-1-thread-2}
>>>>>> RemoteInterpreter.java[init]:182) - Failed to create interpreter:
>>>>>> org.apache.zeppelin.spark.SparkInterpreter
>>>>>> ERROR [2016-09-15 16:49:15,118] ({pool-1-thread-2} Job.java[run]:189)
>>>>>> - Job failed
>>>>>> org.apache.zeppelin.interpreter.InterpreterException:
>>>>>> org.apache.thrift.transport.TTransportException
>>>>>> at org.apache.zeppelin.interprete
>>>>>> r.remote.RemoteInterpreter.init(RemoteInterpreter.java:183)
>>>>>> at org.apache.zeppelin.interprete
>>>>>> r.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:328)
>>>>>> at org.apache.zeppelin.interprete
>>>>>> r.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
>>>>>> at org.apache.zeppelin.notebook.P
>>>>>> aragraph.jobRun(Paragraph.java:260)
>>>>>> at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
>>>>>> at org.apache.zeppelin.scheduler.
>>>>>> RemoteScheduler$JobRunner.run(RemoteScheduler.java:328)
>>>>>> at java.util.concurrent.Executors
>>>>>> $RunnableAdapter.call(Executors.java:511)
>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>>>> at java.util.concurrent.Scheduled
>>>>>> ThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledT
>>>>>> hreadPoolExecutor.java:180)
>>>>>> at java.util.concurrent.Scheduled
>>>>>> ThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPo
>>>>>> olExecutor.java:293)
>>>>>> at java.util.concurrent.ThreadPoo
>>>>>> lExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>>>> at java.util.concurrent.ThreadPoo
>>>>>> lExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>>> Caused by: org.apache.thrift.transport.TTransportException
>>>>>> at org.apache.thrift.transport.TI
>>>>>> OStreamTransport.read(TIOStreamTransport.java:132)
>>>>>> at org.apache.thrift.transport.TT
>>>>>> ransport.readAll(TTransport.java:86)
>>>>>> at org.apache.thrift.protocol.TBi
>>>>>> naryProtocol.readAll(TBinaryProtocol.java:429)
>>>>>> at org.apache.thrift.protocol.TBi
>>>>>> naryProtocol.readI32(TBinaryProtocol.java:318)
>>>>>> at org.apache.thrift.protocol.TBi
>>>>>> naryProtocol.readMessageBegin(TBinaryProtocol.java:219)
>>>>>> at org.apache.thrift.TServiceClie
>>>>>> nt.receiveBase(TServiceClient.java:69)
>>>>>> at org.apache.zeppelin.interprete
>>>>>> r.thrift.RemoteInterpreterService$Client.recv_createInterpre
>>>>>> ter(RemoteInterpreterService.java:184)
>>>>>> at org.apache.zeppelin.interprete
>>>>>> r.thrift.RemoteInterpreterService$Client.createInterpreter(R
>>>>>> emoteInterpreterService.java:168)
>>>>>> at org.apache.zeppelin.interprete
>>>>>> r.remote.RemoteInterpreter.init(RemoteInterpreter.java:172)
>>>>>> ... 12 more
>>>>>> ERROR [2016-09-15 16:49:15,118] ({Thread-18}
>>>>>> RemoteInterpreterEventPoller.java[run]:88) - Can't get
>>>>>> RemoteInterpreterEvent
>>>>>>
>>>>>> Thanks,
>>>>>> Arpan.
>>>>>>
>>>>>> ________________________________________
>>>>>> From: DuyHai Doan [doanduyhai@gmail.com<ma...@gmail.com>]
>>>>>> Sent: Thursday, September 15, 2016 4:29 AM
>>>>>> To: users@zeppelin.apache.org<ma...@zeppelin.apache.org>
>>>>>> Subject: Re: Issue with Zeppelin setup on Datastax-Spark
>>>>>>
>>>>>> Can you provide the complete stack trace ?
>>>>>>
>>>>>> Also, worth reading my blog post about integrating Spark/Zeppelin/C*
>>>>>> or DSE : http://www.doanduyhai.com/blog/?p=2325
>>>>>>
>>>>>> On Thu, Sep 15, 2016 at 5:17 AM, Arpan Saha <Arpan.Saha@mindtree.com
>>>>>> <ma...@mindtree.com><mailto:Arpan.Saha@mindtree.com<ma
>>>>>> ilto:Arpan.Saha@mindtree.com>>> wrote:
>>>>>> Hi Team,
>>>>>>
>>>>>> I am trying to integrate Zeppelin 0.6.0 with DataStax 4.8.8 (which
>>>>>> has Spark 1.4.2). After I configured following properties in
>>>>>> zeppelin-env.sh when I start zeppelin daemon it started and in the browser
>>>>>> I can see zeppelin is running but when I am trying to execute spark query
>>>>>> in the notebook it is throwing below Error. Could you please help me to
>>>>>> solve this issue.
>>>>>>
>>>>>> export JAVA_HOME=<JAVA Home Path>
>>>>>> export SPARK_HOME=/etc/dse/spark
>>>>>> export HADOOP_CONF_DIR=/etc/dse/hadoop
>>>>>> export MASTER=spark://<masterhost>:7077
>>>>>>
>>>>>> Also, added/updated below properties in the Spark Interpreter screen
>>>>>> of Zeppelin UI.
>>>>>>
>>>>>> Master
>>>>>> spark.app.name<http://spark.app.name><http://spark.app.name>
>>>>>> spark.cassandra.auth.password
>>>>>> spark.cassandra.auth.username
>>>>>> spark.cassandra.connection.host
>>>>>> spark.cores.max
>>>>>> spark.executor.memory
>>>>>> zeppelin.interpreter.host
>>>>>> zeppelin.interpreter.port
>>>>>>
>>>>>> Trying to execute below statement in the notebook:
>>>>>> %spark
>>>>>> sc.version
>>>>>>
>>>>>> Below Error is getting:
>>>>>> java.net.SocketException: Connection reset at
>>>>>> java.net.SocketInputStream.read(SocketInputStream.java:209) at
>>>>>> java.net.SocketInputStream.read(SocketInputStream.java:141) at
>>>>>> java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at
>>>>>> java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at
>>>>>> java.io.BufferedInputStream.read(BufferedInputStream.java:345) at
>>>>>> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
>>>>>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
>>>>>> at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
>>>>>> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
>>>>>> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
>>>>>> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
>>>>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>>>>> ice$Client.recv_createInterpreter(RemoteInterpreterService.java:184)
>>>>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>>>>> ice$Client.createInterpreter(RemoteInterpreterService.java:168) at
>>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:172)
>>>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>>>>>> FormType(RemoteInterpreter.java:328) at
>>>>>> org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormT
>>>>>> ype(LazyOpenInterpreter.java:105)
>>>>>>
>>>>>> users-subscribe@zeppelin.apache.org<mailto:users-subscribe@z
>>>>>> eppelin.apache.org><mailto:users-subscribe@zeppelin.apache.org
>>>>>> <ma...@zeppelin.apache.org>>
>>>>>>
>>>>>> Thanks,
>>>>>> Arpan.
>>>>>>
>>>>>> ________________________________
>>>>>> Notice: This e-mail message has been sent by an employee or
>>>>>> contractor of American Express Global Business Travel (“GBT”), a joint
>>>>>> venture that is not wholly-owned by American Express Company or any of its
>>>>>> subsidiaries (“American Express”). E-mail from GBT employees and
>>>>>> contractors may continue to utilize an American Express domain for some
>>>>>> period; however American Express is not responsible for the content of this
>>>>>> email, which is controlled by GBT. “American Express Global Business
>>>>>> Travel”, “American Express” and the American Express Logo are trademarks of
>>>>>> American Express and are used by GBT under limited license.
>>>>>> ________________________________
>>>>>> Important : Cet email a été envoyé par un employé ou un sous-traitant
>>>>>> d’American Express Global Business Travel (« GBT »), une joint-venture qui
>>>>>> n’est pas intégralement détenue par American Express Company ou l’une de
>>>>>> ses filiales (« American Express »). Les emails des employés et
>>>>>> sous-traitants de GBT peuvent continuer de comporter un nom de domaine
>>>>>> d’American Express pour une certaine période ; toutefois, American Express
>>>>>> n’est pas responsable du contenu de cet email, qui est contrôlé par GBT. «
>>>>>> American Express Global Business Travel », « American Express » et le logo
>>>>>> American Express sont des marques d’American Express et sont utilisées par
>>>>>> GBT aux termes d’une licence limitée.
>>>>>>
>>>>>> ________________________________
>>>>>>
>>>>>> http://www.mindtree.com/email/disclaimer.html
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: FW: Issue with Zeppelin setup on Datastax-Spark
Posted by Abul Basar <ab...@einext.com>.
Hello DuyHai,
Original problem reported by Arpan Saha is related to Datastax. I am using
Spark + Zeppelin. Below is the configuration.
- Spark 2.0.2
- Zeppelin: 0.6.2
- Java 1.8.0_111
- R: 3.3.1
Thanks,
Abul
On Wed, Nov 16, 2016 at 3:44 PM, DuyHai Doan <do...@gmail.com> wrote:
> I recommend downloading my pre-built Zeppelin for Datastax. Shared folder
> link: https://drive.google.com/folderview?id=0B6wR2aj4Cb6wQ01aR3ItR0xUNms
>
> On Wed, Nov 16, 2016 at 11:13 AM, DuyHai Doan <do...@gmail.com>
> wrote:
>
>> Ok I understand why you have issue.
>>
>> You are using Spark 2.0.2 and latest Datastax 5.0.3 is still using Spark
>> version 1.6.X
>>
>>
>>
>> On Wed, Nov 16, 2016 at 10:23 AM, Abul Basar <ab...@einext.com> wrote:
>>
>>> I am facing a similar issue while using Spark R.
>>>
>>> My environment:
>>>
>>> - Spark 2.0.2
>>> - Zeppelin: 0.6.2
>>> - Java 1.8.0_111
>>> - R: 3.3.1
>>>
>>>
>>> SPARK_HOME is set. I am trying to run a simple paragraph.
>>>
>>> %r
>>> print("hello ...")
>>>
>>> I get the following exception
>>>
>>> *Interpreter Log*
>>> > # getZeppelinR
>>>
>>> > .zeppelinR = SparkR:::callJStatic("org.apache.zeppelin.spark.ZeppelinR",
>>> "getZeppelinR", hashCode)
>>>
>>> at org.apache.zeppelin.spark.ZeppelinR.waitForRScriptInitialize
>>> d(ZeppelinR.java:295)
>>> at org.apache.zeppelin.spark.ZeppelinR.request(ZeppelinR.java:235)
>>> at org.apache.zeppelin.spark.ZeppelinR.eval(ZeppelinR.java:183)
>>> at org.apache.zeppelin.spark.ZeppelinR.open(ZeppelinR.java:172)
>>> at org.apache.zeppelin.spark.SparkRInterpreter.open(SparkRInter
>>> preter.java:85)
>>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(Laz
>>> yOpenInterpreter.java:69)
>>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgr
>>> ess(LazyOpenInterpreter.java:110)
>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServ
>>> er.getProgress(RemoteInterpreterServer.java:404)
>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>> ice$Processor$getProgress.getResult(RemoteInterpreterService.java:1509)
>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>> ice$Processor$getProgress.getResult(RemoteInterpreterService.java:1494)
>>> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>>> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>>> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run
>>> (TThreadPoolServer.java:285)
>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>>> Executor.java:1142)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>>> lExecutor.java:617)
>>> at java.lang.Thread.run(Thread.java:745)
>>>
>>>
>>> *Zeppelin Log*
>>> ERROR [2016-11-16 14:42:05,664] ({Thread-377}
>>> JobProgressPoller.java[run]:54) - Can not get or update progress
>>> org.apache.zeppelin.interpreter.InterpreterException:
>>> org.apache.thrift.transport.TTransportException
>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>>> Progress(RemoteInterpreter.java:373)
>>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgr
>>> ess(LazyOpenInterpreter.java:111)
>>> at org.apache.zeppelin.notebook.Paragraph.progress(Paragraph.java:237)
>>> at org.apache.zeppelin.scheduler.JobProgressPoller.run(JobProgr
>>> essPoller.java:51)
>>> Caused by: org.apache.thrift.transport.TTransportException
>>> at org.apache.thrift.transport.TIOStreamTransport.read(TIOStrea
>>> mTransport.java:132)
>>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
>>> at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryPr
>>> otocol.java:429)
>>> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryPr
>>> otocol.java:318)
>>> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(
>>> TBinaryProtocol.java:219)
>>> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>> ice$Client.recv_getProgress(RemoteInterpreterService.java:296)
>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>> ice$Client.getProgress(RemoteInterpreterService.java:281)
>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>>> Progress(RemoteInterpreter.java:370)
>>> ... 3 more
>>>
>>>
>>>
>>> When I tried run R commands using sparkR by launching
>>> $SPARK_HOME/bin/sparkR, it works.
>>>
>>>
>>> Would much appreciate any help.
>>>
>>> Thanks!
>>> Abul
>>>
>>>
>>> On Fri, Sep 16, 2016 at 5:36 PM, Hyung Sung Shim <hs...@nflabs.com>
>>> wrote:
>>>
>>>> Hello.
>>>> I think that `zeppelin.interpreter.host` and `zeppelin.interpreter.port`
>>>> values on Interpreter setting UI should be removed because Interpreter
>>>> will automatically be launched when you run a paragraph.
>>>>
>>>> 2016-09-16 3:56 GMT+09:00 Arpan Saha <Ar...@mindtree.com>:
>>>>
>>>>> Yes. It is a zeppelin log. Not spark interpreter log. Can you please
>>>>> tell me where can I get spark interpreter log? I don't find anything under
>>>>> zeppelin/logs/ path.
>>>>>
>>>>> Thanks,
>>>>> Arpan
>>>>> ________________________________________
>>>>> From: DuyHai Doan [doanduyhai@gmail.com]
>>>>> Sent: Thursday, September 15, 2016 11:20 AM
>>>>> To: users@zeppelin.apache.org
>>>>> Subject: Re: FW: Issue with Zeppelin setup on Datastax-Spark
>>>>>
>>>>> Is it Zeppelin log or Spark interpreter log ? I guess we need also the
>>>>> spark interpreter log to understand what is breaking Spark under the hood.
>>>>> org.apache.thrift.transport.TTransportException is just technical
>>>>> exception on inter-jvm communication
>>>>>
>>>>> On Thu, Sep 15, 2016 at 7:13 PM, Arpan Saha <Arpan.Saha@mindtree.com
>>>>> <ma...@mindtree.com>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> Please find below full stack trace.
>>>>>
>>>>> ERROR [2016-09-15 16:49:15,117] ({pool-1-thread-2}
>>>>> RemoteInterpreter.java[init]:182) - Failed to create interpreter:
>>>>> org.apache.zeppelin.spark.SparkInterpreter
>>>>> ERROR [2016-09-15 16:49:15,118] ({pool-1-thread-2} Job.java[run]:189)
>>>>> - Job failed
>>>>> org.apache.zeppelin.interpreter.InterpreterException:
>>>>> org.apache.thrift.transport.TTransportException
>>>>> at org.apache.zeppelin.interprete
>>>>> r.remote.RemoteInterpreter.init(RemoteInterpreter.java:183)
>>>>> at org.apache.zeppelin.interprete
>>>>> r.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:328)
>>>>> at org.apache.zeppelin.interprete
>>>>> r.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
>>>>> at org.apache.zeppelin.notebook.P
>>>>> aragraph.jobRun(Paragraph.java:260)
>>>>> at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
>>>>> at org.apache.zeppelin.scheduler.
>>>>> RemoteScheduler$JobRunner.run(RemoteScheduler.java:328)
>>>>> at java.util.concurrent.Executors
>>>>> $RunnableAdapter.call(Executors.java:511)
>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>>> at java.util.concurrent.Scheduled
>>>>> ThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledT
>>>>> hreadPoolExecutor.java:180)
>>>>> at java.util.concurrent.Scheduled
>>>>> ThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPo
>>>>> olExecutor.java:293)
>>>>> at java.util.concurrent.ThreadPoo
>>>>> lExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>>> at java.util.concurrent.ThreadPoo
>>>>> lExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>> Caused by: org.apache.thrift.transport.TTransportException
>>>>> at org.apache.thrift.transport.TI
>>>>> OStreamTransport.read(TIOStreamTransport.java:132)
>>>>> at org.apache.thrift.transport.TT
>>>>> ransport.readAll(TTransport.java:86)
>>>>> at org.apache.thrift.protocol.TBi
>>>>> naryProtocol.readAll(TBinaryProtocol.java:429)
>>>>> at org.apache.thrift.protocol.TBi
>>>>> naryProtocol.readI32(TBinaryProtocol.java:318)
>>>>> at org.apache.thrift.protocol.TBi
>>>>> naryProtocol.readMessageBegin(TBinaryProtocol.java:219)
>>>>> at org.apache.thrift.TServiceClie
>>>>> nt.receiveBase(TServiceClient.java:69)
>>>>> at org.apache.zeppelin.interprete
>>>>> r.thrift.RemoteInterpreterService$Client.recv_createInterpre
>>>>> ter(RemoteInterpreterService.java:184)
>>>>> at org.apache.zeppelin.interprete
>>>>> r.thrift.RemoteInterpreterService$Client.createInterpreter(R
>>>>> emoteInterpreterService.java:168)
>>>>> at org.apache.zeppelin.interprete
>>>>> r.remote.RemoteInterpreter.init(RemoteInterpreter.java:172)
>>>>> ... 12 more
>>>>> ERROR [2016-09-15 16:49:15,118] ({Thread-18}
>>>>> RemoteInterpreterEventPoller.java[run]:88) - Can't get
>>>>> RemoteInterpreterEvent
>>>>>
>>>>> Thanks,
>>>>> Arpan.
>>>>>
>>>>> ________________________________________
>>>>> From: DuyHai Doan [doanduyhai@gmail.com<ma...@gmail.com>]
>>>>> Sent: Thursday, September 15, 2016 4:29 AM
>>>>> To: users@zeppelin.apache.org<ma...@zeppelin.apache.org>
>>>>> Subject: Re: Issue with Zeppelin setup on Datastax-Spark
>>>>>
>>>>> Can you provide the complete stack trace ?
>>>>>
>>>>> Also, worth reading my blog post about integrating Spark/Zeppelin/C*
>>>>> or DSE : http://www.doanduyhai.com/blog/?p=2325
>>>>>
>>>>> On Thu, Sep 15, 2016 at 5:17 AM, Arpan Saha <Arpan.Saha@mindtree.com
>>>>> <ma...@mindtree.com><mailto:Arpan.Saha@mindtree.com<ma
>>>>> ilto:Arpan.Saha@mindtree.com>>> wrote:
>>>>> Hi Team,
>>>>>
>>>>> I am trying to integrate Zeppelin 0.6.0 with DataStax 4.8.8 (which has
>>>>> Spark 1.4.2). After I configured following properties in zeppelin-env.sh
>>>>> when I start zeppelin daemon it started and in the browser I can see
>>>>> zeppelin is running but when I am trying to execute spark query in the
>>>>> notebook it is throwing below Error. Could you please help me to solve this
>>>>> issue.
>>>>>
>>>>> export JAVA_HOME=<JAVA Home Path>
>>>>> export SPARK_HOME=/etc/dse/spark
>>>>> export HADOOP_CONF_DIR=/etc/dse/hadoop
>>>>> export MASTER=spark://<masterhost>:7077
>>>>>
>>>>> Also, added/updated below properties in the Spark Interpreter screen
>>>>> of Zeppelin UI.
>>>>>
>>>>> Master
>>>>> spark.app.name<http://spark.app.name><http://spark.app.name>
>>>>> spark.cassandra.auth.password
>>>>> spark.cassandra.auth.username
>>>>> spark.cassandra.connection.host
>>>>> spark.cores.max
>>>>> spark.executor.memory
>>>>> zeppelin.interpreter.host
>>>>> zeppelin.interpreter.port
>>>>>
>>>>> Trying to execute below statement in the notebook:
>>>>> %spark
>>>>> sc.version
>>>>>
>>>>> Below Error is getting:
>>>>> java.net.SocketException: Connection reset at
>>>>> java.net.SocketInputStream.read(SocketInputStream.java:209) at
>>>>> java.net.SocketInputStream.read(SocketInputStream.java:141) at
>>>>> java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at
>>>>> java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at
>>>>> java.io.BufferedInputStream.read(BufferedInputStream.java:345) at
>>>>> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
>>>>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
>>>>> at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
>>>>> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
>>>>> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
>>>>> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
>>>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>>>> ice$Client.recv_createInterpreter(RemoteInterpreterService.java:184)
>>>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>>>> ice$Client.createInterpreter(RemoteInterpreterService.java:168) at
>>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:172)
>>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>>>>> FormType(RemoteInterpreter.java:328) at org.apache.zeppelin.interprete
>>>>> r.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
>>>>>
>>>>> users-subscribe@zeppelin.apache.org<mailto:users-subscribe@z
>>>>> eppelin.apache.org><mailto:users-subscribe@zeppelin.apache.org<mailto:
>>>>> users-subscribe@zeppelin.apache.org>>
>>>>>
>>>>> Thanks,
>>>>> Arpan.
>>>>>
>>>>> ________________________________
>>>>> Notice: This e-mail message has been sent by an employee or contractor
>>>>> of American Express Global Business Travel (“GBT”), a joint venture that is
>>>>> not wholly-owned by American Express Company or any of its subsidiaries
>>>>> (“American Express”). E-mail from GBT employees and contractors may
>>>>> continue to utilize an American Express domain for some period; however
>>>>> American Express is not responsible for the content of this email, which is
>>>>> controlled by GBT. “American Express Global Business Travel”, “American
>>>>> Express” and the American Express Logo are trademarks of American Express
>>>>> and are used by GBT under limited license.
>>>>> ________________________________
>>>>> Important : Cet email a été envoyé par un employé ou un sous-traitant
>>>>> d’American Express Global Business Travel (« GBT »), une joint-venture qui
>>>>> n’est pas intégralement détenue par American Express Company ou l’une de
>>>>> ses filiales (« American Express »). Les emails des employés et
>>>>> sous-traitants de GBT peuvent continuer de comporter un nom de domaine
>>>>> d’American Express pour une certaine période ; toutefois, American Express
>>>>> n’est pas responsable du contenu de cet email, qui est contrôlé par GBT. «
>>>>> American Express Global Business Travel », « American Express » et le logo
>>>>> American Express sont des marques d’American Express et sont utilisées par
>>>>> GBT aux termes d’une licence limitée.
>>>>>
>>>>> ________________________________
>>>>>
>>>>> http://www.mindtree.com/email/disclaimer.html
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: FW: Issue with Zeppelin setup on Datastax-Spark
Posted by DuyHai Doan <do...@gmail.com>.
I recommend downloading my pre-built Zeppelin for Datastax. Shared folder
link: https://drive.google.com/folderview?id=0B6wR2aj4Cb6wQ01aR3ItR0xUNms
On Wed, Nov 16, 2016 at 11:13 AM, DuyHai Doan <do...@gmail.com> wrote:
> Ok I understand why you have issue.
>
> You are using Spark 2.0.2 and latest Datastax 5.0.3 is still using Spark
> version 1.6.X
>
>
>
> On Wed, Nov 16, 2016 at 10:23 AM, Abul Basar <ab...@einext.com> wrote:
>
>> I am facing a similar issue while using Spark R.
>>
>> My environment:
>>
>> - Spark 2.0.2
>> - Zeppelin: 0.6.2
>> - Java 1.8.0_111
>> - R: 3.3.1
>>
>>
>> SPARK_HOME is set. I am trying to run a simple paragraph.
>>
>> %r
>> print("hello ...")
>>
>> I get the following exception
>>
>> *Interpreter Log*
>> > # getZeppelinR
>>
>> > .zeppelinR = SparkR:::callJStatic("org.apache.zeppelin.spark.ZeppelinR",
>> "getZeppelinR", hashCode)
>>
>> at org.apache.zeppelin.spark.ZeppelinR.waitForRScriptInitialize
>> d(ZeppelinR.java:295)
>> at org.apache.zeppelin.spark.ZeppelinR.request(ZeppelinR.java:235)
>> at org.apache.zeppelin.spark.ZeppelinR.eval(ZeppelinR.java:183)
>> at org.apache.zeppelin.spark.ZeppelinR.open(ZeppelinR.java:172)
>> at org.apache.zeppelin.spark.SparkRInterpreter.open(SparkRInter
>> preter.java:85)
>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(Laz
>> yOpenInterpreter.java:69)
>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgr
>> ess(LazyOpenInterpreter.java:110)
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServ
>> er.getProgress(RemoteInterpreterServer.java:404)
>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>> ice$Processor$getProgress.getResult(RemoteInterpreterService.java:1509)
>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>> ice$Processor$getProgress.getResult(RemoteInterpreterService.java:1494)
>> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
>> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
>> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run
>> (TThreadPoolServer.java:285)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>> Executor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>> lExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>>
>>
>> *Zeppelin Log*
>> ERROR [2016-11-16 14:42:05,664] ({Thread-377}
>> JobProgressPoller.java[run]:54) - Can not get or update progress
>> org.apache.zeppelin.interpreter.InterpreterException:
>> org.apache.thrift.transport.TTransportException
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>> Progress(RemoteInterpreter.java:373)
>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgr
>> ess(LazyOpenInterpreter.java:111)
>> at org.apache.zeppelin.notebook.Paragraph.progress(Paragraph.java:237)
>> at org.apache.zeppelin.scheduler.JobProgressPoller.run(JobProgr
>> essPoller.java:51)
>> Caused by: org.apache.thrift.transport.TTransportException
>> at org.apache.thrift.transport.TIOStreamTransport.read(TIOStrea
>> mTransport.java:132)
>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
>> at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryPr
>> otocol.java:429)
>> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryPr
>> otocol.java:318)
>> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(
>> TBinaryProtocol.java:219)
>> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>> ice$Client.recv_getProgress(RemoteInterpreterService.java:296)
>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>> ice$Client.getProgress(RemoteInterpreterService.java:281)
>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>> Progress(RemoteInterpreter.java:370)
>> ... 3 more
>>
>>
>>
>> When I tried run R commands using sparkR by launching
>> $SPARK_HOME/bin/sparkR, it works.
>>
>>
>> Would much appreciate any help.
>>
>> Thanks!
>> Abul
>>
>>
>> On Fri, Sep 16, 2016 at 5:36 PM, Hyung Sung Shim <hs...@nflabs.com>
>> wrote:
>>
>>> Hello.
>>> I think that `zeppelin.interpreter.host` and `zeppelin.interpreter.port`
>>> values on Interpreter setting UI should be removed because Interpreter
>>> will automatically be launched when you run a paragraph.
>>>
>>> 2016-09-16 3:56 GMT+09:00 Arpan Saha <Ar...@mindtree.com>:
>>>
>>>> Yes. It is a zeppelin log. Not spark interpreter log. Can you please
>>>> tell me where can I get spark interpreter log? I don't find anything under
>>>> zeppelin/logs/ path.
>>>>
>>>> Thanks,
>>>> Arpan
>>>> ________________________________________
>>>> From: DuyHai Doan [doanduyhai@gmail.com]
>>>> Sent: Thursday, September 15, 2016 11:20 AM
>>>> To: users@zeppelin.apache.org
>>>> Subject: Re: FW: Issue with Zeppelin setup on Datastax-Spark
>>>>
>>>> Is it Zeppelin log or Spark interpreter log ? I guess we need also the
>>>> spark interpreter log to understand what is breaking Spark under the hood.
>>>> org.apache.thrift.transport.TTransportException is just technical
>>>> exception on inter-jvm communication
>>>>
>>>> On Thu, Sep 15, 2016 at 7:13 PM, Arpan Saha <Arpan.Saha@mindtree.com
>>>> <ma...@mindtree.com>> wrote:
>>>>
>>>> Hi,
>>>>
>>>> Please find below full stack trace.
>>>>
>>>> ERROR [2016-09-15 16:49:15,117] ({pool-1-thread-2}
>>>> RemoteInterpreter.java[init]:182) - Failed to create interpreter:
>>>> org.apache.zeppelin.spark.SparkInterpreter
>>>> ERROR [2016-09-15 16:49:15,118] ({pool-1-thread-2} Job.java[run]:189) -
>>>> Job failed
>>>> org.apache.zeppelin.interpreter.InterpreterException:
>>>> org.apache.thrift.transport.TTransportException
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.ini
>>>> t(RemoteInterpreter.java:183)
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>>>> FormType(RemoteInterpreter.java:328)
>>>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormT
>>>> ype(LazyOpenInterpreter.java:105)
>>>> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java
>>>> :260)
>>>> at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
>>>> at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(
>>>> RemoteScheduler.java:328)
>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executor
>>>> s.java:511)
>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu
>>>> tureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu
>>>> tureTask.run(ScheduledThreadPoolExecutor.java:293)
>>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>>>> Executor.java:1142)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>>>> lExecutor.java:617)
>>>> at java.lang.Thread.run(Thread.java:745)
>>>> Caused by: org.apache.thrift.transport.TTransportException
>>>> at org.apache.thrift.transport.TIOStreamTransport.read(TIOStrea
>>>> mTransport.java:132)
>>>> at org.apache.thrift.transport.TTransport.readAll(TTransport.ja
>>>> va:86)
>>>> at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryPr
>>>> otocol.java:429)
>>>> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryPr
>>>> otocol.java:318)
>>>> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(
>>>> TBinaryProtocol.java:219)
>>>> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.
>>>> java:69)
>>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>>> ice$Client.recv_createInterpreter(RemoteInterpreterService.java:184)
>>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>>> ice$Client.createInterpreter(RemoteInterpreterService.java:168)
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.ini
>>>> t(RemoteInterpreter.java:172)
>>>> ... 12 more
>>>> ERROR [2016-09-15 16:49:15,118] ({Thread-18}
>>>> RemoteInterpreterEventPoller.java[run]:88) - Can't get
>>>> RemoteInterpreterEvent
>>>>
>>>> Thanks,
>>>> Arpan.
>>>>
>>>> ________________________________________
>>>> From: DuyHai Doan [doanduyhai@gmail.com<ma...@gmail.com>]
>>>> Sent: Thursday, September 15, 2016 4:29 AM
>>>> To: users@zeppelin.apache.org<ma...@zeppelin.apache.org>
>>>> Subject: Re: Issue with Zeppelin setup on Datastax-Spark
>>>>
>>>> Can you provide the complete stack trace ?
>>>>
>>>> Also, worth reading my blog post about integrating Spark/Zeppelin/C* or
>>>> DSE : http://www.doanduyhai.com/blog/?p=2325
>>>>
>>>> On Thu, Sep 15, 2016 at 5:17 AM, Arpan Saha <Arpan.Saha@mindtree.com
>>>> <ma...@mindtree.com><mailto:Arpan.Saha@mindtree.com<mailto:
>>>> Arpan.Saha@mindtree.com>>> wrote:
>>>> Hi Team,
>>>>
>>>> I am trying to integrate Zeppelin 0.6.0 with DataStax 4.8.8 (which has
>>>> Spark 1.4.2). After I configured following properties in zeppelin-env.sh
>>>> when I start zeppelin daemon it started and in the browser I can see
>>>> zeppelin is running but when I am trying to execute spark query in the
>>>> notebook it is throwing below Error. Could you please help me to solve this
>>>> issue.
>>>>
>>>> export JAVA_HOME=<JAVA Home Path>
>>>> export SPARK_HOME=/etc/dse/spark
>>>> export HADOOP_CONF_DIR=/etc/dse/hadoop
>>>> export MASTER=spark://<masterhost>:7077
>>>>
>>>> Also, added/updated below properties in the Spark Interpreter screen of
>>>> Zeppelin UI.
>>>>
>>>> Master
>>>> spark.app.name<http://spark.app.name><http://spark.app.name>
>>>> spark.cassandra.auth.password
>>>> spark.cassandra.auth.username
>>>> spark.cassandra.connection.host
>>>> spark.cores.max
>>>> spark.executor.memory
>>>> zeppelin.interpreter.host
>>>> zeppelin.interpreter.port
>>>>
>>>> Trying to execute below statement in the notebook:
>>>> %spark
>>>> sc.version
>>>>
>>>> Below Error is getting:
>>>> java.net.SocketException: Connection reset at
>>>> java.net.SocketInputStream.read(SocketInputStream.java:209) at
>>>> java.net.SocketInputStream.read(SocketInputStream.java:141) at
>>>> java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at
>>>> java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at
>>>> java.io.BufferedInputStream.read(BufferedInputStream.java:345) at
>>>> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
>>>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
>>>> at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
>>>> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
>>>> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
>>>> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
>>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>>> ice$Client.recv_createInterpreter(RemoteInterpreterService.java:184)
>>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>>> ice$Client.createInterpreter(RemoteInterpreterService.java:168) at
>>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:172)
>>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>>>> FormType(RemoteInterpreter.java:328) at org.apache.zeppelin.interprete
>>>> r.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
>>>>
>>>> users-subscribe@zeppelin.apache.org<mailto:users-subscribe@z
>>>> eppelin.apache.org><mailto:users-subscribe@zeppelin.apache.org<mailto:
>>>> users-subscribe@zeppelin.apache.org>>
>>>>
>>>> Thanks,
>>>> Arpan.
>>>>
>>>> ________________________________
>>>> Notice: This e-mail message has been sent by an employee or contractor
>>>> of American Express Global Business Travel (“GBT”), a joint venture that is
>>>> not wholly-owned by American Express Company or any of its subsidiaries
>>>> (“American Express”). E-mail from GBT employees and contractors may
>>>> continue to utilize an American Express domain for some period; however
>>>> American Express is not responsible for the content of this email, which is
>>>> controlled by GBT. “American Express Global Business Travel”, “American
>>>> Express” and the American Express Logo are trademarks of American Express
>>>> and are used by GBT under limited license.
>>>> ________________________________
>>>> Important : Cet email a été envoyé par un employé ou un sous-traitant
>>>> d’American Express Global Business Travel (« GBT »), une joint-venture qui
>>>> n’est pas intégralement détenue par American Express Company ou l’une de
>>>> ses filiales (« American Express »). Les emails des employés et
>>>> sous-traitants de GBT peuvent continuer de comporter un nom de domaine
>>>> d’American Express pour une certaine période ; toutefois, American Express
>>>> n’est pas responsable du contenu de cet email, qui est contrôlé par GBT. «
>>>> American Express Global Business Travel », « American Express » et le logo
>>>> American Express sont des marques d’American Express et sont utilisées par
>>>> GBT aux termes d’une licence limitée.
>>>>
>>>> ________________________________
>>>>
>>>> http://www.mindtree.com/email/disclaimer.html
>>>>
>>>>
>>>>
>>>
>>
>
Re: FW: Issue with Zeppelin setup on Datastax-Spark
Posted by DuyHai Doan <do...@gmail.com>.
Ok I understand why you have issue.
You are using Spark 2.0.2 and latest Datastax 5.0.3 is still using Spark
version 1.6.X
On Wed, Nov 16, 2016 at 10:23 AM, Abul Basar <ab...@einext.com> wrote:
> I am facing a similar issue while using Spark R.
>
> My environment:
>
> - Spark 2.0.2
> - Zeppelin: 0.6.2
> - Java 1.8.0_111
> - R: 3.3.1
>
>
> SPARK_HOME is set. I am trying to run a simple paragraph.
>
> %r
> print("hello ...")
>
> I get the following exception
>
> *Interpreter Log*
> > # getZeppelinR
>
> > .zeppelinR = SparkR:::callJStatic("org.apache.zeppelin.spark.ZeppelinR",
> "getZeppelinR", hashCode)
>
> at org.apache.zeppelin.spark.ZeppelinR.waitForRScriptInitialize
> d(ZeppelinR.java:295)
> at org.apache.zeppelin.spark.ZeppelinR.request(ZeppelinR.java:235)
> at org.apache.zeppelin.spark.ZeppelinR.eval(ZeppelinR.java:183)
> at org.apache.zeppelin.spark.ZeppelinR.open(ZeppelinR.java:172)
> at org.apache.zeppelin.spark.SparkRInterpreter.open(SparkRInter
> preter.java:85)
> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(Laz
> yOpenInterpreter.java:69)
> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgr
> ess(LazyOpenInterpreter.java:110)
> at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServ
> er.getProgress(RemoteInterpreterServer.java:404)
> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
> ice$Processor$getProgress.getResult(RemoteInterpreterService.java:1509)
> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
> ice$Processor$getProgress.getResult(RemoteInterpreterService.java:1494)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.
> run(TThreadPoolServer.java:285)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
> Executor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
> lExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>
>
> *Zeppelin Log*
> ERROR [2016-11-16 14:42:05,664] ({Thread-377}
> JobProgressPoller.java[run]:54) - Can not get or update progress
> org.apache.zeppelin.interpreter.InterpreterException:
> org.apache.thrift.transport.TTransportException
> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.
> getProgress(RemoteInterpreter.java:373)
> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgr
> ess(LazyOpenInterpreter.java:111)
> at org.apache.zeppelin.notebook.Paragraph.progress(Paragraph.java:237)
> at org.apache.zeppelin.scheduler.JobProgressPoller.run(JobProgr
> essPoller.java:51)
> Caused by: org.apache.thrift.transport.TTransportException
> at org.apache.thrift.transport.TIOStreamTransport.read(TIOStrea
> mTransport.java:132)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
> at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryPr
> otocol.java:429)
> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryPr
> otocol.java:318)
> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(
> TBinaryProtocol.java:219)
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
> ice$Client.recv_getProgress(RemoteInterpreterService.java:296)
> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
> ice$Client.getProgress(RemoteInterpreterService.java:281)
> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.
> getProgress(RemoteInterpreter.java:370)
> ... 3 more
>
>
>
> When I tried run R commands using sparkR by launching
> $SPARK_HOME/bin/sparkR, it works.
>
>
> Would much appreciate any help.
>
> Thanks!
> Abul
>
>
> On Fri, Sep 16, 2016 at 5:36 PM, Hyung Sung Shim <hs...@nflabs.com>
> wrote:
>
>> Hello.
>> I think that `zeppelin.interpreter.host` and `zeppelin.interpreter.port`
>> values on Interpreter setting UI should be removed because Interpreter
>> will automatically be launched when you run a paragraph.
>>
>> 2016-09-16 3:56 GMT+09:00 Arpan Saha <Ar...@mindtree.com>:
>>
>>> Yes. It is a zeppelin log. Not spark interpreter log. Can you please
>>> tell me where can I get spark interpreter log? I don't find anything under
>>> zeppelin/logs/ path.
>>>
>>> Thanks,
>>> Arpan
>>> ________________________________________
>>> From: DuyHai Doan [doanduyhai@gmail.com]
>>> Sent: Thursday, September 15, 2016 11:20 AM
>>> To: users@zeppelin.apache.org
>>> Subject: Re: FW: Issue with Zeppelin setup on Datastax-Spark
>>>
>>> Is it Zeppelin log or Spark interpreter log ? I guess we need also the
>>> spark interpreter log to understand what is breaking Spark under the hood.
>>> org.apache.thrift.transport.TTransportException is just technical
>>> exception on inter-jvm communication
>>>
>>> On Thu, Sep 15, 2016 at 7:13 PM, Arpan Saha <Arpan.Saha@mindtree.com
>>> <ma...@mindtree.com>> wrote:
>>>
>>> Hi,
>>>
>>> Please find below full stack trace.
>>>
>>> ERROR [2016-09-15 16:49:15,117] ({pool-1-thread-2}
>>> RemoteInterpreter.java[init]:182) - Failed to create interpreter:
>>> org.apache.zeppelin.spark.SparkInterpreter
>>> ERROR [2016-09-15 16:49:15,118] ({pool-1-thread-2} Job.java[run]:189) -
>>> Job failed
>>> org.apache.zeppelin.interpreter.InterpreterException:
>>> org.apache.thrift.transport.TTransportException
>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.ini
>>> t(RemoteInterpreter.java:183)
>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>>> FormType(RemoteInterpreter.java:328)
>>> at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getFormT
>>> ype(LazyOpenInterpreter.java:105)
>>> at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java
>>> :260)
>>> at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
>>> at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(
>>> RemoteScheduler.java:328)
>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executor
>>> s.java:511)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu
>>> tureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu
>>> tureTask.run(ScheduledThreadPoolExecutor.java:293)
>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>>> Executor.java:1142)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>>> lExecutor.java:617)
>>> at java.lang.Thread.run(Thread.java:745)
>>> Caused by: org.apache.thrift.transport.TTransportException
>>> at org.apache.thrift.transport.TIOStreamTransport.read(TIOStrea
>>> mTransport.java:132)
>>> at org.apache.thrift.transport.TTransport.readAll(TTransport.ja
>>> va:86)
>>> at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryPr
>>> otocol.java:429)
>>> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryPr
>>> otocol.java:318)
>>> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(
>>> TBinaryProtocol.java:219)
>>> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.
>>> java:69)
>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>> ice$Client.recv_createInterpreter(RemoteInterpreterService.java:184)
>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>> ice$Client.createInterpreter(RemoteInterpreterService.java:168)
>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.ini
>>> t(RemoteInterpreter.java:172)
>>> ... 12 more
>>> ERROR [2016-09-15 16:49:15,118] ({Thread-18}
>>> RemoteInterpreterEventPoller.java[run]:88) - Can't get
>>> RemoteInterpreterEvent
>>>
>>> Thanks,
>>> Arpan.
>>>
>>> ________________________________________
>>> From: DuyHai Doan [doanduyhai@gmail.com<ma...@gmail.com>]
>>> Sent: Thursday, September 15, 2016 4:29 AM
>>> To: users@zeppelin.apache.org<ma...@zeppelin.apache.org>
>>> Subject: Re: Issue with Zeppelin setup on Datastax-Spark
>>>
>>> Can you provide the complete stack trace ?
>>>
>>> Also, worth reading my blog post about integrating Spark/Zeppelin/C* or
>>> DSE : http://www.doanduyhai.com/blog/?p=2325
>>>
>>> On Thu, Sep 15, 2016 at 5:17 AM, Arpan Saha <Arpan.Saha@mindtree.com
>>> <ma...@mindtree.com><mailto:Arpan.Saha@mindtree.com<mailto:
>>> Arpan.Saha@mindtree.com>>> wrote:
>>> Hi Team,
>>>
>>> I am trying to integrate Zeppelin 0.6.0 with DataStax 4.8.8 (which has
>>> Spark 1.4.2). After I configured following properties in zeppelin-env.sh
>>> when I start zeppelin daemon it started and in the browser I can see
>>> zeppelin is running but when I am trying to execute spark query in the
>>> notebook it is throwing below Error. Could you please help me to solve this
>>> issue.
>>>
>>> export JAVA_HOME=<JAVA Home Path>
>>> export SPARK_HOME=/etc/dse/spark
>>> export HADOOP_CONF_DIR=/etc/dse/hadoop
>>> export MASTER=spark://<masterhost>:7077
>>>
>>> Also, added/updated below properties in the Spark Interpreter screen of
>>> Zeppelin UI.
>>>
>>> Master
>>> spark.app.name<http://spark.app.name><http://spark.app.name>
>>> spark.cassandra.auth.password
>>> spark.cassandra.auth.username
>>> spark.cassandra.connection.host
>>> spark.cores.max
>>> spark.executor.memory
>>> zeppelin.interpreter.host
>>> zeppelin.interpreter.port
>>>
>>> Trying to execute below statement in the notebook:
>>> %spark
>>> sc.version
>>>
>>> Below Error is getting:
>>> java.net.SocketException: Connection reset at
>>> java.net.SocketInputStream.read(SocketInputStream.java:209) at
>>> java.net.SocketInputStream.read(SocketInputStream.java:141) at
>>> java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at
>>> java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at
>>> java.io.BufferedInputStream.read(BufferedInputStream.java:345) at
>>> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
>>> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
>>> at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
>>> at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
>>> at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
>>> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
>>> at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>> ice$Client.recv_createInterpreter(RemoteInterpreterService.java:184) at
>>> org.apache.zeppelin.interpreter.thrift.RemoteInterpreterServ
>>> ice$Client.createInterpreter(RemoteInterpreterService.java:168) at
>>> org.apache.zeppelin.interpreter.remote.RemoteInterpreter.init(RemoteInterpreter.java:172)
>>> at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.get
>>> FormType(RemoteInterpreter.java:328) at org.apache.zeppelin.interprete
>>> r.LazyOpenInterpreter.getFormType(LazyOpenInterpreter.java:105)
>>>
>>> users-subscribe@zeppelin.apache.org<mailto:users-subscribe@z
>>> eppelin.apache.org><mailto:users-subscribe@zeppelin.apache.org<mailto:
>>> users-subscribe@zeppelin.apache.org>>
>>>
>>> Thanks,
>>> Arpan.
>>>
>>> ________________________________
>>> Notice: This e-mail message has been sent by an employee or contractor
>>> of American Express Global Business Travel (“GBT”), a joint venture that is
>>> not wholly-owned by American Express Company or any of its subsidiaries
>>> (“American Express”). E-mail from GBT employees and contractors may
>>> continue to utilize an American Express domain for some period; however
>>> American Express is not responsible for the content of this email, which is
>>> controlled by GBT. “American Express Global Business Travel”, “American
>>> Express” and the American Express Logo are trademarks of American Express
>>> and are used by GBT under limited license.
>>> ________________________________
>>> Important : Cet email a été envoyé par un employé ou un sous-traitant
>>> d’American Express Global Business Travel (« GBT »), une joint-venture qui
>>> n’est pas intégralement détenue par American Express Company ou l’une de
>>> ses filiales (« American Express »). Les emails des employés et
>>> sous-traitants de GBT peuvent continuer de comporter un nom de domaine
>>> d’American Express pour une certaine période ; toutefois, American Express
>>> n’est pas responsable du contenu de cet email, qui est contrôlé par GBT. «
>>> American Express Global Business Travel », « American Express » et le logo
>>> American Express sont des marques d’American Express et sont utilisées par
>>> GBT aux termes d’une licence limitée.
>>>
>>> ________________________________
>>>
>>> http://www.mindtree.com/email/disclaimer.html
>>>
>>>
>>>
>>
>