You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by Richard Xin <ri...@yahoo.com> on 2017/06/22 21:59:37 UTC
NPE in SparkInterpreter.java
apache/zeppelin
|
|
|
| | |
|
|
|
| |
apache/zeppelin
zeppelin - Mirror of Apache Zeppelin
|
|
|
in the code I see // Some case, scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call throws an NPE
We had issues that Zeppelin note errored out immediately without any hint messages, and we are seeing this NPE in the spark_interpreter log (Zeppelin 0.7.1 on AWS EMR cluster). can someone explain what's the likely cause for this error and how to prevent it from happening again?
Thanks,
Re: NPE in SparkInterpreter.java
Posted by Serega Sheypak <se...@gmail.com>.
It was my fault, I'm so sorry, I've recompiled zeppelin for scala 2.11 to
make it run with cloudera spark 2.0 and used scala 2.10 third party libs. I
replaced them with 2.11 versions and it started to work
вт, 27 июня 2017 г. в 9:52, Serega Sheypak <se...@gmail.com>:
> Hi, I'm stuck.
> I think I've localized the problem. I get non-obvious NPE when I try to
> add 3rd party jars to spark interpreter using any approach metioned in
> documentation.
> Zeppelin works fine for me when I just read text (no external dependencies
> needed.)
>
> 2017-06-26 23:29 GMT+02:00 Richard Xin <ri...@yahoo.com>:
>
>> just try to understand the comment in the code:
>> // Some case, scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call
>> throws an NPE
>>
>> what are the "Some case", it sounds like Zeppelin developers are not sure
>> how it happened and how to prevent it?
>>
>> I am trying to make a strong case for my company to switch from other
>> notebook application to Zeppelin, Zeppelin looks good and only this issue
>> concerns me.
>>
>> I'm looking forward for any insights, thanks.
>> On Monday, June 26, 2017, 11:56:45 AM PDT, Serega Sheypak <
>> serega.sheypak@gmail.com> wrote:
>>
>>
>> Hi, I have more or less the same symptom
>>
>>
>> if (Utils.isScala2_10()) {
>> binder = (Map<String, Object>) getValue("_binder");
>> } else {
>> binder = (Map<String, Object>) getLastObject();
>> }
>> binder.put("sc", sc); // EXCEPTION HERE
>>
>>
>> java.lang.NullPointerException at
>> org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:887)
>> at
>> org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70)
>> at
>> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:489)
>> at org.apache.zeppelin.scheduler.Job.run(Job.java:181) at
>> org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:262) at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>> at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:745)
>>
>> I'm going to check this one stuff:
>>
>> https://stackoverflow.com/questions/43289067/getting-nullpointerexception-when-running-spark-code-in-zeppelin-0-7-1
>>
>>
>> 2017-06-26 20:41 GMT+02:00 Richard Xin <ri...@yahoo.com>:
>>
>> Anyone?
>>
>>
>>
>> On Thursday, June 22, 2017, 3:00:01 PM PDT, Richard Xin <
>> richardxin168@yahoo.com> wrote:
>>
>>
>> apache/zeppelin
>> <https://github.com/apache/zeppelin/blob/master/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java#L1294>
>>
>> apache/zeppelin
>>
>> zeppelin - Mirror of Apache Zeppelin
>>
>> <https://github.com/apache/zeppelin/blob/master/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java#L1294>
>>
>> in the code I see // Some case, scala.tools.nsc.interpreter.
>> IMain$ReadEvalPrint.call throws an NPE
>>
>> We had issues that Zeppelin note errored out immediately without any hint
>> messages, and we are seeing this NPE in the spark_interpreter log (Zeppelin
>> 0.7.1 on AWS EMR cluster). can someone explain what's the likely cause for
>> this error and how to prevent it from happening again?
>>
>> Thanks,
>>
>>
>>
>
Re: NPE in SparkInterpreter.java
Posted by Serega Sheypak <se...@gmail.com>.
Hi, I'm stuck.
I think I've localized the problem. I get non-obvious NPE when I try to add
3rd party jars to spark interpreter using any approach metioned in
documentation.
Zeppelin works fine for me when I just read text (no external dependencies
needed.)
2017-06-26 23:29 GMT+02:00 Richard Xin <ri...@yahoo.com>:
> just try to understand the comment in the code:
> // Some case, scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call throws
> an NPE
>
> what are the "Some case", it sounds like Zeppelin developers are not sure
> how it happened and how to prevent it?
>
> I am trying to make a strong case for my company to switch from other
> notebook application to Zeppelin, Zeppelin looks good and only this issue
> concerns me.
>
> I'm looking forward for any insights, thanks.
> On Monday, June 26, 2017, 11:56:45 AM PDT, Serega Sheypak <
> serega.sheypak@gmail.com> wrote:
>
>
> Hi, I have more or less the same symptom
>
>
> if (Utils.isScala2_10()) {
> binder = (Map<String, Object>) getValue("_binder");
> } else {
> binder = (Map<String, Object>) getLastObject();
> }
> binder.put("sc", sc); // EXCEPTION HERE
>
>
> java.lang.NullPointerException at org.apache.zeppelin.spark.
> SparkInterpreter.open(SparkInterpreter.java:887) at org.apache.zeppelin.
> interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70) at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$
> InterpretJob.jobRun(RemoteInterpreterServer.java:489) at
> org.apache.zeppelin.scheduler.Job.run(Job.java:181) at
> org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) at
> java.util.concurrent.ScheduledThreadPoolExecutor$
> ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(
> ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.
> ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
>
> I'm going to check this one stuff:
> https://stackoverflow.com/questions/43289067/getting-
> nullpointerexception-when-running-spark-code-in-zeppelin-0-7-1
>
>
> 2017-06-26 20:41 GMT+02:00 Richard Xin <ri...@yahoo.com>:
>
> Anyone?
>
>
>
> On Thursday, June 22, 2017, 3:00:01 PM PDT, Richard Xin <
> richardxin168@yahoo.com> wrote:
>
>
> apache/zeppelin
> <https://github.com/apache/zeppelin/blob/master/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java#L1294>
>
> apache/zeppelin
>
> zeppelin - Mirror of Apache Zeppelin
>
> <https://github.com/apache/zeppelin/blob/master/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java#L1294>
>
> in the code I see // Some case, scala.tools.nsc.interpreter.
> IMain$ReadEvalPrint.call throws an NPE
>
> We had issues that Zeppelin note errored out immediately without any hint
> messages, and we are seeing this NPE in the spark_interpreter log (Zeppelin
> 0.7.1 on AWS EMR cluster). can someone explain what's the likely cause for
> this error and how to prevent it from happening again?
>
> Thanks,
>
>
>
Re: NPE in SparkInterpreter.java
Posted by Richard Xin <ri...@yahoo.com>.
just try to understand the comment in the code:// Some case, scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call throws an NPE
what are the "Some case", it sounds like Zeppelin developers are not sure how it happened and how to prevent it?
I am trying to make a strong case for my company to switch from other notebook application to Zeppelin, Zeppelin looks good and only this issue concerns me.
I'm looking forward for any insights, thanks.On Monday, June 26, 2017, 11:56:45 AM PDT, Serega Sheypak <se...@gmail.com> wrote:
Hi, I have more or less the same symptom
if (Utils.isScala2_10()) {
binder = (Map<String, Object>) getValue("_binder");
} else {
binder = (Map<String, Object>) getLastObject();
}
binder.put("sc", sc); // EXCEPTION HERE
java.lang.NullPointerException at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:887) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:489) at org.apache.zeppelin.scheduler.Job.run(Job.java:181) at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)
I'm going to check this one stuff:https://stackoverflow.com/questions/43289067/getting-nullpointerexception-when-running-spark-code-in-zeppelin-0-7-1
2017-06-26 20:41 GMT+02:00 Richard Xin <ri...@yahoo.com>:
Anyone?
On Thursday, June 22, 2017, 3:00:01 PM PDT, Richard Xin <ri...@yahoo.com> wrote:
apache/zeppelin
|
|
|
| | |
|
|
|
| |
apache/zeppelin
zeppelin - Mirror of Apache Zeppelin
|
|
|
in the code I see // Some case, scala.tools.nsc.interpreter. IMain$ReadEvalPrint.call throws an NPE
We had issues that Zeppelin note errored out immediately without any hint messages, and we are seeing this NPE in the spark_interpreter log (Zeppelin 0.7.1 on AWS EMR cluster). can someone explain what's the likely cause for this error and how to prevent it from happening again?
Thanks,
Re: NPE in SparkInterpreter.java
Posted by Serega Sheypak <se...@gmail.com>.
Hi, I have more or less the same symptom
if (Utils.isScala2_10()) {
binder = (Map<String, Object>) getValue("_binder");
} else {
binder = (Map<String, Object>) getLastObject();
}
binder.put("sc", sc); // EXCEPTION HERE
java.lang.NullPointerException at
org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:887)
at
org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70)
at
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:489)
at org.apache.zeppelin.scheduler.Job.run(Job.java:181) at
org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262) at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
I'm going to check this one stuff:
https://stackoverflow.com/questions/43289067/getting-nullpointerexception-when-running-spark-code-in-zeppelin-0-7-1
2017-06-26 20:41 GMT+02:00 Richard Xin <ri...@yahoo.com>:
> Anyone?
>
>
>
> On Thursday, June 22, 2017, 3:00:01 PM PDT, Richard Xin <
> richardxin168@yahoo.com> wrote:
>
>
> apache/zeppelin
> <https://github.com/apache/zeppelin/blob/master/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java#L1294>
>
> apache/zeppelin
>
> zeppelin - Mirror of Apache Zeppelin
>
> <https://github.com/apache/zeppelin/blob/master/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java#L1294>
>
> in the code I see // Some case, scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call
> throws an NPE
>
> We had issues that Zeppelin note errored out immediately without any hint
> messages, and we are seeing this NPE in the spark_interpreter log (Zeppelin
> 0.7.1 on AWS EMR cluster). can someone explain what's the likely cause for
> this error and how to prevent it from happening again?
>
> Thanks,
>
>
Re: NPE in SparkInterpreter.java
Posted by Richard Xin <ri...@yahoo.com>.
Anyone?
On Thursday, June 22, 2017, 3:00:01 PM PDT, Richard Xin <ri...@yahoo.com> wrote:
apache/zeppelin
|
|
|
| | |
|
|
|
| |
apache/zeppelin
zeppelin - Mirror of Apache Zeppelin
|
|
|
in the code I see // Some case, scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call throws an NPE
We had issues that Zeppelin note errored out immediately without any hint messages, and we are seeing this NPE in the spark_interpreter log (Zeppelin 0.7.1 on AWS EMR cluster). can someone explain what's the likely cause for this error and how to prevent it from happening again?
Thanks,