You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by Richard Xin <ri...@yahoo.com> on 2017/08/03 20:34:28 UTC

Zeppelin postgres interpreter throws NPE

on AWS EMR1. Tested remote postgres connection using psql, works OK2. added postges interpreter via Interpreter UI3. create new note:%postgresql   (or %psql)
select * from test.batch_report; 
got NPE
4. here are what's in the log:
WARN [2017-08-03 20:22:46,859] ({qtp459296537-18} LoginRestApi.java[postLogin]:119) - {"status":"OK","message":"","body":{"principal":"richard.xin","ticket":"d86ab3d9-3976-43a4-a274-df50ee3b627a","roles":"[]"}}
 INFO [2017-08-03 20:22:50,370] ({qtp459296537-21} NotebookServer.java[sendNote]:705) - New operation from 172.17.197.49 : 57990 : richard.xin : GET_NOTE : 2CNXQE5CR
 INFO [2017-08-03 20:22:50,656] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance spark for note 2CNXQE5CR
 INFO [2017-08-03 20:22:50,661] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkInterpreter 2080711349 created
 INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.PySparkInterpreter 506689376 created
 INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkSqlInterpreter 1462545045 created
 INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance postgresql for note 2CNXQE5CR
 INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter 340437485 created
 INFO [2017-08-03 20:22:53,610] ({pool-2-thread-2} SchedulerFactory.java[jobStarted]:131) - Job paragraph_1501783535283_1713771734 started by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422
 INFO [2017-08-03 20:22:53,611] ({pool-2-thread-2} Paragraph.java[jobRun]:362) - run paragraph 20170803-180535_552293631 using psql org.apache.zeppelin.interpreter.LazyOpenInterpreter@144aa9ed
 INFO [2017-08-03 20:22:53,620] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:126) - Run interpreter process [/usr/lib/zeppelin/bin/interpreter.sh, -d, /usr/lib/zeppelin/interpreter/postgresql, -p, 42610, -l, /usr/lib/zeppelin/local-repo/2CPQEJPGC]
 INFO [2017-08-03 20:22:54,190] ({pool-2-thread-2} RemoteInterpreter.java[init]:221) - Create remote interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter
 INFO [2017-08-03 20:22:54,370] ({pool-2-thread-2} RemoteInterpreter.java[pushAngularObjectRegistryToRemote]:551) - Push local angular object registry from ZeppelinServer to remote interpreter group 2CPQEJPGC:shared_process
 WARN [2017-08-03 20:22:54,606] ({pool-2-thread-2} NotebookServer.java[afterStatusChange]:2058) - Job 20170803-180535_552293631 is finished, status: ERROR, exception: null, result: %text java.lang.NullPointerException
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.executeSql(PostgreSqlInterpreter.java:202)
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.interpret(PostgreSqlInterpreter.java:289)
    at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:498)
    at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
    at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

 INFO [2017-08-03 20:22:54,673] ({pool-2-thread-2} SchedulerFactory.java[jobFinished]:137) - Job paragraph_1501783535283_1713771734 finished by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422

Re: Zeppelin postgres interpreter throws NPE

Posted by Richard Xin <ri...@yahoo.com>.
[ZEPPELIN-2828] unhandled NullPointerException in PostgreSqlInterpreter when unable to connect - ASF JIRA


| 
| 
|  | 
[ZEPPELIN-2828] unhandled NullPointerException in PostgreSqlInterpreter ...


 |

 |

 |





On Thursday, August 3, 2017, 4:53:05 PM PDT, Jianfeng (Jeff) Zhang <jz...@hortonworks.com> wrote:

Agree, you can create a ticket for better exception message. And the error is better to be displayed in frontend



Best Regard,Jeff Zhang

From: Richard Xin <ri...@yahoo.com>
Reply-To: "users@zeppelin.apache.org" <us...@zeppelin.apache.org>
Date: Friday, August 4, 2017 at 4:40 AM
To: "users@zeppelin.apache.org" <us...@zeppelin.apache.org>
Subject: Re: Zeppelin postgres interpreter throws NPE

oh, just figured it out myselfit seems that the property I aded in interpreter config postgres.database is not recognized by zeppelin, I solved it by adding dbname to url.
anyway, the exception is better to be handled and emit more friendly error message.

On Thursday, August 3, 2017, 1:35:04 PM PDT, Richard Xin <ri...@yahoo.com> wrote:

on AWS EMR1. Tested remote postgres connection using psql, works OK2. added postges interpreter via Interpreter UI3. create new note:%postgresql   (or %psql)
select * from test.batch_report; 
got NPE
4. here are what's in the log:
WARN [2017-08-03 20:22:46,859] ({qtp459296537-18} LoginRestApi.java[postLogin]:119) - {"status":"OK","message":"","body":{"principal":"richard.xin","ticket":"d86ab3d9-3976-43a4-a274-df50ee3b627a","roles":"[]"}}
INFO [2017-08-03 20:22:50,370] ({qtp459296537-21} NotebookServer.java[sendNote]:705) - New operation from 172.17.197.49 : 57990 : richard.xin : GET_NOTE : 2CNXQE5CR
INFO [2017-08-03 20:22:50,656] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance spark for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,661] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkInterpreter 2080711349 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.PySparkInterpreter 506689376 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkSqlInterpreter 1462545045 created
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance postgresql for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter 340437485 created
INFO [2017-08-03 20:22:53,610] ({pool-2-thread-2} SchedulerFactory.java[jobStarted]:131) - Job paragraph_1501783535283_1713771734 started by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422
INFO [2017-08-03 20:22:53,611] ({pool-2-thread-2} Paragraph.java[jobRun]:362) - run paragraph 20170803-180535_552293631 using psql org.apache.zeppelin.interpreter.LazyOpenInterpreter@144aa9ed
INFO [2017-08-03 20:22:53,620] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:126) - Run interpreter process [/usr/lib/zeppelin/bin/interpreter.sh, -d, /usr/lib/zeppelin/interpreter/postgresql, -p, 42610, -l, /usr/lib/zeppelin/local-repo/2CPQEJPGC]
INFO [2017-08-03 20:22:54,190] ({pool-2-thread-2} RemoteInterpreter.java[init]:221) - Create remote interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter
INFO [2017-08-03 20:22:54,370] ({pool-2-thread-2} RemoteInterpreter.java[pushAngularObjectRegistryToRemote]:551) - Push local angular object registry from ZeppelinServer to remote interpreter group 2CPQEJPGC:shared_process
WARN [2017-08-03 20:22:54,606] ({pool-2-thread-2} NotebookServer.java[afterStatusChange]:2058) - Job 20170803-180535_552293631 is finished, status: ERROR, exception: null, result: %text java.lang.NullPointerException
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.executeSql(PostgreSqlInterpreter.java:202)
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.interpret(PostgreSqlInterpreter.java:289)
    at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:498)
    at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
    at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

INFO [2017-08-03 20:22:54,673] ({pool-2-thread-2} SchedulerFactory.java[jobFinished]:137) - Job paragraph_1501783535283_1713771734 finished by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422

Re: Zeppelin postgres interpreter throws NPE

Posted by "Jianfeng (Jeff) Zhang" <jz...@hortonworks.com>.
Agree, you can create a ticket for better exception message. And the error is better to be displayed in frontend




Best Regard,
Jeff Zhang


From: Richard Xin <ri...@yahoo.com>>
Reply-To: "users@zeppelin.apache.org<ma...@zeppelin.apache.org>" <us...@zeppelin.apache.org>>
Date: Friday, August 4, 2017 at 4:40 AM
To: "users@zeppelin.apache.org<ma...@zeppelin.apache.org>" <us...@zeppelin.apache.org>>
Subject: Re: Zeppelin postgres interpreter throws NPE

oh, just figured it out myself
it seems that the property I aded in interpreter config postgres.database is not recognized by zeppelin, I solved it by adding dbname to url.

anyway, the exception is better to be handled and emit more friendly error message.


On Thursday, August 3, 2017, 1:35:04 PM PDT, Richard Xin <ri...@yahoo.com>> wrote:


on AWS EMR
1. Tested remote postgres connection using psql, works OK
2. added postges interpreter via Interpreter UI
3. create new note:
%postgresql   (or %psql)
select * from test.batch_report;
got NPE

4. here are what's in the log:

WARN [2017-08-03 20:22:46,859] ({qtp459296537-18} LoginRestApi.java[postLogin]:119) - {"status":"OK","message":"","body":{"principal":"richard.xin","ticket":"d86ab3d9-3976-43a4-a274-df50ee3b627a","roles":"[]"}}
INFO [2017-08-03 20:22:50,370] ({qtp459296537-21} NotebookServer.java[sendNote]:705) - New operation from 172.17.197.49 : 57990 : richard.xin : GET_NOTE : 2CNXQE5CR
INFO [2017-08-03 20:22:50,656] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance spark for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,661] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkInterpreter 2080711349 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.PySparkInterpreter 506689376 created
INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkSqlInterpreter 1462545045 created
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance postgresql for note 2CNXQE5CR
INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter 340437485 created
INFO [2017-08-03 20:22:53,610] ({pool-2-thread-2} SchedulerFactory.java[jobStarted]:131) - Job paragraph_1501783535283_1713771734 started by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422
INFO [2017-08-03 20:22:53,611] ({pool-2-thread-2} Paragraph.java[jobRun]:362) - run paragraph 20170803-180535_552293631 using psql org.apache.zeppelin.interpreter.LazyOpenInterpreter@144aa9ed
INFO [2017-08-03 20:22:53,620] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:126) - Run interpreter process [/usr/lib/zeppelin/bin/interpreter.sh, -d, /usr/lib/zeppelin/interpreter/postgresql, -p, 42610, -l, /usr/lib/zeppelin/local-repo/2CPQEJPGC]
INFO [2017-08-03 20:22:54,190] ({pool-2-thread-2} RemoteInterpreter.java[init]:221) - Create remote interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter
INFO [2017-08-03 20:22:54,370] ({pool-2-thread-2} RemoteInterpreter.java[pushAngularObjectRegistryToRemote]:551) - Push local angular object registry from ZeppelinServer to remote interpreter group 2CPQEJPGC:shared_process
WARN [2017-08-03 20:22:54,606] ({pool-2-thread-2} NotebookServer.java[afterStatusChange]:2058) - Job 20170803-180535_552293631 is finished, status: ERROR, exception: null, result: %text java.lang.NullPointerException
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.executeSql(PostgreSqlInterpreter.java:202)
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.interpret(PostgreSqlInterpreter.java:289)
    at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:498)
    at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
    at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

INFO [2017-08-03 20:22:54,673] ({pool-2-thread-2} SchedulerFactory.java[jobFinished]:137) - Job paragraph_1501783535283_1713771734 finished by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422

Re: Zeppelin postgres interpreter throws NPE

Posted by Richard Xin <ri...@yahoo.com>.
oh, just figured it out myselfit seems that the property I aded in interpreter config postgres.database is not recognized by zeppelin, I solved it by adding dbname to url.
anyway, the exception is better to be handled and emit more friendly error message.

On Thursday, August 3, 2017, 1:35:04 PM PDT, Richard Xin <ri...@yahoo.com> wrote:

on AWS EMR1. Tested remote postgres connection using psql, works OK2. added postges interpreter via Interpreter UI3. create new note:%postgresql   (or %psql)
select * from test.batch_report; 
got NPE
4. here are what's in the log:
WARN [2017-08-03 20:22:46,859] ({qtp459296537-18} LoginRestApi.java[postLogin]:119) - {"status":"OK","message":"","body":{"principal":"richard.xin","ticket":"d86ab3d9-3976-43a4-a274-df50ee3b627a","roles":"[]"}}
 INFO [2017-08-03 20:22:50,370] ({qtp459296537-21} NotebookServer.java[sendNote]:705) - New operation from 172.17.197.49 : 57990 : richard.xin : GET_NOTE : 2CNXQE5CR
 INFO [2017-08-03 20:22:50,656] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance spark for note 2CNXQE5CR
 INFO [2017-08-03 20:22:50,661] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkInterpreter 2080711349 created
 INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.PySparkInterpreter 506689376 created
 INFO [2017-08-03 20:22:50,662] ({qtp459296537-18} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.spark.SparkSqlInterpreter 1462545045 created
 INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:188) - Create interpreter instance postgresql for note 2CNXQE5CR
 INFO [2017-08-03 20:22:50,776] ({qtp459296537-22} InterpreterFactory.java[createInterpretersForNote]:221) - Interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter 340437485 created
 INFO [2017-08-03 20:22:53,610] ({pool-2-thread-2} SchedulerFactory.java[jobStarted]:131) - Job paragraph_1501783535283_1713771734 started by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422
 INFO [2017-08-03 20:22:53,611] ({pool-2-thread-2} Paragraph.java[jobRun]:362) - run paragraph 20170803-180535_552293631 using psql org.apache.zeppelin.interpreter.LazyOpenInterpreter@144aa9ed
 INFO [2017-08-03 20:22:53,620] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:126) - Run interpreter process [/usr/lib/zeppelin/bin/interpreter.sh, -d, /usr/lib/zeppelin/interpreter/postgresql, -p, 42610, -l, /usr/lib/zeppelin/local-repo/2CPQEJPGC]
 INFO [2017-08-03 20:22:54,190] ({pool-2-thread-2} RemoteInterpreter.java[init]:221) - Create remote interpreter org.apache.zeppelin.postgresql.PostgreSqlInterpreter
 INFO [2017-08-03 20:22:54,370] ({pool-2-thread-2} RemoteInterpreter.java[pushAngularObjectRegistryToRemote]:551) - Push local angular object registry from ZeppelinServer to remote interpreter group 2CPQEJPGC:shared_process
 WARN [2017-08-03 20:22:54,606] ({pool-2-thread-2} NotebookServer.java[afterStatusChange]:2058) - Job 20170803-180535_552293631 is finished, status: ERROR, exception: null, result: %text java.lang.NullPointerException
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.executeSql(PostgreSqlInterpreter.java:202)
    at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.interpret(PostgreSqlInterpreter.java:289)
    at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:498)
    at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
    at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

 INFO [2017-08-03 20:22:54,673] ({pool-2-thread-2} SchedulerFactory.java[jobFinished]:137) - Job paragraph_1501783535283_1713771734 finished by scheduler org.apache.zeppelin.interpreter.remote.RemoteInterpretershared_session392982422