You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by Jeff Zhang <zj...@gmail.com> on 2015/08/10 14:16:45 UTC
Zeppelin job hang there
I have 2 paragraph, one is spark job the other is spark sql.
The first job (spark job) run successfully, but the second spark sql job
hangs.
*The first paragraph:*
> case class Order(orderDate:String, region:String, rep:String,
item:String, units:Int, cost: Double, total:Double)
> val orders =
sc.textFile("file:///Users/jzhang/Downloads/SampleData.csv").map{
> line=>line.split(",")}.map{
> tokens=>Order(tokens(0).trim, tokens(1).trim,
tokens(2).trim, tokens(3).trim, tokens(4).toInt, tokens(5).toDouble,
tokens(6).toDouble)}.toDF
> orders.registerTempTable("orders")
*The second paragraph:*
> %sql
> select count(*) from orders
*Logs in spark interpreter *
INFO [2015-08-10 20:09:24,710] ({pool-2-thread-3}
SchedulerFactory.java[jobStarted]:132) - Job
remoteInterpretJob_1439208564691 started by scheduler
org.apache.zeppelin.spark.SparkInterpreter646357311
INFO [2015-08-10 20:09:25,069] ({pool-2-thread-3}
Logging.scala[logInfo]:59) - Using SparkSqlSerializer2.
INFO [2015-08-10 20:09:25,151] ({pool-2-thread-3}
Logging.scala[logInfo]:59) - Starting job: take at
NativeMethodAccessorImpl.java:-2
INFO [2015-08-10 20:14:07,715] ({pool-1-thread-5}
Logging.scala[logInfo]:59) - Asked to cancel job group
zeppelin-20150810-172402_743954903
INFO [2015-08-10 20:14:07,908] ({pool-1-thread-5}
Logging.scala[logInfo]:59) - Asked to cancel job group
zeppelin-20150810-172402_743954903
*Logs in zeppelin*
INFO [2015-08-10 20:09:24,684] ({pool-1-thread-2}
Paragraph.java[jobRun]:194) - run paragraph 20150810-172402_743954903 using
sql org.apache.zeppelin.interpreter.LazyOpenInterpreter@5a2251a4
INFO [2015-08-10 20:09:24,685] ({pool-1-thread-2}
Paragraph.java[jobRun]:211) - RUN :
select count(1) from orders
INFO [2015-08-10 20:09:24,785] ({Thread-40}
NotebookServer.java[broadcast]:251) - SEND >> NOTE
INFO [2015-08-10 20:09:25,192] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:25,698] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:26,204] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:26,710] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:27,219] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:27,728] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:28,235] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:28,741] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:29,248] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:29,754] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:30,262] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:30,767] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:31,272] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:31,779] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:32,284] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:32,788] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:33,294] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:33,798] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:34,304] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:34,808] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:35,313] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:35,818] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:36,321] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
INFO [2015-08-10 20:09:36,827] ({Thread-41}
NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
--
Best Regards
Jeff Zhang
Re: Zeppelin job hang there
Posted by moon soo Lee <mo...@apache.org>.
Hi,
Can not see anything wrong in your logs.
Do you have any non-default configuration for Spark interpreter?
Or do you have hive-site.xml in you classpath?
Thanks,
moon
On Mon, Aug 10, 2015 at 5:16 AM Jeff Zhang <zj...@gmail.com> wrote:
>
> I have 2 paragraph, one is spark job the other is spark sql.
> The first job (spark job) run successfully, but the second spark sql job
> hangs.
>
> *The first paragraph:*
> > case class Order(orderDate:String, region:String, rep:String,
> item:String, units:Int, cost: Double, total:Double)
> > val orders =
> sc.textFile("file:///Users/jzhang/Downloads/SampleData.csv").map{
> > line=>line.split(",")}.map{
> > tokens=>Order(tokens(0).trim, tokens(1).trim,
> tokens(2).trim, tokens(3).trim, tokens(4).toInt, tokens(5).toDouble,
> tokens(6).toDouble)}.toDF
> > orders.registerTempTable("orders")
>
> *The second paragraph:*
> > %sql
> > select count(*) from orders
>
> *Logs in spark interpreter *
> INFO [2015-08-10 20:09:24,710] ({pool-2-thread-3}
> SchedulerFactory.java[jobStarted]:132) - Job
> remoteInterpretJob_1439208564691 started by scheduler
> org.apache.zeppelin.spark.SparkInterpreter646357311
> INFO [2015-08-10 20:09:25,069] ({pool-2-thread-3}
> Logging.scala[logInfo]:59) - Using SparkSqlSerializer2.
> INFO [2015-08-10 20:09:25,151] ({pool-2-thread-3}
> Logging.scala[logInfo]:59) - Starting job: take at
> NativeMethodAccessorImpl.java:-2
> INFO [2015-08-10 20:14:07,715] ({pool-1-thread-5}
> Logging.scala[logInfo]:59) - Asked to cancel job group
> zeppelin-20150810-172402_743954903
> INFO [2015-08-10 20:14:07,908] ({pool-1-thread-5}
> Logging.scala[logInfo]:59) - Asked to cancel job group
> zeppelin-20150810-172402_743954903
>
> *Logs in zeppelin*
>
> INFO [2015-08-10 20:09:24,684] ({pool-1-thread-2}
> Paragraph.java[jobRun]:194) - run paragraph 20150810-172402_743954903 using
> sql org.apache.zeppelin.interpreter.LazyOpenInterpreter@5a2251a4
> INFO [2015-08-10 20:09:24,685] ({pool-1-thread-2}
> Paragraph.java[jobRun]:211) - RUN :
> select count(1) from orders
> INFO [2015-08-10 20:09:24,785] ({Thread-40}
> NotebookServer.java[broadcast]:251) - SEND >> NOTE
> INFO [2015-08-10 20:09:25,192] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:25,698] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:26,204] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:26,710] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:27,219] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:27,728] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:28,235] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:28,741] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:29,248] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:29,754] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:30,262] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:30,767] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:31,272] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:31,779] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:32,284] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:32,788] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:33,294] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:33,798] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:34,304] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:34,808] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:35,313] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:35,818] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:36,321] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
> INFO [2015-08-10 20:09:36,827] ({Thread-41}
> NotebookServer.java[broadcast]:251) - SEND >> PROGRESS
>
>
> --
> Best Regards
>
> Jeff Zhang
>