You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by "Jain, Nishit" <nj...@underarmour.com> on 2017/06/05 21:51:53 UTC
Spark Streaming Job Stuck
I have a very simple spark streaming job running locally in standalone mode. There is a customer receiver which reads from database and pass it to the main job which prints the total. Not an actual use case but I am playing around to learn. Problem is that job gets stuck forever, logic is very simple so I think it is neither doing any processing nor memory issue. What is strange is if I STOP the job, suddenly in logs I see the output of job execution and other backed jobs follow! Can some one help me understand what is going on here?
val spark = SparkSession
.builder()
.master("local[1]")
.appName("SocketStream")
.getOrCreate()
val ssc = new StreamingContext(spark.sparkContext,Seconds(5))
val lines = ssc.receiverStream(new HanaCustomReceiver())
lines.foreachRDD{x => println("==============" + x.count())}
ssc.start()
ssc.awaitTermination()
[enter image description here]<https://i.stack.imgur.com/y1GGr.png>
After terminating program following logs roll which shows execution of the batch -
17/06/05 15:56:16 INFO JobGenerator: Stopping JobGenerator immediately 17/06/05 15:56:16 INFO RecurringTimer: Stopped timer for JobGenerator after time 1496696175000 17/06/05 15:56:16 INFO JobGenerator: Stopped JobGenerator ==============100
Thanks!
Re: Spark Streaming Job Stuck
Posted by Richard Moorhead <ri...@c2fo.com>.
Set your master to local[10]; you are only allocating one core currently.
. . . . . . . . . . . . . . . . . . . . . . . . . . .
Richard Moorhead
Software Engineer
richard.moorhead@c2fo.com<ma...@gmail.com>
C2FO: The World's Market for Working Capital®
[http://c2fo.com/wp-content/uploads/sites/1/2016/03/LinkedIN.png] <https://www.linkedin.com/company/c2fo?trk=vsrp_companies_res_name&trkInfo=VSRPsearchId%3A125658601427902817660%2CVSRPtargetId%3A1555109%2CVSRPcmpt%3Aprimary> [http://c2fo.com/wp-content/uploads/sites/1/2016/03/YouTube.png] <https://www.youtube.com/c/C2FOMarket> [http://c2fo.com/wp-content/uploads/sites/1/2016/03/Twitter.png] <https://twitter.com/C2FO> [http://c2fo.com/wp-content/uploads/sites/1/2016/03/Googleplus.png] <https://plus.google.com/+C2foMarket/posts> [http://c2fo.com/wp-content/uploads/sites/1/2016/03/Facebook.png] <https://www.facebook.com/C2FOMarketplace> [http://c2fo.com/wp-content/uploads/sites/1/2016/03/Forbes-Fintech-50.png] <https://c2fo.com/media-coverage/c2fo-included-forbes-fintech-50>
The information contained in this message and any attachment may be privileged, confidential, and protected from disclosure. If you are not the intended recipient, or an employee, or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to the message and deleting from your computer.
________________________________
From: Jain, Nishit <nj...@underarmour.com>
Sent: Tuesday, June 6, 2017 9:54 AM
To: Tathagata Das
Cc: user@spark.apache.org
Subject: Re: Spark Streaming Job Stuck
That helped, thanks TD! :D
From: Tathagata Das <ta...@gmail.com>>
Date: Tuesday, June 6, 2017 at 3:26 AM
To: "Jain, Nishit" <nj...@underarmour.com>>
Cc: "user@spark.apache.org<ma...@spark.apache.org>" <us...@spark.apache.org>>
Subject: Re: Spark Streaming Job Stuck
http://spark.apache.org/docs/latest/streaming-programming-guide.html#points-to-remember-1
Hope this helps.
On Mon, Jun 5, 2017 at 2:51 PM, Jain, Nishit <nj...@underarmour.com>> wrote:
I have a very simple spark streaming job running locally in standalone mode. There is a customer receiver which reads from database and pass it to the main job which prints the total. Not an actual use case but I am playing around to learn. Problem is that job gets stuck forever, logic is very simple so I think it is neither doing any processing nor memory issue. What is strange is if I STOP the job, suddenly in logs I see the output of job execution and other backed jobs follow! Can some one help me understand what is going on here?
val spark = SparkSession
.builder()
.master("local[1]")
.appName("SocketStream")
.getOrCreate()
val ssc = new StreamingContext(spark.sparkContext,Seconds(5))
val lines = ssc.receiverStream(new HanaCustomReceiver())
lines.foreachRDD{x => println("==============" + x.count())}
ssc.start()
ssc.awaitTermination()
[enter image description here]<https://i.stack.imgur.com/y1GGr.png>
After terminating program following logs roll which shows execution of the batch -
17/06/05 15:56:16 INFO JobGenerator: Stopping JobGenerator immediately 17/06/05 15:56:16 INFO RecurringTimer: Stopped timer for JobGenerator after time 1496696175000 17/06/05 15:56:16 INFO JobGenerator: Stopped JobGenerator ==============100
Thanks!
Re: Spark Streaming Job Stuck
Posted by "Jain, Nishit" <nj...@underarmour.com>.
That helped, thanks TD! :D
From: Tathagata Das <ta...@gmail.com>>
Date: Tuesday, June 6, 2017 at 3:26 AM
To: "Jain, Nishit" <nj...@underarmour.com>>
Cc: "user@spark.apache.org<ma...@spark.apache.org>" <us...@spark.apache.org>>
Subject: Re: Spark Streaming Job Stuck
http://spark.apache.org/docs/latest/streaming-programming-guide.html#points-to-remember-1
Hope this helps.
On Mon, Jun 5, 2017 at 2:51 PM, Jain, Nishit <nj...@underarmour.com>> wrote:
I have a very simple spark streaming job running locally in standalone mode. There is a customer receiver which reads from database and pass it to the main job which prints the total. Not an actual use case but I am playing around to learn. Problem is that job gets stuck forever, logic is very simple so I think it is neither doing any processing nor memory issue. What is strange is if I STOP the job, suddenly in logs I see the output of job execution and other backed jobs follow! Can some one help me understand what is going on here?
val spark = SparkSession
.builder()
.master("local[1]")
.appName("SocketStream")
.getOrCreate()
val ssc = new StreamingContext(spark.sparkContext,Seconds(5))
val lines = ssc.receiverStream(new HanaCustomReceiver())
lines.foreachRDD{x => println("==============" + x.count())}
ssc.start()
ssc.awaitTermination()
[enter image description here]<https://i.stack.imgur.com/y1GGr.png>
After terminating program following logs roll which shows execution of the batch -
17/06/05 15:56:16 INFO JobGenerator: Stopping JobGenerator immediately 17/06/05 15:56:16 INFO RecurringTimer: Stopped timer for JobGenerator after time 1496696175000 17/06/05 15:56:16 INFO JobGenerator: Stopped JobGenerator ==============100
Thanks!
Re: Spark Streaming Job Stuck
Posted by Tathagata Das <ta...@gmail.com>.
http://spark.apache.org/docs/latest/streaming-programming-guide.html#points-to-remember-1
Hope this helps.
On Mon, Jun 5, 2017 at 2:51 PM, Jain, Nishit <nj...@underarmour.com> wrote:
> I have a very simple spark streaming job running locally in standalone
> mode. There is a customer receiver which reads from database and pass it to
> the main job which prints the total. Not an actual use case but I am
> playing around to learn. Problem is that job gets stuck forever, logic is
> very simple so I think it is neither doing any processing nor memory issue.
> What is strange is if I STOP the job, suddenly in logs I see the output of
> job execution and other backed jobs follow! Can some one help me understand
> what is going on here?
>
> val spark = SparkSession
> .builder()
> .master("local[1]")
> .appName("SocketStream")
> .getOrCreate()
>
> val ssc = new StreamingContext(spark.sparkContext,Seconds(5))
> val lines = ssc.receiverStream(new HanaCustomReceiver())
>
>
> lines.foreachRDD{x => println("==============" + x.count())}
>
> ssc.start()
> ssc.awaitTermination()
>
> [image: enter image description here]
> <https://i.stack.imgur.com/y1GGr.png>
>
> After terminating program following logs roll which shows execution of the
> batch -
>
> 17/06/05 15:56:16 INFO JobGenerator: Stopping JobGenerator immediately
> 17/06/05 15:56:16 INFO RecurringTimer: Stopped timer for JobGenerator after
> time 1496696175000 17/06/05 15:56:16 INFO JobGenerator: Stopped
> JobGenerator ==============100
>
> Thanks!
>