You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by unk1102 <um...@gmail.com> on 2016/01/06 19:33:12 UTC
Why is this job running since one hour?
Hi I have one main Spark job which spawns multiple child spark jobs. One of
the child spark job is running for an hour and it keeps on hanging there I
have taken snap shot please see
<http://apache-spark-user-list.1001560.n3.nabble.com/file/n25899/Screen_Shot_2016-01-06_at_11.jpg>
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Why-is-this-job-running-since-one-hour-tp25899.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Why is this job running since one hour?
Posted by Umesh Kacha <um...@gmail.com>.
Hi thanks for the response. Each Job is processing around 5gb of skewed
data does group by multiple fields and does aggregation and does
coalesce(1) and saves csv file in gzip format. I think coalesce is causing
problem but data is not that huge I don't understand why it keeps on
running for an hour and avoiding other jobs to run. Please guide.
On Jan 7, 2016 3:58 AM, "Jakob Odersky" <jo...@gmail.com> wrote:
> What is the job doing? How much data are you processing?
>
> On 6 January 2016 at 10:33, unk1102 <um...@gmail.com> wrote:
>
>> Hi I have one main Spark job which spawns multiple child spark jobs. One
>> of
>> the child spark job is running for an hour and it keeps on hanging there I
>> have taken snap shot please see
>> <
>> http://apache-spark-user-list.1001560.n3.nabble.com/file/n25899/Screen_Shot_2016-01-06_at_11.jpg
>> >
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/Why-is-this-job-running-since-one-hour-tp25899.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> For additional commands, e-mail: user-help@spark.apache.org
>>
>>
>
Re: Why is this job running since one hour?
Posted by Jakob Odersky <jo...@gmail.com>.
What is the job doing? How much data are you processing?
On 6 January 2016 at 10:33, unk1102 <um...@gmail.com> wrote:
> Hi I have one main Spark job which spawns multiple child spark jobs. One of
> the child spark job is running for an hour and it keeps on hanging there I
> have taken snap shot please see
> <
> http://apache-spark-user-list.1001560.n3.nabble.com/file/n25899/Screen_Shot_2016-01-06_at_11.jpg
> >
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Why-is-this-job-running-since-one-hour-tp25899.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>