You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Akash Mishra <ak...@gmail.com> on 2015/08/26 15:37:14 UTC
Spark Job Stuck in on stage.
I am currently running a Spark job on Mesos. On Stages UI page, I see that *GC
Time* is too much for each task. 16.2hr in 18.2hr Duration.
Please see attached pic
[image: Inline image 1]
Does this means that there are lots of Garbage Collection with are
happening and hence the job are stuck and not able to complete.
Thanks,
--
Regards,
Akash Mishra.
"Its not our abilities that make us, but our decisions."--Albus Dumbledore
Re: Spark Job Stuck in on stage.
Posted by Akash Mishra <ak...@gmail.com>.
PFA executor List.
[image: Inline image 2]
On Wed, Aug 26, 2015 at 4:37 PM, Sameer Farooqui <sa...@databricks.com>
wrote:
> Hmm... how much memory does each Executor JVM have? And how many total
> Executors are there? Also is the task you're running very computationally
> expensive?
>
> A task should ideally run for like 100 ms - 10 seconds or so. Hours for
> each task looks off to me. Also, why did some tasks finish in 8 hours,
> which others are running for 18th?
>
>
>
> On Wed, Aug 26, 2015 at 2:37 PM, Akash Mishra <ak...@gmail.com>
> wrote:
>
>> I am currently running a Spark job on Mesos. On Stages UI page, I see
>> that *GC Time* is too much for each task. 16.2hr in 18.2hr Duration.
>>
>> Please see attached pic
>>
>> [image: Inline image 1]
>>
>>
>> Does this means that there are lots of Garbage Collection with are
>> happening and hence the job are stuck and not able to complete.
>>
>> Thanks,
>> --
>>
>> Regards,
>> Akash Mishra.
>>
>>
>> "Its not our abilities that make us, but our decisions."--Albus Dumbledore
>>
>
>
--
Regards,
Akash Mishra.
"Its not our abilities that make us, but our decisions."--Albus Dumbledore
Re: Spark Job Stuck in on stage.
Posted by Sameer Farooqui <sa...@databricks.com>.
Hmm... how much memory does each Executor JVM have? And how many total
Executors are there? Also is the task you're running very computationally
expensive?
A task should ideally run for like 100 ms - 10 seconds or so. Hours for
each task looks off to me. Also, why did some tasks finish in 8 hours,
which others are running for 18th?
On Wed, Aug 26, 2015 at 2:37 PM, Akash Mishra <ak...@gmail.com>
wrote:
> I am currently running a Spark job on Mesos. On Stages UI page, I see that *GC
> Time* is too much for each task. 16.2hr in 18.2hr Duration.
>
> Please see attached pic
>
> [image: Inline image 1]
>
>
> Does this means that there are lots of Garbage Collection with are
> happening and hence the job are stuck and not able to complete.
>
> Thanks,
> --
>
> Regards,
> Akash Mishra.
>
>
> "Its not our abilities that make us, but our decisions."--Albus Dumbledore
>