You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Bill Jay <bi...@gmail.com> on 2014/07/22 21:52:59 UTC

Spark Streaming: no job has started yet

Hi all,

I am running a spark streaming job. The job hangs on one stage, which shows
as follows:

Details for Stage 4
Summary MetricsNo tasks have started yetTasksNo tasks have started yet



Does anyone have an idea on this?

Thanks!

Bill
Bill

Re: Spark Streaming: no job has started yet

Posted by Bill Jay <bi...@gmail.com>.

The code is pretty long. But the main idea is to consume from Kafka,
preprocess the data, and groupBy a field. I use mutliple DStream to add
parallelism to the consumer. It seems when the number of DStreams is large,
this happens often.

Thanks,

Bill

On Tue, Jul 22, 2014 at 11:13 PM, Akhil Das <ak...@sigmoidanalytics.com>
wrote:

> Can you paste the piece of code?
>
> Thanks
> Best Regards
>
>
> On Wed, Jul 23, 2014 at 1:22 AM, Bill Jay <bi...@gmail.com>
> wrote:
>
>> Hi all,
>>
>> I am running a spark streaming job. The job hangs on one stage, which
>> shows as follows:
>>
>> Details for Stage 4
>> Summary Metrics No tasks have started yetTasksNo tasks have started yet
>>
>>
>>
>> Does anyone have an idea on this?
>>
>> Thanks!
>>
>> Bill
>> Bill
>>
>
>

Re: Spark Streaming: no job has started yet

Posted by Akhil Das <ak...@sigmoidanalytics.com>.

Can you paste the piece of code?

Thanks
Best Regards


On Wed, Jul 23, 2014 at 1:22 AM, Bill Jay <bi...@gmail.com>
wrote:

> Hi all,
>
> I am running a spark streaming job. The job hangs on one stage, which
> shows as follows:
>
> Details for Stage 4
> Summary Metrics No tasks have started yetTasksNo tasks have started yet
>
>
>
> Does anyone have an idea on this?
>
> Thanks!
>
> Bill
> Bill
>