You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Snehal Nagmote <na...@gmail.com> on 2015/09/02 21:55:11 UTC
Understanding Batch Processing Time
Hi All,
I have spark job where I read data from Kafka every 5 seconds interval and
query Cassandra based on Kafka data using spark Cassandra Connector ,
I am using spark 1.4 , Often the batch gets stuck in processing after job
Id 352 . Spark takes long time to spawn job 353 where it reads from
Cassandra .
I want to understand since job processing time are in seconds, what else
contribute to processing time ?
What could be the potential issues which is adding processing time here ?
Any inputs would be helpful.
[image: Inline images 2]
Thanks,
Snehal
Re: Understanding Batch Processing Time
Posted by Snehal Nagmote <na...@gmail.com>.
Hi TD,
Please find attached jstack for application which I submit as driver in
client mode.
I think , it the same driver you were referring to ? Can you confirm if not
though,
Thanks,
Snehal
On 2 September 2015 at 14:38, Snehal Nagmote <na...@gmail.com>
wrote:
> Hi ,
>
> Thank you for your reply ,
>
> Yes , I confirm , the system is stuck after job 352 finishes and before
> job 353 starts (shows up in the UI)
>
> I will start job again and will take jstack if I can reproduce the problem
>
> Thanks,
> Snehal
>
>
> On 2 September 2015 at 13:34, Tathagata Das <td...@databricks.com> wrote:
>
>> Can you jstack into the driver and see what is process doing after job
>> 352?
>>
>> Also to confirm, the system is stuck after job 352 finishes, and before
>> job 353 starts (shows up in the UI), right?
>>
>> TD
>>
>> On Wed, Sep 2, 2015 at 12:55 PM, Snehal Nagmote <nagmote.snehal@gmail.com
>> > wrote:
>>
>>> Hi All,
>>>
>>> I have spark job where I read data from Kafka every 5 seconds interval
>>> and query Cassandra based on Kafka data using spark Cassandra Connector ,
>>>
>>> I am using spark 1.4 , Often the batch gets stuck in processing after
>>> job Id 352 . Spark takes long time to spawn job 353 where it reads from
>>> Cassandra .
>>>
>>> I want to understand since job processing time are in seconds, what else
>>> contribute to processing time ?
>>>
>>> What could be the potential issues which is adding processing time here
>>> ? Any inputs would be helpful.
>>>
>>>
>>>
>>>
>>> [image: Inline images 2]
>>>
>>> Thanks,
>>> Snehal
>>>
>>
>>
>
Re: Understanding Batch Processing Time
Posted by Snehal Nagmote <na...@gmail.com>.
Hi ,
Thank you for your reply ,
Yes , I confirm , the system is stuck after job 352 finishes and before job
353 starts (shows up in the UI)
I will start job again and will take jstack if I can reproduce the problem
Thanks,
Snehal
On 2 September 2015 at 13:34, Tathagata Das <td...@databricks.com> wrote:
> Can you jstack into the driver and see what is process doing after job 352?
>
> Also to confirm, the system is stuck after job 352 finishes, and before
> job 353 starts (shows up in the UI), right?
>
> TD
>
> On Wed, Sep 2, 2015 at 12:55 PM, Snehal Nagmote <na...@gmail.com>
> wrote:
>
>> Hi All,
>>
>> I have spark job where I read data from Kafka every 5 seconds interval
>> and query Cassandra based on Kafka data using spark Cassandra Connector ,
>>
>> I am using spark 1.4 , Often the batch gets stuck in processing after job
>> Id 352 . Spark takes long time to spawn job 353 where it reads from
>> Cassandra .
>>
>> I want to understand since job processing time are in seconds, what else
>> contribute to processing time ?
>>
>> What could be the potential issues which is adding processing time here ?
>> Any inputs would be helpful.
>>
>>
>>
>>
>> [image: Inline images 2]
>>
>> Thanks,
>> Snehal
>>
>
>
Re: Understanding Batch Processing Time
Posted by Tathagata Das <td...@databricks.com>.
Can you jstack into the driver and see what is process doing after job 352?
Also to confirm, the system is stuck after job 352 finishes, and before job
353 starts (shows up in the UI), right?
TD
On Wed, Sep 2, 2015 at 12:55 PM, Snehal Nagmote <na...@gmail.com>
wrote:
> Hi All,
>
> I have spark job where I read data from Kafka every 5 seconds interval and
> query Cassandra based on Kafka data using spark Cassandra Connector ,
>
> I am using spark 1.4 , Often the batch gets stuck in processing after job
> Id 352 . Spark takes long time to spawn job 353 where it reads from
> Cassandra .
>
> I want to understand since job processing time are in seconds, what else
> contribute to processing time ?
>
> What could be the potential issues which is adding processing time here ?
> Any inputs would be helpful.
>
>
>
>
> [image: Inline images 2]
>
> Thanks,
> Snehal
>