You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Jacek Laskowski <ja...@japila.pl> on 2016/01/17 13:49:30 UTC
Incorrect timeline for Scheduling Delay in Streaming page in web UI?
Hi,
I'm trying to understand how Scheduling Delays are displayed in
Streaming page in web UI and think the values are displayed
incorrectly in the Timelines column. I'm only concerned with the
scheduling delays (on y axis) per batch times (x axis). It appears
that the values (on y axis) are correct, but not how they are
displayed per batch times.
See the second screenshot in
https://jaceklaskowski.gitbooks.io/mastering-apache-spark/content/spark-streaming-webui.html#scheduling-delay.
Can anyone explain how the delays for batches per batch time should be
read? I'm specifically asking about the timeline (not histogram as it
seems fine).
Pozdrawiam,
Jacek
Jacek Laskowski | https://medium.com/@jaceklaskowski/
Mastering Apache Spark
==> https://jaceklaskowski.gitbooks.io/mastering-apache-spark/
Follow me at https://twitter.com/jaceklaskowski
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Incorrect timeline for Scheduling Delay in Streaming page in web UI?
Posted by Jacek Laskowski <ja...@japila.pl>.
Hi Ryan,
Ah, you might be right! I didn't think about the batches queued up
(and hence without scheduling delay since they're not started yet).
Thanks a lot for responding!
Pozdrawiam,
Jacek
Jacek Laskowski | https://medium.com/@jaceklaskowski/
Mastering Apache Spark
==> https://jaceklaskowski.gitbooks.io/mastering-apache-spark/
Follow me at https://twitter.com/jaceklaskowski
On Mon, Jan 18, 2016 at 8:03 PM, Shixiong(Ryan) Zhu
<sh...@databricks.com> wrote:
> Hey, did you mean that the scheduling delay timeline is incorrect because
> it's too short and some values are missing? A batch won't have a scheduling
> delay until it starts to run. In your example, a lot of batches are waiting
> so that they don't have the scheduling delay.
>
> On Sun, Jan 17, 2016 at 4:49 AM, Jacek Laskowski <ja...@japila.pl> wrote:
>>
>> Hi,
>>
>> I'm trying to understand how Scheduling Delays are displayed in
>> Streaming page in web UI and think the values are displayed
>> incorrectly in the Timelines column. I'm only concerned with the
>> scheduling delays (on y axis) per batch times (x axis). It appears
>> that the values (on y axis) are correct, but not how they are
>> displayed per batch times.
>>
>> See the second screenshot in
>>
>> https://jaceklaskowski.gitbooks.io/mastering-apache-spark/content/spark-streaming-webui.html#scheduling-delay.
>>
>> Can anyone explain how the delays for batches per batch time should be
>> read? I'm specifically asking about the timeline (not histogram as it
>> seems fine).
>>
>> Pozdrawiam,
>> Jacek
>>
>> Jacek Laskowski | https://medium.com/@jaceklaskowski/
>> Mastering Apache Spark
>> ==> https://jaceklaskowski.gitbooks.io/mastering-apache-spark/
>> Follow me at https://twitter.com/jaceklaskowski
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> For additional commands, e-mail: user-help@spark.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: Incorrect timeline for Scheduling Delay in Streaming page in web UI?
Posted by "Shixiong(Ryan) Zhu" <sh...@databricks.com>.
Hey, did you mean that the scheduling delay timeline is incorrect because
it's too short and some values are missing? A batch won't have a scheduling
delay until it starts to run. In your example, a lot of batches are waiting
so that they don't have the scheduling delay.
On Sun, Jan 17, 2016 at 4:49 AM, Jacek Laskowski <ja...@japila.pl> wrote:
> Hi,
>
> I'm trying to understand how Scheduling Delays are displayed in
> Streaming page in web UI and think the values are displayed
> incorrectly in the Timelines column. I'm only concerned with the
> scheduling delays (on y axis) per batch times (x axis). It appears
> that the values (on y axis) are correct, but not how they are
> displayed per batch times.
>
> See the second screenshot in
>
> https://jaceklaskowski.gitbooks.io/mastering-apache-spark/content/spark-streaming-webui.html#scheduling-delay
> .
>
> Can anyone explain how the delays for batches per batch time should be
> read? I'm specifically asking about the timeline (not histogram as it
> seems fine).
>
> Pozdrawiam,
> Jacek
>
> Jacek Laskowski | https://medium.com/@jaceklaskowski/
> Mastering Apache Spark
> ==> https://jaceklaskowski.gitbooks.io/mastering-apache-spark/
> Follow me at https://twitter.com/jaceklaskowski
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>