You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Apache Spark (JIRA)" <ji...@apache.org> on 2016/03/02 14:52:18 UTC
[jira] [Assigned] (SPARK-13618) Make Streaming web UI display
rate-limit lines in the statistics graph
[ https://issues.apache.org/jira/browse/SPARK-13618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-13618:
------------------------------------
Assignee: Apache Spark
> Make Streaming web UI display rate-limit lines in the statistics graph
> ----------------------------------------------------------------------
>
> Key: SPARK-13618
> URL: https://issues.apache.org/jira/browse/SPARK-13618
> Project: Spark
> Issue Type: Improvement
> Components: Streaming, Web UI
> Affects Versions: 1.6.0, 2.0.0
> Reporter: Liwei Lin
> Assignee: Apache Spark
> Attachments: 1.png, 2.png
>
>
> This JIRA propose to make Streaming web UI display rate-limit lines in the statistics graph.
> Specifically, this JIRA propose to:
> 1. add in `RateLimiter` a data structure keeping history of rate limit changes, so that calculating the upper bound of how many records we can receive in a block interval is possible;
> 2. add the `numRecordsLimit` information into the path from `BlockGenerator` generates a `Block` to the `ReceivedBlockInfo` (so that `numRecordsLimit` can be transferred on wire to the driver side's `ReceivedBlockTracker`);
> 3. make changes in `StreamingJobProgressListener` and related places, so that the aggregated `numRecordsLimit` information for every batch can be calculated;
> 4. make changes in `StreamingPage` and related places, so two or more lines can be drawn on a single statistics graph.
> [Screenshots]
> without back pressure:
> !https://issues.apache.org/jira/secure/attachment/12790928/1.png!
> with back pressure:
> !https://issues.apache.org/jira/secure/attachment/12790928/2.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org