You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Shixiong Zhu (JIRA)" <ji...@apache.org> on 2016/02/17 04:45:18 UTC
[jira] [Updated] (SPARK-11627) Spark Streaming backpressure
mechanism has no initial input rate limit,receivers receive data at the
maximum speed , it might cause OOM exception
[ https://issues.apache.org/jira/browse/SPARK-11627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shixiong Zhu updated SPARK-11627:
---------------------------------
Affects Version/s: 1.6.0
> Spark Streaming backpressure mechanism has no initial input rate limit,receivers receive data at the maximum speed , it might cause OOM exception
> --------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-11627
> URL: https://issues.apache.org/jira/browse/SPARK-11627
> Project: Spark
> Issue Type: Bug
> Components: Streaming
> Affects Versions: 1.5.1, 1.6.0
> Reporter: junhaoMg
> Assignee: junhaoMg
> Fix For: 2.0.0
>
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> Spark Streaming backpressure mechanism has no initial input rate limit, receivers receive data at the maximum speed they can reach in the first batch, the data received might exhaust executors memory resources and cause out of memory exception. Eventually the streaming job would failed, the backpressure mechanism become invalid.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org