You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Saisai Shao (JIRA)" <ji...@apache.org> on 2015/01/23 03:12:34 UTC
[jira] [Created] (SPARK-5372) Change the default storage level of
window operators
Saisai Shao created SPARK-5372:
----------------------------------
Summary: Change the default storage level of window operators
Key: SPARK-5372
URL: https://issues.apache.org/jira/browse/SPARK-5372
Project: Spark
Issue Type: Bug
Components: Streaming
Affects Versions: 1.2.0
Reporter: Saisai Shao
Current storage level of window operators is MEMORY_ONLY_SER, if the memory is not enough to hold all the window data, cached RDD will be discarded, which will lead to unexpected behavior.
Besides the default storage level of input data is MEMORY_AND_DISK_SER_2, it is better to align to this storage level to change the storage level of window operators to MEMORY_AND_DISK_SER.
This changing has no effect when memory is enough. So I'd propose to change the default storage level to MEMORY_AND_DISK_SER.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org