[jira] [Comment Edited] (SPARK-5535) Add parameter for storage levels

["Joseph K. Bradley (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-5535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15796294#comment-15796294 ] 

Joseph K. Bradley edited comment on SPARK-5535 at 1/3/17 10:02 PM:
-------------------------------------------------------------------

This issue came up in [SPARK-19007], so I'd like to pick it up again.  I'd like people's thoughts on two questions:
* Should we add a special parameter value which matches in the input DataFrame's storage level?
* What should the default value be?
** Maintain current behavior: MEMORY_ONLY
** (Probably) optimize for most use cases: Match input storage level



was (Author: josephkb):
This issue came up in [SPARK-19007], so I'd like to pick it up again.  I'd like people's thoughts on one question:
Should we add a special parameter value which matches in the input DataFrame's storage level?  It might be reasonable to use that as the default.

> Add parameter for storage levels
> --------------------------------
>
>                 Key: SPARK-5535
>                 URL: https://issues.apache.org/jira/browse/SPARK-5535
>             Project: Spark
>          Issue Type: New Feature
>          Components: ML
>            Reporter: Xiangrui Meng
>
> Add a special parameter type for storage levels that takes the string representation of StorageLevels.
> This value can be used when ML algorithms persist data internally.
> Specifically, add a new {{Param[String]}} which takes the storage level.  This can go in sharedParams.  It should be added to algorithms individually in subtasks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org