You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Florian Verhein (JIRA)" <ji...@apache.org> on 2015/02/13 04:38:12 UTC

[jira] [Updated] (SPARK-5641) Allow spark_ec2.py to copy arbitrary files to cluster

     [ https://issues.apache.org/jira/browse/SPARK-5641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Florian Verhein updated SPARK-5641:
-----------------------------------
    Summary: Allow spark_ec2.py to copy arbitrary files to cluster  (was: Allow spark_ec2.py to copy arbitrary files to cluster via deploy.generic)

> Allow spark_ec2.py to copy arbitrary files to cluster
> -----------------------------------------------------
>
>                 Key: SPARK-5641
>                 URL: https://issues.apache.org/jira/browse/SPARK-5641
>             Project: Spark
>          Issue Type: Improvement
>          Components: EC2
>            Reporter: Florian Verhein
>            Priority: Minor
>
> Useful if binary files need to be uploaded. E.g. I use this for rpm transfer to install extra stuff at cluster deployment time.
> However note that it could also be used to override either:
> - what's on the image
> - what gets cloned from spark-ec2 (since deploy_files runs afterwards)
> The idea is that the user can just dump the files into ec2/deploy.generic/. 
> This can be implemented by modifying deploy_files so that it simply copies the file (if it is of certain types), rather than treating it as a text file and attempting to replace template variables.
> Detecting binary files is non-trivial. So the proposal is to have a list of file extensions that will trigger simple file copying.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org