You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by pwendell <gi...@git.apache.org> on 2014/02/27 20:05:54 UTC

[GitHub] spark pull request: [WIP] [SPARK-1146] Vagrant support for Spark

Github user pwendell commented on the pull request:

    https://github.com/apache/spark/pull/26#issuecomment-36278255
  
    Hey @ngbinh this is a cool feature, but it might be best as a standalone project that we can refer to on the Spark website or in a separate contrib repo (similar to https://github.com/nathanmarz/storm-contrib - an idea we've been kicking around). The only thing really similar to this is the EC2 scripts, which are actually hosted largely outside of Spark's own repo, and we are thinking of further de-coupling them from the spark codebase. 
    
    Given that the number of ways people deploy spark is proliferating, I think the project is best off pointing people towards other maintained repositories rather than embedding and supporting a bunch of different ways of doing this. For instance, a while before there was a proposed 1000-line patch to add openstack support:
    
    https://github.com/apache/incubator-spark/pull/91
    
    And offline people have talked to me about contributing patches for docker (a user facing docker thing, not just one we use for tests), google compute engine, and other deployment modes. 
    
    Merging all of these into Spark would make the project unwieldy. However, pointing people towards them would be great. So if you are interested in maintaining this as its own repo, we could probably codify that and point users towards it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---