You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Nicholas Chammas (JIRA)" <ji...@apache.org> on 2014/11/10 22:35:34 UTC
[jira] [Comment Edited] (SPARK-3821) Develop an automated way of
creating Spark images (AMI, Docker, and others)
[ https://issues.apache.org/jira/browse/SPARK-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14205358#comment-14205358 ]
Nicholas Chammas edited comment on SPARK-3821 at 11/10/14 9:34 PM:
-------------------------------------------------------------------
Here's the [benchmark of the launch times with the new AMIs that don't have Spark or Hadoop pre-installed | https://github.com/nchammas/spark-ec2/blob/packer/packer/proposal.md#latest-os-updates-and-ganglia-pre-installed-best-run-of-4].
Yeah, there are several optimizations to {{setup.sh}} that I can submit this week, mostly related to parallelizing things properly. I've created SPARK-4325 to track those improvements.
was (Author: nchammas):
Here's the [benchmark of the launch times with the new AMIs that don't have Spark or Hadoop pre-installed | https://github.com/nchammas/spark-ec2/blob/packer/packer/proposal.md#latest-os-updates-and-ganglia-pre-installed-best-run-of-4].
Yeah, there are several optimizations to {{setup.sh}} that I can submit this week, mostly related to parallelizing things properly. Should I submit those separately, or roll them into this AMI work?
> Develop an automated way of creating Spark images (AMI, Docker, and others)
> ---------------------------------------------------------------------------
>
> Key: SPARK-3821
> URL: https://issues.apache.org/jira/browse/SPARK-3821
> Project: Spark
> Issue Type: Improvement
> Components: Build, EC2
> Reporter: Nicholas Chammas
> Assignee: Nicholas Chammas
> Attachments: packer-proposal.html
>
>
> Right now the creation of Spark AMIs or Docker containers is done manually. With tools like [Packer|http://www.packer.io/], we should be able to automate this work, and do so in such a way that multiple types of machine images can be created from a single template.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org