You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Han JU <ju...@gmail.com> on 2014/05/12 15:16:30 UTC
[EC2] r3 instance type
Hi,
I'm modifying the ec2 script for the new r3 instance support, but there's a
problem with the instance storage.
For example, `r3.large` has a single 32GB SSD disk, the problem is that
it's a SSD with TRIM technology and is not automatically formatted and
mounted, `lsblk` gives me this after ec2_script's setup:
xvda 202:0 0 8G 0 disk
└─xvda1 202:1 0 8G 0 part /
xvdb 202:16 0 30G 0 disk
I think there's some workarounds of this problem, for example we could
treat it like an EBS device and check `/dev/xvdb` by using `blkid`, howver
this needs modifying the deployment script inside the AMI and I don't know
if it's the preferred way .
Some ideas or suggestions?
Thanks.
--
*JU Han*
Data Engineer @ Botify.com
+33 0619608888
Re: [EC2] r3 instance type
Posted by Shivaram Venkataraman <sh...@eecs.berkeley.edu>.
I ran into this a couple of days back as well. Yes, we need to check if
/dev/xvdb is formatted and if not create xfs or some such filesystem on it.
We will need to change the deployment script and you can do that (similar
to EBS volumes) at https://github.com/mesos/spark-ec2/blob/v2/setup-slave.sh
Thanks
Shivaram
On Mon, May 12, 2014 at 6:16 AM, Han JU <ju...@gmail.com> wrote:
> Hi,
>
> I'm modifying the ec2 script for the new r3 instance support, but there's a
> problem with the instance storage.
>
> For example, `r3.large` has a single 32GB SSD disk, the problem is that
> it's a SSD with TRIM technology and is not automatically formatted and
> mounted, `lsblk` gives me this after ec2_script's setup:
>
> xvda 202:0 0 8G 0 disk
> └─xvda1 202:1 0 8G 0 part /
> xvdb 202:16 0 30G 0 disk
>
> I think there's some workarounds of this problem, for example we could
> treat it like an EBS device and check `/dev/xvdb` by using `blkid`, howver
> this needs modifying the deployment script inside the AMI and I don't know
> if it's the preferred way .
>
> Some ideas or suggestions?
>
> Thanks.
> --
> *JU Han*
>
> Data Engineer @ Botify.com
>
> +33 0619608888
>