Trouble with spark-ec2 script: --ebs-vol-size

[Ben Horner <be...@atigeo.com>]

Hello,

I'm using the spark-0.9.1-bin-hadoop1 distribution, and the ec2/spark-ec2
script within it to spin up a cluster.  I tried running my processing just
using the default (ephemeral) HDFS configuration, but my job errored out,
saying that there was no space left.  So now I'm trying to increase the size
of HDFS on the cluster.

My launch command:
ec2/spark-ec2 -k ... -i ... -z us-east-1d -s 4 -t m3.2xlarge
--ebs-vol-size=250 -m r3.2xlarge launch ...

My understanding is that I should get a cluster, where each slave node has
an ebs backed drive with 250 GB of storage, with a persistent HDFS set to
use these slave drives.

I turn off the ephemeral HDFS on the cluster master:
ephemeral-hdfs/bin/stop-all.sh

Then I turn on the persistent HDFS on the cluster master:
persistent-hdfs/bin/start-all.sh

Once I discovered the proper URL to hit the persistent name node page (not
the ephemeral standard 50070 port):
http://<master>:60070/dfshealth.jsp

The page shows 4 nodes as expected, but the configured capacity shows as
31.5 GB, not the expected 1 TB (250 GB x 4)


Please help!
Don't be shy to let me know if I've made mis-steps, or if I'm not
understanding things correctly!
Thanks.




--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Trouble-with-spark-ec2-script-ebs-vol-size-tp9619.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Trouble with spark-ec2 script: --ebs-vol-size

[Ben Horner <be...@atigeo.com>]

I've just tried it again, with the same results.

When you say "it worked" what does the NameNode page list as your
"Configured Capacity"?
For me, (apparently) regardless of what I specify in the --ebs-vol-size
parameter, I get a persistent HDFS capacity of 31.5 GB.

I also used df -h to see what drives were mounted, and I get the same drives
regardless of whether I specify the --ebs-vol-size parameter when I execute
the script...

Can you give me an exact command line that works for you?  I will try and
execute it verbatim.

Thanks,
-Ben




--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Trouble-with-spark-ec2-script-ebs-vol-size-tp9619p10208.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Trouble with spark-ec2 script: --ebs-vol-size

[Chris DuBois <ch...@gmail.com>]

Hi Ben,

It worked for me, but only when using the default region. Using
--region=us-west-2 resulted in errors about security groups.

Chris


On Wed, Jul 16, 2014 at 8:53 AM, Ben Horner <be...@atigeo.com> wrote:

> please add
>
>   From: "Ben Horner [via Apache Spark User List]" <[hidden email]
> <http://user/SendEmail.jtp?type=node&node=9935&i=0>>
> Date: Wednesday, July 16, 2014 at 8:47 AM
> To: Ben Horner <[hidden email]
> <http://user/SendEmail.jtp?type=node&node=9935&i=1>>
> Subject: Re: Trouble with spark-ec2 script: --ebs-vol-size
>
>  Should I take it from the lack of replies that the --ebs-vol-size
> feature doesn't work?
> -Ben
>
>
> ------------------------------
>  If you reply to this email, your message will be added to the discussion
> below:
>
> http://apache-spark-user-list.1001560.n3.nabble.com/Trouble-with-spark-ec2-script-ebs-vol-size-tp9619p9934.html
>  To unsubscribe from Trouble with spark-ec2 script: --ebs-vol-size, click
> here.
> NAML
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
> ------------------------------
> View this message in context: Re: Trouble with spark-ec2 script:
> --ebs-vol-size
> <http://apache-spark-user-list.1001560.n3.nabble.com/Trouble-with-spark-ec2-script-ebs-vol-size-tp9619p9935.html>
>
> Sent from the Apache Spark User List mailing list archive
> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com.
>

Re: Trouble with spark-ec2 script: --ebs-vol-size

[Ben Horner <be...@atigeo.com>]

please add

From: "Ben Horner [via Apache Spark User List]" <ml...@n3.nabble.com>>
Date: Wednesday, July 16, 2014 at 8:47 AM
To: Ben Horner <be...@atigeo.com>>
Subject: Re: Trouble with spark-ec2 script: --ebs-vol-size

Should I take it from the lack of replies that the --ebs-vol-size feature doesn't work?
-Ben


________________________________
If you reply to this email, your message will be added to the discussion below:
http://apache-spark-user-list.1001560.n3.nabble.com/Trouble-with-spark-ec2-script-ebs-vol-size-tp9619p9934.html
To unsubscribe from Trouble with spark-ec2 script: --ebs-vol-size, click here<http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=9619&code=YmVuLmhvcm5lckBhdGlnZW8uY29tfDk2MTl8MTk2OTU5NjEyOQ==>.
NAML<http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>




--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Trouble-with-spark-ec2-script-ebs-vol-size-tp9619p9935.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Trouble with spark-ec2 script: --ebs-vol-size

[Ben Horner <be...@atigeo.com>]

Should I take it from the lack of replies that the --ebs-vol-size feature
doesn't work?
-Ben




--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Trouble-with-spark-ec2-script-ebs-vol-size-tp9619p9934.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.