You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by "Gangl, Michael E (388K)" <Mi...@jpl.nasa.gov> on 2010/11/17 19:41:55 UTC

AWS Hadoop 20.2 AMIs

I've been running into an issue today.

I'm trying to procure 5 c1.xlarge instances on Amazon EC2. I was able to use the 453820947548/bixolabs-hadoop-0.20.2-i386 AMI for my previous m1.large instances, so I figured I could use the c1.xlarge instances with the x86_64 versions.

When I start these with the src/contrib/ec2/bin scripts, however, the master starts but then I'm unable to connect to the xlarge instances. I can still use the 1.large instances, but these are too slow for me, so I'd like to use the bigger machine. Has anyone else been having problems today, or in the past with getting an AMI to work on the xlarge instances?

Thanks,

Mike

Re: AWS Hadoop 20.2 AMIs

Posted by Chris Schneider <Sc...@TransPac.com>.
Hi Mike,

At 10:41 AM -0800 11/17/10, Gangl, Michael E (388K) wrote:
>I've been running into an issue today.
>
>I'm trying to procure 5 c1.xlarge instances on Amazon EC2. I was able to use the 453820947548/bixolabs-hadoop-0.20.2-i386 AMI for my previous m1.large instances, so I figured I could use the c1.xlarge instances with the x86_64 versions.

Note that the 453820947548/bixolabs-hadoop-0.20.2-i386 AMI (ami-48b54021) was designed for m1.small and c1.medium instances. We put together a different public AMI (ami-827c8beb) for m1.large instances.

>When I start these with the src/contrib/ec2/bin scripts, however, the master starts but then I'm unable to connect to the xlarge instances. I can still use the 1.large instances, but these are too slow for me, so I'd like to use the bigger machine. Has anyone else been having problems today, or in the past with getting an AMI to work on the xlarge instances?

At 11:26 AM -0800 11/17/10, Gangl, Michael E (388K) wrote:
>FYI, I commented out the Kernal version in the hadoop-ec2-env.sh script for the c1.xlarge if statements (at the bottom).
>
>Before it was using aki-427d952b
>
>Now it's using aki-b51cf9dc
>
>And I'm able to connect. Turns out the problem was a hang during the boot. This should probably be changed in the future releases of the ec2 scripts (if it's not changed already :) )

The Hadoop 0.20.2's /src/contrib/ec2/bin/hadoop-ec2-env.sh.template actually specifies $AMI_KERNEL=aki-800e5f2 for c1.xlarge:

http://svn.apache.org/repos/asf/hadoop/common/tags/release-0.20.2/src/contrib/ec2/bin/hadoop-ec2-env.sh.template

I haven't tried launching any c1.xlarge instances myself, though.

FYI,

- Chris
-- 
------------------------
Chris Schneider
TransPac Software, Inc.
Schmed@TransPac.com
------------------------

Re: AWS Hadoop 20.2 AMIs

Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hey Mike,

Do you have time to submit a patch? You could probably create a jira issue here [1] and then attach a diff of your update...

Cheers,
Chris

[1] http://issues.apache.org/jira/browse/HADOOP

On Nov 17, 2010, at 11:26 AM, Gangl, Michael E (388K) wrote:

> FYI, I commented out the Kernal version in the hadoop-ec2-env.sh script for the c1.xlarge if statements (at the bottom).
> 
> Before it was using aki-427d952b
> 
> Now it's using aki-b51cf9dc
> 
> And I'm able to connect. Turns out the problem was a hang during the boot. This should probably be changed in the future releases of the ec2 scripts (if it's not changed already :) )
> 
> -Mike
> 
> 
> On 11/17/10 10:41 AM, "Michael Gangl" <Mi...@jpl.nasa.gov> wrote:
> 
> I've been running into an issue today.
> 
> I'm trying to procure 5 c1.xlarge instances on Amazon EC2. I was able to use the 453820947548/bixolabs-hadoop-0.20.2-i386 AMI for my previous m1.large instances, so I figured I could use the c1.xlarge instances with the x86_64 versions.
> 
> When I start these with the src/contrib/ec2/bin scripts, however, the master starts but then I'm unable to connect to the xlarge instances. I can still use the 1.large instances, but these are too slow for me, so I'd like to use the bigger machine. Has anyone else been having problems today, or in the past with getting an AMI to work on the xlarge instances?
> 
> Thanks,
> 
> Mike
> 


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: Chris.Mattmann@jpl.nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Re: AWS Hadoop 20.2 AMIs

Posted by "Gangl, Michael E (388K)" <Mi...@jpl.nasa.gov>.
FYI, I commented out the Kernal version in the hadoop-ec2-env.sh script for the c1.xlarge if statements (at the bottom).

Before it was using aki-427d952b

Now it's using aki-b51cf9dc

And I'm able to connect. Turns out the problem was a hang during the boot. This should probably be changed in the future releases of the ec2 scripts (if it's not changed already :) )

-Mike


On 11/17/10 10:41 AM, "Michael Gangl" <Mi...@jpl.nasa.gov> wrote:

I've been running into an issue today.

I'm trying to procure 5 c1.xlarge instances on Amazon EC2. I was able to use the 453820947548/bixolabs-hadoop-0.20.2-i386 AMI for my previous m1.large instances, so I figured I could use the c1.xlarge instances with the x86_64 versions.

When I start these with the src/contrib/ec2/bin scripts, however, the master starts but then I'm unable to connect to the xlarge instances. I can still use the 1.large instances, but these are too slow for me, so I'd like to use the bigger machine. Has anyone else been having problems today, or in the past with getting an AMI to work on the xlarge instances?

Thanks,

Mike