You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@whirr.apache.org by Xiaomeng Wan <sh...@gmail.com> on 2012/05/21 21:42:47 UTC
whirr on rackspace
Hi,
I am trying to use whirr to start a hadoop cluster on rackspace. I
couldn't find any instructions for rackspace online, so I just
followed the quick start. When I launch the cluster, it gives the
following message saying the namenode is shutdown.
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at xxxx
************************************************************/
If I try to run the wordcount example, it gives connection refused errors.
Is there any other config I should setup in hadoop.properties for it
to work with rackspace? I am using 0.7.1.
Thanks!
Shawn
Re: whirr on rackspace
Posted by Andrei Savu <sa...@gmail.com>.
Here is the recipe I've just tried and seems to be working fine for me with
Whirr 0.7.1:
whirr.cluster-name=hadoop-asavu
whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,1
hadoop-datanode+hadoop-tasktracker
# Uncomment out these lines to run CDH
whirr.hadoop.install-function=install_cdh_hadoop
whirr.hadoop.configure-function=configure_cdh_hadoop
# For Rackspace set RACKSPACE_USERNAME and RACKSPACE_API_KEY environment
variables.
whirr.provider=cloudservers-uk
whirr.identity=${env:RACKSPACE_USERNAME}
whirr.credential=${env:RACKSPACE_API_KEY}
Please notice that the order of the roles is different in
whirr.instance-templates.
Cheers,
-- Andrei Savu / axemblr.com / Tools for Clouds
On Tue, May 22, 2012 at 10:44 PM, Xiaomeng Wan <sh...@gmail.com> wrote:
> Hi Andrei,
>
> Here is the hadoop.properties file I used:
>
> whirr.cluster-name=myhadoopcluster
> whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,1
> hadoop-datanode+hadoop-tasktracker
> whirr.provider=cloudservers-us
> whirr.identity=xxx
> whirr.credential=xxx
> whirr.private-key-file=id_rsa
> whirr.public-key-file=id_rsa.pub
> whirr.hadoop.install-function=install_cdh_hadoop
> whirr.hadoop.configure-function=configure_cdh_hadoop
>
> I run the wordcount example from namenode (ssh to it) and from local
> machine, both gave the same connection error
>
> INFO ipc.Client: Retrying connect to server:
> xxx.static.cloud-ips.com/xxx:8021. Already tried 0 time(s).
>
> when I run netstat on namenode, it shows 8020 is open, but no 8021.
>
> Shawn
>
> On Mon, May 21, 2012 at 1:50 PM, Andrei Savu <sa...@gmail.com>
> wrote:
> > That namenode message is expected after format.
> >
> > Can you share the .properties file you are using?
> >
> > How are you running the wordcount example? From the local machine?
> >
> > Important Note: there is no support for setting firewall rules on
> rackspace.
> > All ports are open.
> >
> > Cheers,
> >
> > -- Andrei Savu / axemblr.com / Tools for Clouds
> >
> >
> > On Mon, May 21, 2012 at 10:42 PM, Xiaomeng Wan <sh...@gmail.com>
> wrote:
> >>
> >> Hi,
> >> I am trying to use whirr to start a hadoop cluster on rackspace. I
> >> couldn't find any instructions for rackspace online, so I just
> >> followed the quick start. When I launch the cluster, it gives the
> >> following message saying the namenode is shutdown.
> >>
> >>
> >> /************************************************************
> >> SHUTDOWN_MSG: Shutting down NameNode at xxxx
> >> ************************************************************/
> >>
> >> If I try to run the wordcount example, it gives connection refused
> errors.
> >>
> >> Is there any other config I should setup in hadoop.properties for it
> >> to work with rackspace? I am using 0.7.1.
> >>
> >> Thanks!
> >> Shawn
> >
> >
>
Re: whirr on rackspace
Posted by Xiaomeng Wan <sh...@gmail.com>.
Hi Andrei,
Here is the hadoop.properties file I used:
whirr.cluster-name=myhadoopcluster
whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,1
hadoop-datanode+hadoop-tasktracker
whirr.provider=cloudservers-us
whirr.identity=xxx
whirr.credential=xxx
whirr.private-key-file=id_rsa
whirr.public-key-file=id_rsa.pub
whirr.hadoop.install-function=install_cdh_hadoop
whirr.hadoop.configure-function=configure_cdh_hadoop
I run the wordcount example from namenode (ssh to it) and from local
machine, both gave the same connection error
INFO ipc.Client: Retrying connect to server:
xxx.static.cloud-ips.com/xxx:8021. Already tried 0 time(s).
when I run netstat on namenode, it shows 8020 is open, but no 8021.
Shawn
On Mon, May 21, 2012 at 1:50 PM, Andrei Savu <sa...@gmail.com> wrote:
> That namenode message is expected after format.
>
> Can you share the .properties file you are using?
>
> How are you running the wordcount example? From the local machine?
>
> Important Note: there is no support for setting firewall rules on rackspace.
> All ports are open.
>
> Cheers,
>
> -- Andrei Savu / axemblr.com / Tools for Clouds
>
>
> On Mon, May 21, 2012 at 10:42 PM, Xiaomeng Wan <sh...@gmail.com> wrote:
>>
>> Hi,
>> I am trying to use whirr to start a hadoop cluster on rackspace. I
>> couldn't find any instructions for rackspace online, so I just
>> followed the quick start. When I launch the cluster, it gives the
>> following message saying the namenode is shutdown.
>>
>>
>> /************************************************************
>> SHUTDOWN_MSG: Shutting down NameNode at xxxx
>> ************************************************************/
>>
>> If I try to run the wordcount example, it gives connection refused errors.
>>
>> Is there any other config I should setup in hadoop.properties for it
>> to work with rackspace? I am using 0.7.1.
>>
>> Thanks!
>> Shawn
>
>
Re: whirr on rackspace
Posted by Andrei Savu <sa...@gmail.com>.
That namenode message is expected after format.
Can you share the .properties file you are using?
How are you running the wordcount example? From the local machine?
Important Note: there is no support for setting firewall rules on
rackspace. All ports are open.
Cheers,
-- Andrei Savu / axemblr.com / Tools for Clouds
On Mon, May 21, 2012 at 10:42 PM, Xiaomeng Wan <sh...@gmail.com> wrote:
> Hi,
> I am trying to use whirr to start a hadoop cluster on rackspace. I
> couldn't find any instructions for rackspace online, so I just
> followed the quick start. When I launch the cluster, it gives the
> following message saying the namenode is shutdown.
>
>
> /************************************************************
> SHUTDOWN_MSG: Shutting down NameNode at xxxx
> ************************************************************/
>
> If I try to run the wordcount example, it gives connection refused errors.
>
> Is there any other config I should setup in hadoop.properties for it
> to work with rackspace? I am using 0.7.1.
>
> Thanks!
> Shawn
>