You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@whirr.apache.org by "Periya.Data" <pe...@gmail.com> on 2011/12/04 20:16:23 UTC
DEPRECATED: hadoop-site.xml
Hi,
I just began using Whirr to deploy clusters on EC2. I am following the
instructions mentioned in Whirr documentation/quick start guide. I am able
to start clusters and destroy from my laptop. However, I see an issue when
I try to connect to a machine on EC2.
I am using Whirr 0.5, hadoop-0.20.2-cdh3u2 on my Ubuntu 11.10 laptop.
The instances I created on EC2 are Ubuntu 11.10 Oneiric with
hadoop-0.20.2-cdh3u2.
Here is what I am doing:
(on teminal -1)
$ *. ~/.whirr/HadoopCluster/hadoop-proxy.sh *
Running proxy to Hadoop cluster at ec2-75-101-193-84.compute-1.amazonaws.com.
Use Ctrl-c to quit.
Warning: Permanently added
'ec2-75-101-193-84.compute-1.amazonaws.com,75.101.193.84'
(ECDSA) to the list of known hosts.
channel 2: open failed: connect failed: Connection refused
channel 2: open failed: connect failed: Connection refused
(on a different terminal , terminal - 2)
$ *export HADOOP_CONF_DIR=~/.whirr/HadoopCluster/*
$
$
$ *hadoop fs -ls /*
11/12/03 21:49:30 WARN conf.Configuration: DEPRECATED: hadoop-site.xml
found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use
core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of
core-default.xml, mapred-default.xml and hdfs-default.xml respectively
11/12/03 21:49:32 INFO ipc.Client: Retrying connect to server:
ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020. Already tried
0 time(s).
11/12/03 21:49:33 INFO ipc.Client: Retrying connect to server:
ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020. Already tried
1 time(s).
11/12/03 21:49:35 INFO ipc.Client: Retrying connect to server:
ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020. Already tried
2 time(s).
========================
I am executing all the above commands from my local laptop and nothing is
done on the remote instance.
Your suggestions would be helpful.
Thanks,
PD.
Re: DEPRECATED: hadoop-site.xml
Posted by Andrei Savu <sa...@gmail.com>.
On Sun, Dec 4, 2011 at 9:47 PM, Periya.Data <pe...@gmail.com> wrote:
> Ok. Thanks Stephen. But, any idea why I am unable to run hadoop commands?
> I believe my next post...(regarding lack of JAVA and Hadoop on Ubuntu 11.10
> AMI) may be the reason...
You are right. See my advice about using Ubuntu 10.04 LTS on the remote
machines, at least until we fix:
https://issues.apache.org/jira/browse/WHIRR-435
-- Andrei Savu / andreisavu.ro
Re: DEPRECATED: hadoop-site.xml
Posted by Paul Baclace <pa...@gmail.com>.
hadoop-site.xml has been deprecated for a few years now. I'm not sure
if the benefits of splitting it up paid off, but it does mean one must
determine which property goes into which -site.xml file. What I do when
I install Hadoop is:
mkdir -p $HADOOP_HOME/conf/defaults
cd $HADOOP_HOME/conf/defaults
jar xvf ../../hadoop-*core.jar core-default.xml hdfs-default.xml
mapred-default.xml
so that the documented properties are easily at hand.
Paul
On 20111204 11:49 , Stephen Boesch wrote:
> OK I'm using 11.10 (full install not AMI but should not matter i
> think). I think I've used just hadoop-site.xml and it still works,
> will double check.
>
> 2011/12/4 Periya.Data <periya.data@gmail.com
> <ma...@gmail.com>>
>
> Ok. Thanks Stephen. But, any idea why I am unable to run hadoop
> commands? I believe my next post...(regarding lack of JAVA and
> Hadoop on Ubuntu 11.10 AMI) may be the reason...
>
> PD
>
>
> On Sun, Dec 4, 2011 at 11:42 AM, Stephen Boesch <javadba@gmail.com
> <ma...@gmail.com>> wrote:
>
> it sill works but is deprecated in favor of core-site.xml,
> hdfs-site.xml, and mapred-site.xml
>
> 2011/12/4 Periya.Data <periya.data@gmail.com
> <ma...@gmail.com>>
>
> Hi,
> I just began using Whirr to deploy clusters on EC2. I
> am following the instructions mentioned in Whirr
> documentation/quick start guide. I am able to start
> clusters and destroy from my laptop. However, I see an
> issue when I try to connect to a machine on EC2.
>
> I am using Whirr 0.5, hadoop-0.20.2-cdh3u2 on my Ubuntu
> 11.10 laptop.
> The instances I created on EC2 are Ubuntu 11.10 Oneiric
> with hadoop-0.20.2-cdh3u2.
>
>
> Here is what I am doing:
>
> (on teminal -1)
> $ *. ~/.whirr/HadoopCluster/hadoop-proxy.sh *
> Running proxy to Hadoop cluster at
> ec2-75-101-193-84.compute-1.amazonaws.com
> <http://ec2-75-101-193-84.compute-1.amazonaws.com>. Use
> Ctrl-c to quit.
> Warning: Permanently added
> 'ec2-75-101-193-84.compute-1.amazonaws.com
> <http://ec2-75-101-193-84.compute-1.amazonaws.com>,75.101.193.84'
> (ECDSA) to the list of known hosts.
> channel 2: open failed: connect failed: Connection refused
> channel 2: open failed: connect failed: Connection refused
>
> (on a different terminal , terminal - 2)
> $ *export HADOOP_CONF_DIR=~/.whirr/HadoopCluster/*
> $
> $
> $ *hadoop fs -ls /*
> 11/12/03 21:49:30 WARN conf.Configuration: DEPRECATED:
> hadoop-site.xml found in the classpath. Usage of
> hadoop-site.xml is deprecated. Instead use core-site.xml,
> mapred-site.xml and hdfs-site.xml to override properties
> of core-default.xml, mapred-default.xml and
> hdfs-default.xml respectively
> 11/12/03 21:49:32 INFO ipc.Client: Retrying connect to
> server:
> ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020
> <http://ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020>.
> Already tried 0 time(s).
> 11/12/03 21:49:33 INFO ipc.Client: Retrying connect to
> server:
> ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020
> <http://ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020>.
> Already tried 1 time(s).
> 11/12/03 21:49:35 INFO ipc.Client: Retrying connect to
> server:
> ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020
> <http://ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020>.
> Already tried 2 time(s).
> ========================
>
> I am executing all the above commands from my local laptop
> and nothing is done on the remote instance.
>
> Your suggestions would be helpful.
>
> Thanks,
> PD.
>
>
>
>
>
>
>
>
Re: DEPRECATED: hadoop-site.xml
Posted by Stephen Boesch <ja...@gmail.com>.
OK I'm using 11.10 (full install not AMI but should not matter i think).
I think I've used just hadoop-site.xml and it still works, will double
check.
2011/12/4 Periya.Data <pe...@gmail.com>
> Ok. Thanks Stephen. But, any idea why I am unable to run hadoop commands?
> I believe my next post...(regarding lack of JAVA and Hadoop on Ubuntu 11.10
> AMI) may be the reason...
>
> PD
>
>
> On Sun, Dec 4, 2011 at 11:42 AM, Stephen Boesch <ja...@gmail.com> wrote:
>
>> it sill works but is deprecated in favor of core-site.xml, hdfs-site.xml,
>> and mapred-site.xml
>>
>> 2011/12/4 Periya.Data <pe...@gmail.com>
>>
>> Hi,
>>> I just began using Whirr to deploy clusters on EC2. I am following
>>> the instructions mentioned in Whirr documentation/quick start guide. I am
>>> able to start clusters and destroy from my laptop. However, I see an issue
>>> when I try to connect to a machine on EC2.
>>>
>>> I am using Whirr 0.5, hadoop-0.20.2-cdh3u2 on my Ubuntu 11.10 laptop.
>>> The instances I created on EC2 are Ubuntu 11.10 Oneiric with
>>> hadoop-0.20.2-cdh3u2.
>>>
>>>
>>> Here is what I am doing:
>>>
>>> (on teminal -1)
>>> $ *. ~/.whirr/HadoopCluster/hadoop-proxy.sh *
>>> Running proxy to Hadoop cluster at
>>> ec2-75-101-193-84.compute-1.amazonaws.com. Use Ctrl-c to quit.
>>> Warning: Permanently added 'ec2-75-101-193-84.compute-1.amazonaws.com,75.101.193.84'
>>> (ECDSA) to the list of known hosts.
>>> channel 2: open failed: connect failed: Connection refused
>>> channel 2: open failed: connect failed: Connection refused
>>>
>>> (on a different terminal , terminal - 2)
>>> $ *export HADOOP_CONF_DIR=~/.whirr/HadoopCluster/*
>>> $
>>> $
>>> $ *hadoop fs -ls /*
>>> 11/12/03 21:49:30 WARN conf.Configuration: DEPRECATED: hadoop-site.xml
>>> found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use
>>> core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of
>>> core-default.xml, mapred-default.xml and hdfs-default.xml respectively
>>> 11/12/03 21:49:32 INFO ipc.Client: Retrying connect to server:
>>> ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020. Already
>>> tried 0 time(s).
>>> 11/12/03 21:49:33 INFO ipc.Client: Retrying connect to server:
>>> ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020. Already
>>> tried 1 time(s).
>>> 11/12/03 21:49:35 INFO ipc.Client: Retrying connect to server:
>>> ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020. Already
>>> tried 2 time(s).
>>> ========================
>>>
>>> I am executing all the above commands from my local laptop and nothing
>>> is done on the remote instance.
>>>
>>> Your suggestions would be helpful.
>>>
>>> Thanks,
>>> PD.
>>>
>>>
>>>
>>>
>>>
>>>
>>
>
Re: DEPRECATED: hadoop-site.xml
Posted by "Periya.Data" <pe...@gmail.com>.
Ok. Thanks Stephen. But, any idea why I am unable to run hadoop commands? I
believe my next post...(regarding lack of JAVA and Hadoop on Ubuntu 11.10
AMI) may be the reason...
PD
On Sun, Dec 4, 2011 at 11:42 AM, Stephen Boesch <ja...@gmail.com> wrote:
> it sill works but is deprecated in favor of core-site.xml, hdfs-site.xml,
> and mapred-site.xml
>
> 2011/12/4 Periya.Data <pe...@gmail.com>
>
> Hi,
>> I just began using Whirr to deploy clusters on EC2. I am following
>> the instructions mentioned in Whirr documentation/quick start guide. I am
>> able to start clusters and destroy from my laptop. However, I see an issue
>> when I try to connect to a machine on EC2.
>>
>> I am using Whirr 0.5, hadoop-0.20.2-cdh3u2 on my Ubuntu 11.10 laptop.
>> The instances I created on EC2 are Ubuntu 11.10 Oneiric with
>> hadoop-0.20.2-cdh3u2.
>>
>>
>> Here is what I am doing:
>>
>> (on teminal -1)
>> $ *. ~/.whirr/HadoopCluster/hadoop-proxy.sh *
>> Running proxy to Hadoop cluster at
>> ec2-75-101-193-84.compute-1.amazonaws.com. Use Ctrl-c to quit.
>> Warning: Permanently added 'ec2-75-101-193-84.compute-1.amazonaws.com,75.101.193.84'
>> (ECDSA) to the list of known hosts.
>> channel 2: open failed: connect failed: Connection refused
>> channel 2: open failed: connect failed: Connection refused
>>
>> (on a different terminal , terminal - 2)
>> $ *export HADOOP_CONF_DIR=~/.whirr/HadoopCluster/*
>> $
>> $
>> $ *hadoop fs -ls /*
>> 11/12/03 21:49:30 WARN conf.Configuration: DEPRECATED: hadoop-site.xml
>> found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use
>> core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of
>> core-default.xml, mapred-default.xml and hdfs-default.xml respectively
>> 11/12/03 21:49:32 INFO ipc.Client: Retrying connect to server:
>> ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020. Already
>> tried 0 time(s).
>> 11/12/03 21:49:33 INFO ipc.Client: Retrying connect to server:
>> ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020. Already
>> tried 1 time(s).
>> 11/12/03 21:49:35 INFO ipc.Client: Retrying connect to server:
>> ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020. Already
>> tried 2 time(s).
>> ========================
>>
>> I am executing all the above commands from my local laptop and nothing is
>> done on the remote instance.
>>
>> Your suggestions would be helpful.
>>
>> Thanks,
>> PD.
>>
>>
>>
>>
>>
>>
>
Re: DEPRECATED: hadoop-site.xml
Posted by Stephen Boesch <ja...@gmail.com>.
it sill works but is deprecated in favor of core-site.xml, hdfs-site.xml,
and mapred-site.xml
2011/12/4 Periya.Data <pe...@gmail.com>
> Hi,
> I just began using Whirr to deploy clusters on EC2. I am following the
> instructions mentioned in Whirr documentation/quick start guide. I am able
> to start clusters and destroy from my laptop. However, I see an issue when
> I try to connect to a machine on EC2.
>
> I am using Whirr 0.5, hadoop-0.20.2-cdh3u2 on my Ubuntu 11.10 laptop.
> The instances I created on EC2 are Ubuntu 11.10 Oneiric with
> hadoop-0.20.2-cdh3u2.
>
>
> Here is what I am doing:
>
> (on teminal -1)
> $ *. ~/.whirr/HadoopCluster/hadoop-proxy.sh *
> Running proxy to Hadoop cluster at
> ec2-75-101-193-84.compute-1.amazonaws.com. Use Ctrl-c to quit.
> Warning: Permanently added 'ec2-75-101-193-84.compute-1.amazonaws.com,75.101.193.84'
> (ECDSA) to the list of known hosts.
> channel 2: open failed: connect failed: Connection refused
> channel 2: open failed: connect failed: Connection refused
>
> (on a different terminal , terminal - 2)
> $ *export HADOOP_CONF_DIR=~/.whirr/HadoopCluster/*
> $
> $
> $ *hadoop fs -ls /*
> 11/12/03 21:49:30 WARN conf.Configuration: DEPRECATED: hadoop-site.xml
> found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use
> core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of
> core-default.xml, mapred-default.xml and hdfs-default.xml respectively
> 11/12/03 21:49:32 INFO ipc.Client: Retrying connect to server:
> ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020. Already
> tried 0 time(s).
> 11/12/03 21:49:33 INFO ipc.Client: Retrying connect to server:
> ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020. Already
> tried 1 time(s).
> 11/12/03 21:49:35 INFO ipc.Client: Retrying connect to server:
> ec2-75-101-193-84.compute-1.amazonaws.com/75.101.193.84:8020. Already
> tried 2 time(s).
> ========================
>
> I am executing all the above commands from my local laptop and nothing is
> done on the remote instance.
>
> Your suggestions would be helpful.
>
> Thanks,
> PD.
>
>
>
>
>
>