You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Pedro Sá da Costa <ps...@gmail.com> on 2013/02/13 12:46:45 UTC
configure mapreduce to work with pem files.
I'm trying to configure ssh for the Hadoop mapreduce, but my nodes only
communicate with each others using RSA keys in pem format.
(It doesn't work)
ssh user@host
Permission denied (publickey).
(It works)
ssh -i ~/key.pem user@host
The nodes in mapreduce communicate using ssh. How I configure the ssh, or
the mapreduce to work with the pem file.
--
Best regards,
P
Re: configure mapreduce to work with pem files.
Posted by Nitin Pawar <ni...@gmail.com>.
Pedro,
Hadoop does not need ssh in MR.
hadoop start up script needs ssh setup so that you can start all your
services from a single node. And that is not mandatory. You can login to
all nodes and start individual services on the nodes as needed by the node
with the correct configuration for hadoop
On Wed, Feb 13, 2013 at 7:15 PM, Pedro Sá da Costa <ps...@gmail.com>wrote:
> So, why it is necessary to configure ssh in Hadoop MR?
>
>
> On 13 February 2013 12:58, Harsh J <ha...@cloudera.com> wrote:
>
>> Hi,
>>
>> Nodes in Hadoop do not communicate using SSH. See
>> http://wiki.apache.org/hadoop/FAQ#Does_Hadoop_require_SSH.3F
>>
>> On Wed, Feb 13, 2013 at 5:16 PM, Pedro Sá da Costa <ps...@gmail.com>
>> wrote:
>> > I'm trying to configure ssh for the Hadoop mapreduce, but my nodes only
>> > communicate with each others using RSA keys in pem format.
>> >
>> > (It doesn't work)
>> > ssh user@host
>> > Permission denied (publickey).
>> >
>> > (It works)
>> > ssh -i ~/key.pem user@host
>> >
>> > The nodes in mapreduce communicate using ssh. How I configure the ssh,
>> or
>> > the mapreduce to work with the pem file.
>> >
>> >
>> > --
>> > Best regards,
>> > P
>>
>>
>>
>> --
>> Harsh J
>>
>
>
>
> --
> Best regards,
>
--
Nitin Pawar
Re: configure mapreduce to work with pem files.
Posted by Nitin Pawar <ni...@gmail.com>.
Pedro,
Hadoop does not need ssh in MR.
hadoop start up script needs ssh setup so that you can start all your
services from a single node. And that is not mandatory. You can login to
all nodes and start individual services on the nodes as needed by the node
with the correct configuration for hadoop
On Wed, Feb 13, 2013 at 7:15 PM, Pedro Sá da Costa <ps...@gmail.com>wrote:
> So, why it is necessary to configure ssh in Hadoop MR?
>
>
> On 13 February 2013 12:58, Harsh J <ha...@cloudera.com> wrote:
>
>> Hi,
>>
>> Nodes in Hadoop do not communicate using SSH. See
>> http://wiki.apache.org/hadoop/FAQ#Does_Hadoop_require_SSH.3F
>>
>> On Wed, Feb 13, 2013 at 5:16 PM, Pedro Sá da Costa <ps...@gmail.com>
>> wrote:
>> > I'm trying to configure ssh for the Hadoop mapreduce, but my nodes only
>> > communicate with each others using RSA keys in pem format.
>> >
>> > (It doesn't work)
>> > ssh user@host
>> > Permission denied (publickey).
>> >
>> > (It works)
>> > ssh -i ~/key.pem user@host
>> >
>> > The nodes in mapreduce communicate using ssh. How I configure the ssh,
>> or
>> > the mapreduce to work with the pem file.
>> >
>> >
>> > --
>> > Best regards,
>> > P
>>
>>
>>
>> --
>> Harsh J
>>
>
>
>
> --
> Best regards,
>
--
Nitin Pawar
Re: configure mapreduce to work with pem files.
Posted by Nitin Pawar <ni...@gmail.com>.
Pedro,
Hadoop does not need ssh in MR.
hadoop start up script needs ssh setup so that you can start all your
services from a single node. And that is not mandatory. You can login to
all nodes and start individual services on the nodes as needed by the node
with the correct configuration for hadoop
On Wed, Feb 13, 2013 at 7:15 PM, Pedro Sá da Costa <ps...@gmail.com>wrote:
> So, why it is necessary to configure ssh in Hadoop MR?
>
>
> On 13 February 2013 12:58, Harsh J <ha...@cloudera.com> wrote:
>
>> Hi,
>>
>> Nodes in Hadoop do not communicate using SSH. See
>> http://wiki.apache.org/hadoop/FAQ#Does_Hadoop_require_SSH.3F
>>
>> On Wed, Feb 13, 2013 at 5:16 PM, Pedro Sá da Costa <ps...@gmail.com>
>> wrote:
>> > I'm trying to configure ssh for the Hadoop mapreduce, but my nodes only
>> > communicate with each others using RSA keys in pem format.
>> >
>> > (It doesn't work)
>> > ssh user@host
>> > Permission denied (publickey).
>> >
>> > (It works)
>> > ssh -i ~/key.pem user@host
>> >
>> > The nodes in mapreduce communicate using ssh. How I configure the ssh,
>> or
>> > the mapreduce to work with the pem file.
>> >
>> >
>> > --
>> > Best regards,
>> > P
>>
>>
>>
>> --
>> Harsh J
>>
>
>
>
> --
> Best regards,
>
--
Nitin Pawar
Re: configure mapreduce to work with pem files.
Posted by Nitin Pawar <ni...@gmail.com>.
Pedro,
Hadoop does not need ssh in MR.
hadoop start up script needs ssh setup so that you can start all your
services from a single node. And that is not mandatory. You can login to
all nodes and start individual services on the nodes as needed by the node
with the correct configuration for hadoop
On Wed, Feb 13, 2013 at 7:15 PM, Pedro Sá da Costa <ps...@gmail.com>wrote:
> So, why it is necessary to configure ssh in Hadoop MR?
>
>
> On 13 February 2013 12:58, Harsh J <ha...@cloudera.com> wrote:
>
>> Hi,
>>
>> Nodes in Hadoop do not communicate using SSH. See
>> http://wiki.apache.org/hadoop/FAQ#Does_Hadoop_require_SSH.3F
>>
>> On Wed, Feb 13, 2013 at 5:16 PM, Pedro Sá da Costa <ps...@gmail.com>
>> wrote:
>> > I'm trying to configure ssh for the Hadoop mapreduce, but my nodes only
>> > communicate with each others using RSA keys in pem format.
>> >
>> > (It doesn't work)
>> > ssh user@host
>> > Permission denied (publickey).
>> >
>> > (It works)
>> > ssh -i ~/key.pem user@host
>> >
>> > The nodes in mapreduce communicate using ssh. How I configure the ssh,
>> or
>> > the mapreduce to work with the pem file.
>> >
>> >
>> > --
>> > Best regards,
>> > P
>>
>>
>>
>> --
>> Harsh J
>>
>
>
>
> --
> Best regards,
>
--
Nitin Pawar
Re: configure mapreduce to work with pem files.
Posted by Pedro Sá da Costa <ps...@gmail.com>.
So, why it is necessary to configure ssh in Hadoop MR?
On 13 February 2013 12:58, Harsh J <ha...@cloudera.com> wrote:
> Hi,
>
> Nodes in Hadoop do not communicate using SSH. See
> http://wiki.apache.org/hadoop/FAQ#Does_Hadoop_require_SSH.3F
>
> On Wed, Feb 13, 2013 at 5:16 PM, Pedro Sá da Costa <ps...@gmail.com>
> wrote:
> > I'm trying to configure ssh for the Hadoop mapreduce, but my nodes only
> > communicate with each others using RSA keys in pem format.
> >
> > (It doesn't work)
> > ssh user@host
> > Permission denied (publickey).
> >
> > (It works)
> > ssh -i ~/key.pem user@host
> >
> > The nodes in mapreduce communicate using ssh. How I configure the ssh, or
> > the mapreduce to work with the pem file.
> >
> >
> > --
> > Best regards,
> > P
>
>
>
> --
> Harsh J
>
--
Best regards,
Re: configure mapreduce to work with pem files.
Posted by Pedro Sá da Costa <ps...@gmail.com>.
So, why it is necessary to configure ssh in Hadoop MR?
On 13 February 2013 12:58, Harsh J <ha...@cloudera.com> wrote:
> Hi,
>
> Nodes in Hadoop do not communicate using SSH. See
> http://wiki.apache.org/hadoop/FAQ#Does_Hadoop_require_SSH.3F
>
> On Wed, Feb 13, 2013 at 5:16 PM, Pedro Sá da Costa <ps...@gmail.com>
> wrote:
> > I'm trying to configure ssh for the Hadoop mapreduce, but my nodes only
> > communicate with each others using RSA keys in pem format.
> >
> > (It doesn't work)
> > ssh user@host
> > Permission denied (publickey).
> >
> > (It works)
> > ssh -i ~/key.pem user@host
> >
> > The nodes in mapreduce communicate using ssh. How I configure the ssh, or
> > the mapreduce to work with the pem file.
> >
> >
> > --
> > Best regards,
> > P
>
>
>
> --
> Harsh J
>
--
Best regards,
Re: configure mapreduce to work with pem files.
Posted by Pedro Sá da Costa <ps...@gmail.com>.
So, why it is necessary to configure ssh in Hadoop MR?
On 13 February 2013 12:58, Harsh J <ha...@cloudera.com> wrote:
> Hi,
>
> Nodes in Hadoop do not communicate using SSH. See
> http://wiki.apache.org/hadoop/FAQ#Does_Hadoop_require_SSH.3F
>
> On Wed, Feb 13, 2013 at 5:16 PM, Pedro Sá da Costa <ps...@gmail.com>
> wrote:
> > I'm trying to configure ssh for the Hadoop mapreduce, but my nodes only
> > communicate with each others using RSA keys in pem format.
> >
> > (It doesn't work)
> > ssh user@host
> > Permission denied (publickey).
> >
> > (It works)
> > ssh -i ~/key.pem user@host
> >
> > The nodes in mapreduce communicate using ssh. How I configure the ssh, or
> > the mapreduce to work with the pem file.
> >
> >
> > --
> > Best regards,
> > P
>
>
>
> --
> Harsh J
>
--
Best regards,
Re: configure mapreduce to work with pem files.
Posted by Pedro Sá da Costa <ps...@gmail.com>.
So, why it is necessary to configure ssh in Hadoop MR?
On 13 February 2013 12:58, Harsh J <ha...@cloudera.com> wrote:
> Hi,
>
> Nodes in Hadoop do not communicate using SSH. See
> http://wiki.apache.org/hadoop/FAQ#Does_Hadoop_require_SSH.3F
>
> On Wed, Feb 13, 2013 at 5:16 PM, Pedro Sá da Costa <ps...@gmail.com>
> wrote:
> > I'm trying to configure ssh for the Hadoop mapreduce, but my nodes only
> > communicate with each others using RSA keys in pem format.
> >
> > (It doesn't work)
> > ssh user@host
> > Permission denied (publickey).
> >
> > (It works)
> > ssh -i ~/key.pem user@host
> >
> > The nodes in mapreduce communicate using ssh. How I configure the ssh, or
> > the mapreduce to work with the pem file.
> >
> >
> > --
> > Best regards,
> > P
>
>
>
> --
> Harsh J
>
--
Best regards,
Re: configure mapreduce to work with pem files.
Posted by Harsh J <ha...@cloudera.com>.
Hi,
Nodes in Hadoop do not communicate using SSH. See
http://wiki.apache.org/hadoop/FAQ#Does_Hadoop_require_SSH.3F
On Wed, Feb 13, 2013 at 5:16 PM, Pedro Sá da Costa <ps...@gmail.com> wrote:
> I'm trying to configure ssh for the Hadoop mapreduce, but my nodes only
> communicate with each others using RSA keys in pem format.
>
> (It doesn't work)
> ssh user@host
> Permission denied (publickey).
>
> (It works)
> ssh -i ~/key.pem user@host
>
> The nodes in mapreduce communicate using ssh. How I configure the ssh, or
> the mapreduce to work with the pem file.
>
>
> --
> Best regards,
> P
--
Harsh J
Re: configure mapreduce to work with pem files.
Posted by Harsh J <ha...@cloudera.com>.
Hi,
Nodes in Hadoop do not communicate using SSH. See
http://wiki.apache.org/hadoop/FAQ#Does_Hadoop_require_SSH.3F
On Wed, Feb 13, 2013 at 5:16 PM, Pedro Sá da Costa <ps...@gmail.com> wrote:
> I'm trying to configure ssh for the Hadoop mapreduce, but my nodes only
> communicate with each others using RSA keys in pem format.
>
> (It doesn't work)
> ssh user@host
> Permission denied (publickey).
>
> (It works)
> ssh -i ~/key.pem user@host
>
> The nodes in mapreduce communicate using ssh. How I configure the ssh, or
> the mapreduce to work with the pem file.
>
>
> --
> Best regards,
> P
--
Harsh J
Re: configure mapreduce to work with pem files.
Posted by Harsh J <ha...@cloudera.com>.
Hi,
Nodes in Hadoop do not communicate using SSH. See
http://wiki.apache.org/hadoop/FAQ#Does_Hadoop_require_SSH.3F
On Wed, Feb 13, 2013 at 5:16 PM, Pedro Sá da Costa <ps...@gmail.com> wrote:
> I'm trying to configure ssh for the Hadoop mapreduce, but my nodes only
> communicate with each others using RSA keys in pem format.
>
> (It doesn't work)
> ssh user@host
> Permission denied (publickey).
>
> (It works)
> ssh -i ~/key.pem user@host
>
> The nodes in mapreduce communicate using ssh. How I configure the ssh, or
> the mapreduce to work with the pem file.
>
>
> --
> Best regards,
> P
--
Harsh J
Re: configure mapreduce to work with pem files.
Posted by Chris Embree <ce...@gmail.com>.
You need to configure ssh to use your pem files, by default it uses dsa or
rsa files. Look at man ssh_config.
On Wed, Feb 13, 2013 at 6:46 AM, Pedro Sá da Costa <ps...@gmail.com>wrote:
> I'm trying to configure ssh for the Hadoop mapreduce, but my nodes only
> communicate with each others using RSA keys in pem format.
>
> (It doesn't work)
> ssh user@host
> Permission denied (publickey).
>
> (It works)
> ssh -i ~/key.pem user@host
>
> The nodes in mapreduce communicate using ssh. How I configure the ssh, or
> the mapreduce to work with the pem file.
>
>
> --
> Best regards,
> P
>
Re: configure mapreduce to work with pem files.
Posted by Harsh J <ha...@cloudera.com>.
Hi,
Nodes in Hadoop do not communicate using SSH. See
http://wiki.apache.org/hadoop/FAQ#Does_Hadoop_require_SSH.3F
On Wed, Feb 13, 2013 at 5:16 PM, Pedro Sá da Costa <ps...@gmail.com> wrote:
> I'm trying to configure ssh for the Hadoop mapreduce, but my nodes only
> communicate with each others using RSA keys in pem format.
>
> (It doesn't work)
> ssh user@host
> Permission denied (publickey).
>
> (It works)
> ssh -i ~/key.pem user@host
>
> The nodes in mapreduce communicate using ssh. How I configure the ssh, or
> the mapreduce to work with the pem file.
>
>
> --
> Best regards,
> P
--
Harsh J
Re: configure mapreduce to work with pem files.
Posted by Chris Embree <ce...@gmail.com>.
You need to configure ssh to use your pem files, by default it uses dsa or
rsa files. Look at man ssh_config.
On Wed, Feb 13, 2013 at 6:46 AM, Pedro Sá da Costa <ps...@gmail.com>wrote:
> I'm trying to configure ssh for the Hadoop mapreduce, but my nodes only
> communicate with each others using RSA keys in pem format.
>
> (It doesn't work)
> ssh user@host
> Permission denied (publickey).
>
> (It works)
> ssh -i ~/key.pem user@host
>
> The nodes in mapreduce communicate using ssh. How I configure the ssh, or
> the mapreduce to work with the pem file.
>
>
> --
> Best regards,
> P
>
Re: configure mapreduce to work with pem files.
Posted by Chris Embree <ce...@gmail.com>.
You need to configure ssh to use your pem files, by default it uses dsa or
rsa files. Look at man ssh_config.
On Wed, Feb 13, 2013 at 6:46 AM, Pedro Sá da Costa <ps...@gmail.com>wrote:
> I'm trying to configure ssh for the Hadoop mapreduce, but my nodes only
> communicate with each others using RSA keys in pem format.
>
> (It doesn't work)
> ssh user@host
> Permission denied (publickey).
>
> (It works)
> ssh -i ~/key.pem user@host
>
> The nodes in mapreduce communicate using ssh. How I configure the ssh, or
> the mapreduce to work with the pem file.
>
>
> --
> Best regards,
> P
>
Re: configure mapreduce to work with pem files.
Posted by Chris Embree <ce...@gmail.com>.
You need to configure ssh to use your pem files, by default it uses dsa or
rsa files. Look at man ssh_config.
On Wed, Feb 13, 2013 at 6:46 AM, Pedro Sá da Costa <ps...@gmail.com>wrote:
> I'm trying to configure ssh for the Hadoop mapreduce, but my nodes only
> communicate with each others using RSA keys in pem format.
>
> (It doesn't work)
> ssh user@host
> Permission denied (publickey).
>
> (It works)
> ssh -i ~/key.pem user@host
>
> The nodes in mapreduce communicate using ssh. How I configure the ssh, or
> the mapreduce to work with the pem file.
>
>
> --
> Best regards,
> P
>