Will interactive password authentication fail talk between namenode-datanode/jobtracker-tasktracker?

[jiang licht <li...@yahoo.com>]

I set up a simple cluster with one master (namenode@50001 and jobtracker@50002) and one slave. The problem is that although namenode/datanode and jobracker/tasktracker are running but there is no datanode in the dfs! Both datanode and tasktracker reports similar messages in their logs:

INFO org.apache.hadoop.ipc.RPC: Server at MASTER/IP:50001 not available yet, Zzzzz...
INFO org.apache.hadoop.ipc.RPC: Server at MASTER/IP:50002 not available 
yet, Zzzzz...

However, netstat shows MASTER is listening on port 50001 and 50002 and can telent to both ports on localhost of MASTER. Telnet MASTER 50001/50002 from SLAVE results in "connection refused' instead.

Here's my question, I have to type my password (not PASSPHRASE for key) due to some reverse name resolution problem when I do either SSH MASTER from SLAVE or SSH SLAVE from MASTER. Since my system admin told me all ports are open between them, I am wondering will this interactive authentication prevents datanode/tasktracker from talking to namenode/jobtracker?

Thanks,

Michael


      

Data node cannot talk to name node Re: Will interactive password authentication fail talk between namenode-datanode/jobtracker-tasktracker?

[jiang licht <li...@yahoo.com>]

Thanks Edward. Since the string of "reverse mapping ..." is just a warning, I guess it won't be a issue.

Now, the namenode "A" is listening on port "a". No data node sitting on a different box can talk to A@a to join the cluster. But assign "A" also as a datanode is ok and this datanode can join the cluster since it sits on the same box as namenode. Telnet from other machines to A@a fails. To verify that the slave box is allowed, I also added both IP and name (according to boolean inHostsList(DatanodeID node, String ipAddr) in "FSNamesystem.java" which does access control) to "slaves" and the file specified for "dfs.hosts" but I didn't use a exclude list. Namenode box can ssh to itself and to slaves. Also I verified the IP address of slave is the ip address listed in "slaves" and the file specified for "dfs.hosts". And the ports are open for access from slaves on the namenode machine. For example, from slaves, telnet to 50070/50030@namenode is allowed and by firing a "GET /" returns correct html pages. So, ssh is ok and access control is ok. Really
 confusing. What may cause this problem which prevents data nodes from talking to namenode?  Any thoughts?

Thanks,
--

Michael

--- On Thu, 3/4/10, Edward Capriolo <ed...@gmail.com> wrote:

From: Edward Capriolo <ed...@gmail.com>
Subject: Re: Will interactive password authentication fail talk between  namenode-datanode/jobtracker-tasktracker?
To: common-user@hadoop.apache.org
Date: Thursday, March 4, 2010, 3:23 PM

On Thu, Mar 4, 2010 at 2:01 PM, Edson Ramiro <er...@gmail.com> wrote:
> You don't need DNS.
>
> You can use the /etc/hosts.
>
> I'm using it here and it's working well.
>
> Edson Ramiro
>
>
> On 4 March 2010 14:39, Allen Wittenauer <aw...@linkedin.com> wrote:
>
>>
>>
>>
>> On 3/3/10 3:38 PM, "jiang licht" <li...@yahoo.com> wrote:
>> > Here's my question, I have to type my password (not PASSPHRASE for key)
>> due to
>> > some reverse name resolution problem when I do either SSH MASTER from
>> SLAVE or
>> > SSH SLAVE from MASTER. Since my system admin told me all ports are open
>> > between them, I am wondering will this interactive authentication
>> prevents
>> > datanode/tasktracker from talking to namenode/jobtracker?
>>
>> If reverse name resolution isn't working, then you are going to have all
>> sorts of problems.  DNS needs to be properly configured.  [I wish I knew
>> where the whole "you don't need reverses configured for dns" thing came
>> from
>> amongst admins.]
>>
>>
>


Adding an entry to DNS adds the entry to your forward and reverse
lookup table, thus fixing/hiding the problem. Host file is not a
winning strategy long term for obvious reasons :)



      

Re: Will interactive password authentication fail talk between namenode-datanode/jobtracker-tasktracker?

[Edward Capriolo <ed...@gmail.com>]

On Thu, Mar 4, 2010 at 2:01 PM, Edson Ramiro <er...@gmail.com> wrote:
> You don't need DNS.
>
> You can use the /etc/hosts.
>
> I'm using it here and it's working well.
>
> Edson Ramiro
>
>
> On 4 March 2010 14:39, Allen Wittenauer <aw...@linkedin.com> wrote:
>
>>
>>
>>
>> On 3/3/10 3:38 PM, "jiang licht" <li...@yahoo.com> wrote:
>> > Here's my question, I have to type my password (not PASSPHRASE for key)
>> due to
>> > some reverse name resolution problem when I do either SSH MASTER from
>> SLAVE or
>> > SSH SLAVE from MASTER. Since my system admin told me all ports are open
>> > between them, I am wondering will this interactive authentication
>> prevents
>> > datanode/tasktracker from talking to namenode/jobtracker?
>>
>> If reverse name resolution isn't working, then you are going to have all
>> sorts of problems.  DNS needs to be properly configured.  [I wish I knew
>> where the whole "you don't need reverses configured for dns" thing came
>> from
>> amongst admins.]
>>
>>
>


Adding an entry to DNS adds the entry to your forward and reverse
lookup table, thus fixing/hiding the problem. Host file is not a
winning strategy long term for obvious reasons :)

Re: Will interactive password authentication fail talk between namenode-datanode/jobtracker-tasktracker?

[Edson Ramiro <er...@gmail.com>]

You don't need DNS.

You can use the /etc/hosts.

I'm using it here and it's working well.

Edson Ramiro


On 4 March 2010 14:39, Allen Wittenauer <aw...@linkedin.com> wrote:

>
>
>
> On 3/3/10 3:38 PM, "jiang licht" <li...@yahoo.com> wrote:
> > Here's my question, I have to type my password (not PASSPHRASE for key)
> due to
> > some reverse name resolution problem when I do either SSH MASTER from
> SLAVE or
> > SSH SLAVE from MASTER. Since my system admin told me all ports are open
> > between them, I am wondering will this interactive authentication
> prevents
> > datanode/tasktracker from talking to namenode/jobtracker?
>
> If reverse name resolution isn't working, then you are going to have all
> sorts of problems.  DNS needs to be properly configured.  [I wish I knew
> where the whole "you don't need reverses configured for dns" thing came
> from
> amongst admins.]
>
>

Re: Will interactive password authentication fail talk between namenode-datanode/jobtracker-tasktracker?

[Allen Wittenauer <aw...@linkedin.com>]

On 3/3/10 3:38 PM, "jiang licht" <li...@yahoo.com> wrote:
> Here's my question, I have to type my password (not PASSPHRASE for key) due to
> some reverse name resolution problem when I do either SSH MASTER from SLAVE or
> SSH SLAVE from MASTER. Since my system admin told me all ports are open
> between them, I am wondering will this interactive authentication prevents
> datanode/tasktracker from talking to namenode/jobtracker?

If reverse name resolution isn't working, then you are going to have all
sorts of problems.  DNS needs to be properly configured.  [I wish I knew
where the whole "you don't need reverses configured for dns" thing came from
amongst admins.]