You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Eli Finkelshteyn <ie...@gmail.com> on 2012/01/09 19:14:06 UTC

Netstat Shows Port 8020 Doesn't Seem to Listen

Hi,
I've been googling, but haven't been able to find an answer. I'm 
currently trying to setup Hadoop in pseudo-distributed mode as a first 
step. I'm using the Cloudera distro and installed everything through YUM 
on CentOS 5.7. I can run everything just fine from my one node itself 
(hadoop fs -ls /, test map-red jobs, etc...), but can't get a remote 
client to be able to connect to it. I'm pretty sure the cause of that 
has to do with the fact that port 8020 and port 8021 do not seem to be 
listening (when I do a netstat -a, they don't show up-- all the other 
Hadoop related ports like 50030 and 50070 do show up). I verified that 
the firewall allows connections over 8020 and 8021 for tcp, and can 
connect through my web browser to 50030 and 50070.

Looking at the namenode log, I see the following error which looks 
suspicious and related to me:

    2012-01-09 12:03:38,000 INFO org.apache.hadoop.ipc.Server: IPC
    Server listener on 8020: starting
    2012-01-09 12:03:38,009 INFO org.apache.hadoop.ipc.Server: IPC
    Server handler 0 on 8020: starting
    2012-01-09 12:03:39,187 INFO org.apache.hadoop.ipc.Server: IPC
    Server handler 2 on 8020: starting
    2012-01-09 12:03:39,188 INFO org.apache.hadoop.ipc.Server: IPC
    Server handler 3 on 8020: starting
    2012-01-09 12:03:39,188 INFO org.apache.hadoop.ipc.Server: IPC
    Server handler 4 on 8020: starting
    2012-01-09 12:03:39,188 INFO org.apache.hadoop.ipc.Server: IPC
    Server handler 5 on 8020: starting
    2012-01-09 12:03:39,188 INFO org.apache.hadoop.ipc.Server: IPC
    Server handler 6 on 8020: starting
    2012-01-09 12:03:39,189 INFO org.apache.hadoop.ipc.Server: IPC
    Server handler 1 on 8020: starting
    2012-01-09 12:03:39,189 INFO org.apache.hadoop.ipc.Server: IPC
    Server handler 7 on 8020: starting
    2012-01-09 12:03:39,189 INFO org.apache.hadoop.ipc.Server: IPC
    Server handler 8 on 8020: starting
    2012-01-09 12:03:39,246 INFO org.apache.hadoop.ipc.Server: IPC
    Server handler 9 on 8020: starting
    2012-01-09 12:03:39,258 WARN
    org.apache.hadoop.util.PluginDispatcher: Unable to load
    dfs.namenode.plugins plugins
    2012-01-09 12:03:40,254 INFO org.apache.hadoop.ipc.Server: IPC
    Server handler 8 on 8020, call
    addBlock(/var/lib/hadoop-0.20/cache/mapred/mapred/system/jobtracker.info,
    DFSClient_-1779116177, null) from 127.0.0.1:39785: error:
    java.io.IOException: File
    /var/lib/hadoop-0.20/cache/mapred/mapred/system/jobtracker.info
    could only be replicated to 0 nodes, instead of 1

Anyone have any idea what my problem might be?

Cheers,
Eli

Re: Netstat Shows Port 8020 Doesn't Seem to Listen

Posted by Eli Finkelshteyn <ie...@gmail.com>.
Thanks for the help, Idris. I checked all the confs you mentioned, and 
all is as it should be. jps gives me:

24226 Jps
24073 TaskTracker
23854 JobTracker
23780 DataNode
23921 NameNode
23995 SecondaryNameNode

So that looks good. A majority of this stuff is default as set by 
Cloudera. Any other ideas?

Eli

On 1/9/12 3:22 PM, Idris Ali wrote:
> Hi,
>
> Looks like problem in starting DFS and MR, can you run 'jps' and see if NN,
> DN, SNN, JT and TT are running,
>
> also make sure for pseudo-distributed mode, the following entries are
> present:
>
> 1. In core-site.xml
>   <property>
>      <name>fs.default.name</name>
>      <value>hdfs://localhost:8020</value>
>    </property>
>
>    <property>
>       <name>hadoop.tmp.dir</name>
>       <value><SOME TMP dir with Read/Write acces not system temp></value>
>    </property>
>    <property>
>
> 2.  In hdfs-site.xml
> <property>
>      <name>dfs.replication</name>
>      <value>1</value>
>    </property>
>    <property>
>       <name>dfs.permissions</name>
>       <value>false</value>
>    </property>
>    <property>
>       <!-- specify this so that running 'hadoop namenode -format' formats
> the right dir -->
>       <name>dfs.name.dir</name>
>       <value>Local dir with Read/Write access</value>
>    </property>
>
> 3. In mapred-stie.xml
>    <property>
>      <name>mapred.job.tracker</name>
>      <value>localhost:8021</value>
>    </property>
>
> Thanks,
> -Idris
>
> On Tue, Jan 10, 2012 at 1:07 AM, Eli Finkelshteyn<ie...@gmail.com>wrote:
>
>> Positive. Like I said before, netstat -a | grep 8020 gives me nothing.
>> Even if the firewall was the problem, that should still give me output that
>> the port is listening, but I'd just be unable to hit it from an outside box
>> (I tested this by blocking port 50070, at which point it still showed up in
>> netstat -a, but was inaccessible through http from a remote machine). This
>> problem is something else.
>>
>>
>> On 1/9/12 2:31 PM, zGreenfelder wrote:
>>
>>> On Mon, Jan 9, 2012 at 1:58 PM, Eli Finkelshteyn<ie...@gmail.com>>
>>>   wrote:
>>>
>>>> More info:
>>>>
>>>> In the DataNode log, I'm also seeing:
>>>>
>>>> 2012-01-09 13:06:27,751 INFO org.apache.hadoop.ipc.Client: Retrying
>>>> connect
>>>> to server: localhost/127.0.0.1:8020. Already tried 9 time(s).
>>>>
>>>> Why would things just not load on port 8020? I feel like all the errors
>>>> I'm
>>>> seeing are caused by this, but I can't see any errors about why this
>>>> occurred in the first place.
>>>>
>>>>   are you sure there isn't a firewall in place blocking port 8020?
>>> e.g. iptables on the local machines?   if you do
>>> telnet localhost 8020
>>> do you make a connection? if you use lsof and/or netstat can you see
>>> the port open?
>>> if you have root access you can try turning off the firewall with
>>> iptables -F to see if things work without firewall rules.
>>>
>>


RE: Netstat Shows Port 8020 Doesn't Seem to Listen

Posted by Vivek Shrivastava <vs...@Shopzilla.com>.
One of the option to troubleshoot is to run open your url in the lynx, on the command line it will be not be affected by any firewall and will be a local access...

Regards,

Vivek

-----Original Message-----
From: Eli Finkelshteyn [mailto:iefinkel@gmail.com] 
Sent: Monday, January 09, 2012 2:39 PM
To: common-user@hadoop.apache.org
Subject: Re: Netstat Shows Port 8020 Doesn't Seem to Listen

Good call! netstat -anl gives me:
tcp        0      0 ::ffff:127.0.0.1:8020       
:::*                        LISTEN

Now it just looks like nothing is running on 8021. And now I'm really 
confused about why I get no communication over 8020 from the datanode.

Just to reiterate, this definitely is not the firewall, running iptables 
-nvL gives:

...
     0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            
0.0.0.0/0           state NEW tcp dpt:50070
     1    64 ACCEPT     tcp  --  *      *       0.0.0.0/0            
0.0.0.0/0           state NEW tcp dpt:50030
     0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            
0.0.0.0/0           state NEW tcp dpt:8021
     1    64 ACCEPT     tcp  --  *      *       0.0.0.0/0            
0.0.0.0/0           state NEW tcp dpt:8020
...

On 1/9/12 5:08 PM, alo.alt wrote:
> What happen when you try a "telnet localhost 8020"?
> netstat -anl would also useful.
>
> best,
>   Alex
>
> --
> Alexander Lorenz
> http://mapredit.blogspot.com
>
> On Jan 9, 2012, at 2:02 PM, Eli Finkelshteyn wrote:
>
>> A bit more info:
>>
>> When I start up only the namenode by itself, I'm not seeing any errors, but what I am seeing that's really odd is:
>>
>>    2012-01-09 16:48:45,530 INFO org.apache.hadoop.ipc.Server: Starting
>>    Socket Reader #1 for port 8020
>>    2012-01-09 16:48:45,531 INFO
>>    org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics
>>    with hostName=NameNode, port=8020
>>    2012-01-09 16:48:45,532 INFO
>>    org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC
>>    Metrics with hostName=NameNode, port=8020
>>    2012-01-09 16:48:45,541 INFO
>>    org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at:
>>    localhost.localdomain/127.0.0.1:8020
>>
>> That's despite the fact that doing netstat -a | grep 8020 still returns nothing.  To me, that makes absolutely no sense. I feel like I should be getting an error telling me Namenode did not in fact go up on 8020, but I'm not getting that at all.
>>
>> Eli
>>
>> On 1/9/12 3:22 PM, Idris Ali wrote:
>>> Hi,
>>>
>>> Looks like problem in starting DFS and MR, can you run 'jps' and see if NN,
>>> DN, SNN, JT and TT are running,
>>>
>>> also make sure for pseudo-distributed mode, the following entries are
>>> present:
>>>
>>> 1. In core-site.xml
>>>   <property>
>>>      <name>fs.default.name</name>
>>>      <value>hdfs://localhost:8020</value>
>>>    </property>
>>>
>>>    <property>
>>>       <name>hadoop.tmp.dir</name>
>>>       <value><SOME TMP dir with Read/Write acces not system temp></value>
>>>    </property>
>>>    <property>
>>>
>>> 2.  In hdfs-site.xml
>>> <property>
>>>      <name>dfs.replication</name>
>>>      <value>1</value>
>>>    </property>
>>>    <property>
>>>       <name>dfs.permissions</name>
>>>       <value>false</value>
>>>    </property>
>>>    <property>
>>>       <!-- specify this so that running 'hadoop namenode -format' formats
>>> the right dir -->
>>>       <name>dfs.name.dir</name>
>>>       <value>Local dir with Read/Write access</value>
>>>    </property>
>>>
>>> 3. In mapred-stie.xml
>>>    <property>
>>>      <name>mapred.job.tracker</name>
>>>      <value>localhost:8021</value>
>>>    </property>
>>>
>>> Thanks,
>>> -Idris
>>>
>>> On Tue, Jan 10, 2012 at 1:07 AM, Eli Finkelshteyn<ie...@gmail.com>wrote:
>>>
>>>> Positive. Like I said before, netstat -a | grep 8020 gives me nothing.
>>>> Even if the firewall was the problem, that should still give me output that
>>>> the port is listening, but I'd just be unable to hit it from an outside box
>>>> (I tested this by blocking port 50070, at which point it still showed up in
>>>> netstat -a, but was inaccessible through http from a remote machine). This
>>>> problem is something else.
>>>>
>>>>
>>>> On 1/9/12 2:31 PM, zGreenfelder wrote:
>>>>
>>>>> On Mon, Jan 9, 2012 at 1:58 PM, Eli Finkelshteyn<ie...@gmail.com>>
>>>>>   wrote:
>>>>>
>>>>>> More info:
>>>>>>
>>>>>> In the DataNode log, I'm also seeing:
>>>>>>
>>>>>> 2012-01-09 13:06:27,751 INFO org.apache.hadoop.ipc.Client: Retrying
>>>>>> connect
>>>>>> to server: localhost/127.0.0.1:8020. Already tried 9 time(s).
>>>>>>
>>>>>> Why would things just not load on port 8020? I feel like all the errors
>>>>>> I'm
>>>>>> seeing are caused by this, but I can't see any errors about why this
>>>>>> occurred in the first place.
>>>>>>
>>>>>>   are you sure there isn't a firewall in place blocking port 8020?
>>>>> e.g. iptables on the local machines?   if you do
>>>>> telnet localhost 8020
>>>>> do you make a connection? if you use lsof and/or netstat can you see
>>>>> the port open?
>>>>> if you have root access you can try turning off the firewall with
>>>>> iptables -F to see if things work without firewall rules.
>>>>>



Re: Netstat Shows Port 8020 Doesn't Seem to Listen

Posted by Eli Finkelshteyn <ie...@gmail.com>.
OK, switched over to my site's dns name and I'm golden. Everything works 
both locally and remotely. My suspicion is that the problem all along 
was tied to the firewall not being restarted, and then my exacerbating 
the problem through trying to fix it and corrupting the hdfs file 
system. Anyway, all works now. Thanks for the help, everyone!

Eli

On 1/9/12 6:11 PM, Eli Finkelshteyn wrote:
> OK, not sure what I did (restarting the firewall, perhaps?), but I now 
> have ports 8020 and 8021 listening and no more errors in my logs. 
> Wooo! Only problem is I still can't get any hadoop stuff to work from 
> a remote client:
>
> hadoop fs -ls /
> 2012-01-09 17:53:53.559 java[13396:1903] Unable to load realm info 
> from SCDynamicStore
> 12/01/09 17:53:55 INFO ipc.Client: Retrying connect to server: 
> *my_server/my_ip*:8020. Already tried 0 time(s).
> 12/01/09 17:53:56 INFO ipc.Client: Retrying connect to server: 
> *my_server/my_ip*:8020. Already tried 1 time(s).
> 12/01/09 17:53:57 INFO ipc.Client: Retrying connect to server: 
> *my_server/my_ip*:8020. Already tried 2 time(s).
> ...
>
> I feel like I'm almost there. Might this have to do with the fact that 
> core-site.xml and mapred-site.xml specify localhost for ports 8020 and 
> 8021 (thus not listening to any attempted outside connections?)
>
> Thanks for all the help so far, everyone!
>
> Eli
>
> On 1/9/12 5:43 PM, alo.alt wrote:
>> Firewall online?
>> and be sure that in /etc/hosts ONLY 127.0.0.1 is linked to localhost. Nothing like YOURHOSTNAME.YOURDOMAIN (Redhat kudzu bug)
>>
>> - Alex
>>
>> --
>> Alexander Lorenz
>> http://mapredit.blogspot.com
>>
>> On Jan 9, 2012, at 2:39 PM, Eli Finkelshteyn wrote:
>>
>>> Good call! netstat -anl gives me:
>>> tcp        0      0 ::ffff:127.0.0.1:8020       :::*                        LISTEN
>>>
>>> Now it just looks like nothing is running on 8021. And now I'm really confused about why I get no communication over 8020 from the datanode.
>>>
>>> Just to reiterate, this definitely is not the firewall, running iptables -nvL gives:
>>>
>>> ...
>>>     0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:50070
>>>     1    64 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:50030
>>>     0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:8021
>>>     1    64 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:8020
>>> ...
>>>
>>> On 1/9/12 5:08 PM, alo.alt wrote:
>>>> What happen when you try a "telnet localhost 8020"?
>>>> netstat -anl would also useful.
>>>>
>>>> best,
>>>>   Alex
>>>>
>>>> --
>>>> Alexander Lorenz
>>>> http://mapredit.blogspot.com
>>>>
>>>> On Jan 9, 2012, at 2:02 PM, Eli Finkelshteyn wrote:
>>>>
>>>>> A bit more info:
>>>>>
>>>>> When I start up only the namenode by itself, I'm not seeing any errors, but what I am seeing that's really odd is:
>>>>>
>>>>>    2012-01-09 16:48:45,530 INFO org.apache.hadoop.ipc.Server: Starting
>>>>>    Socket Reader #1 for port 8020
>>>>>    2012-01-09 16:48:45,531 INFO
>>>>>    org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics
>>>>>    with hostName=NameNode, port=8020
>>>>>    2012-01-09 16:48:45,532 INFO
>>>>>    org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC
>>>>>    Metrics with hostName=NameNode, port=8020
>>>>>    2012-01-09 16:48:45,541 INFO
>>>>>    org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at:
>>>>>    localhost.localdomain/127.0.0.1:8020
>>>>>
>>>>> That's despite the fact that doing netstat -a | grep 8020 still returns nothing.  To me, that makes absolutely no sense. I feel like I should be getting an error telling me Namenode did not in fact go up on 8020, but I'm not getting that at all.
>>>>>
>>>>> Eli
>>>>>
>>>>> On 1/9/12 3:22 PM, Idris Ali wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Looks like problem in starting DFS and MR, can you run 'jps' and see if NN,
>>>>>> DN, SNN, JT and TT are running,
>>>>>>
>>>>>> also make sure for pseudo-distributed mode, the following entries are
>>>>>> present:
>>>>>>
>>>>>> 1. In core-site.xml
>>>>>>   <property>
>>>>>>      <name>fs.default.name</name>
>>>>>>      <value>hdfs://localhost:8020</value>
>>>>>>    </property>
>>>>>>
>>>>>>    <property>
>>>>>>       <name>hadoop.tmp.dir</name>
>>>>>>       <value><SOME TMP dir with Read/Write acces not system temp></value>
>>>>>>    </property>
>>>>>>    <property>
>>>>>>
>>>>>> 2.  In hdfs-site.xml
>>>>>> <property>
>>>>>>      <name>dfs.replication</name>
>>>>>>      <value>1</value>
>>>>>>    </property>
>>>>>>    <property>
>>>>>>       <name>dfs.permissions</name>
>>>>>>       <value>false</value>
>>>>>>    </property>
>>>>>>    <property>
>>>>>>       <!-- specify this so that running 'hadoop namenode -format' formats
>>>>>> the right dir -->
>>>>>>       <name>dfs.name.dir</name>
>>>>>>       <value>Local dir with Read/Write access</value>
>>>>>>    </property>
>>>>>>
>>>>>> 3. In mapred-stie.xml
>>>>>>    <property>
>>>>>>      <name>mapred.job.tracker</name>
>>>>>>      <value>localhost:8021</value>
>>>>>>    </property>
>>>>>>
>>>>>> Thanks,
>>>>>> -Idris
>>>>>>
>>>>>> On Tue, Jan 10, 2012 at 1:07 AM, Eli Finkelshteyn<ie...@gmail.com>wrote:
>>>>>>
>>>>>>> Positive. Like I said before, netstat -a | grep 8020 gives me nothing.
>>>>>>> Even if the firewall was the problem, that should still give me output that
>>>>>>> the port is listening, but I'd just be unable to hit it from an outside box
>>>>>>> (I tested this by blocking port 50070, at which point it still showed up in
>>>>>>> netstat -a, but was inaccessible through http from a remote machine). This
>>>>>>> problem is something else.
>>>>>>>
>>>>>>>
>>>>>>> On 1/9/12 2:31 PM, zGreenfelder wrote:
>>>>>>>
>>>>>>>> On Mon, Jan 9, 2012 at 1:58 PM, Eli Finkelshteyn<ie...@gmail.com>>
>>>>>>>>   wrote:
>>>>>>>>
>>>>>>>>> More info:
>>>>>>>>>
>>>>>>>>> In the DataNode log, I'm also seeing:
>>>>>>>>>
>>>>>>>>> 2012-01-09 13:06:27,751 INFO org.apache.hadoop.ipc.Client: Retrying
>>>>>>>>> connect
>>>>>>>>> to server: localhost/127.0.0.1:8020. Already tried 9 time(s).
>>>>>>>>>
>>>>>>>>> Why would things just not load on port 8020? I feel like all the errors
>>>>>>>>> I'm
>>>>>>>>> seeing are caused by this, but I can't see any errors about why this
>>>>>>>>> occurred in the first place.
>>>>>>>>>
>>>>>>>>>   are you sure there isn't a firewall in place blocking port 8020?
>>>>>>>> e.g. iptables on the local machines?   if you do
>>>>>>>> telnet localhost 8020
>>>>>>>> do you make a connection? if you use lsof and/or netstat can you see
>>>>>>>> the port open?
>>>>>>>> if you have root access you can try turning off the firewall with
>>>>>>>> iptables -F to see if things work without firewall rules.
>>>>>>>>
>


Re: Netstat Shows Port 8020 Doesn't Seem to Listen

Posted by Eli Finkelshteyn <ie...@gmail.com>.
OK, not sure what I did (restarting the firewall, perhaps?), but I now 
have ports 8020 and 8021 listening and no more errors in my logs. Wooo! 
Only problem is I still can't get any hadoop stuff to work from a remote 
client:

hadoop fs -ls /
2012-01-09 17:53:53.559 java[13396:1903] Unable to load realm info from 
SCDynamicStore
12/01/09 17:53:55 INFO ipc.Client: Retrying connect to server: 
*my_server/my_ip*:8020. Already tried 0 time(s).
12/01/09 17:53:56 INFO ipc.Client: Retrying connect to server: 
*my_server/my_ip*:8020. Already tried 1 time(s).
12/01/09 17:53:57 INFO ipc.Client: Retrying connect to server: 
*my_server/my_ip*:8020. Already tried 2 time(s).
...

I feel like I'm almost there. Might this have to do with the fact that 
core-site.xml and mapred-site.xml specify localhost for ports 8020 and 
8021 (thus not listening to any attempted outside connections?)

Thanks for all the help so far, everyone!

Eli

On 1/9/12 5:43 PM, alo.alt wrote:
> Firewall online?
> and be sure that in /etc/hosts ONLY 127.0.0.1 is linked to localhost. Nothing like YOURHOSTNAME.YOURDOMAIN (Redhat kudzu bug)
>
> - Alex
>
> --
> Alexander Lorenz
> http://mapredit.blogspot.com
>
> On Jan 9, 2012, at 2:39 PM, Eli Finkelshteyn wrote:
>
>> Good call! netstat -anl gives me:
>> tcp        0      0 ::ffff:127.0.0.1:8020       :::*                        LISTEN
>>
>> Now it just looks like nothing is running on 8021. And now I'm really confused about why I get no communication over 8020 from the datanode.
>>
>> Just to reiterate, this definitely is not the firewall, running iptables -nvL gives:
>>
>> ...
>>     0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:50070
>>     1    64 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:50030
>>     0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:8021
>>     1    64 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:8020
>> ...
>>
>> On 1/9/12 5:08 PM, alo.alt wrote:
>>> What happen when you try a "telnet localhost 8020"?
>>> netstat -anl would also useful.
>>>
>>> best,
>>>   Alex
>>>
>>> --
>>> Alexander Lorenz
>>> http://mapredit.blogspot.com
>>>
>>> On Jan 9, 2012, at 2:02 PM, Eli Finkelshteyn wrote:
>>>
>>>> A bit more info:
>>>>
>>>> When I start up only the namenode by itself, I'm not seeing any errors, but what I am seeing that's really odd is:
>>>>
>>>>    2012-01-09 16:48:45,530 INFO org.apache.hadoop.ipc.Server: Starting
>>>>    Socket Reader #1 for port 8020
>>>>    2012-01-09 16:48:45,531 INFO
>>>>    org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics
>>>>    with hostName=NameNode, port=8020
>>>>    2012-01-09 16:48:45,532 INFO
>>>>    org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC
>>>>    Metrics with hostName=NameNode, port=8020
>>>>    2012-01-09 16:48:45,541 INFO
>>>>    org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at:
>>>>    localhost.localdomain/127.0.0.1:8020
>>>>
>>>> That's despite the fact that doing netstat -a | grep 8020 still returns nothing.  To me, that makes absolutely no sense. I feel like I should be getting an error telling me Namenode did not in fact go up on 8020, but I'm not getting that at all.
>>>>
>>>> Eli
>>>>
>>>> On 1/9/12 3:22 PM, Idris Ali wrote:
>>>>> Hi,
>>>>>
>>>>> Looks like problem in starting DFS and MR, can you run 'jps' and see if NN,
>>>>> DN, SNN, JT and TT are running,
>>>>>
>>>>> also make sure for pseudo-distributed mode, the following entries are
>>>>> present:
>>>>>
>>>>> 1. In core-site.xml
>>>>>   <property>
>>>>>      <name>fs.default.name</name>
>>>>>      <value>hdfs://localhost:8020</value>
>>>>>    </property>
>>>>>
>>>>>    <property>
>>>>>       <name>hadoop.tmp.dir</name>
>>>>>       <value><SOME TMP dir with Read/Write acces not system temp></value>
>>>>>    </property>
>>>>>    <property>
>>>>>
>>>>> 2.  In hdfs-site.xml
>>>>> <property>
>>>>>      <name>dfs.replication</name>
>>>>>      <value>1</value>
>>>>>    </property>
>>>>>    <property>
>>>>>       <name>dfs.permissions</name>
>>>>>       <value>false</value>
>>>>>    </property>
>>>>>    <property>
>>>>>       <!-- specify this so that running 'hadoop namenode -format' formats
>>>>> the right dir -->
>>>>>       <name>dfs.name.dir</name>
>>>>>       <value>Local dir with Read/Write access</value>
>>>>>    </property>
>>>>>
>>>>> 3. In mapred-stie.xml
>>>>>    <property>
>>>>>      <name>mapred.job.tracker</name>
>>>>>      <value>localhost:8021</value>
>>>>>    </property>
>>>>>
>>>>> Thanks,
>>>>> -Idris
>>>>>
>>>>> On Tue, Jan 10, 2012 at 1:07 AM, Eli Finkelshteyn<ie...@gmail.com>wrote:
>>>>>
>>>>>> Positive. Like I said before, netstat -a | grep 8020 gives me nothing.
>>>>>> Even if the firewall was the problem, that should still give me output that
>>>>>> the port is listening, but I'd just be unable to hit it from an outside box
>>>>>> (I tested this by blocking port 50070, at which point it still showed up in
>>>>>> netstat -a, but was inaccessible through http from a remote machine). This
>>>>>> problem is something else.
>>>>>>
>>>>>>
>>>>>> On 1/9/12 2:31 PM, zGreenfelder wrote:
>>>>>>
>>>>>>> On Mon, Jan 9, 2012 at 1:58 PM, Eli Finkelshteyn<ie...@gmail.com>>
>>>>>>>   wrote:
>>>>>>>
>>>>>>>> More info:
>>>>>>>>
>>>>>>>> In the DataNode log, I'm also seeing:
>>>>>>>>
>>>>>>>> 2012-01-09 13:06:27,751 INFO org.apache.hadoop.ipc.Client: Retrying
>>>>>>>> connect
>>>>>>>> to server: localhost/127.0.0.1:8020. Already tried 9 time(s).
>>>>>>>>
>>>>>>>> Why would things just not load on port 8020? I feel like all the errors
>>>>>>>> I'm
>>>>>>>> seeing are caused by this, but I can't see any errors about why this
>>>>>>>> occurred in the first place.
>>>>>>>>
>>>>>>>>   are you sure there isn't a firewall in place blocking port 8020?
>>>>>>> e.g. iptables on the local machines?   if you do
>>>>>>> telnet localhost 8020
>>>>>>> do you make a connection? if you use lsof and/or netstat can you see
>>>>>>> the port open?
>>>>>>> if you have root access you can try turning off the firewall with
>>>>>>> iptables -F to see if things work without firewall rules.
>>>>>>>


Re: Netstat Shows Port 8020 Doesn't Seem to Listen

Posted by "alo.alt" <wg...@googlemail.com>.
Firewall online?
and be sure that in /etc/hosts ONLY 127.0.0.1 is linked to localhost. Nothing like YOURHOSTNAME.YOURDOMAIN (Redhat kudzu bug)

- Alex 

--
Alexander Lorenz
http://mapredit.blogspot.com

On Jan 9, 2012, at 2:39 PM, Eli Finkelshteyn wrote:

> Good call! netstat -anl gives me:
> tcp        0      0 ::ffff:127.0.0.1:8020       :::*                        LISTEN
> 
> Now it just looks like nothing is running on 8021. And now I'm really confused about why I get no communication over 8020 from the datanode.
> 
> Just to reiterate, this definitely is not the firewall, running iptables -nvL gives:
> 
> ...
>    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:50070
>    1    64 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:50030
>    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:8021
>    1    64 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           state NEW tcp dpt:8020
> ...
> 
> On 1/9/12 5:08 PM, alo.alt wrote:
>> What happen when you try a "telnet localhost 8020"?
>> netstat -anl would also useful.
>> 
>> best,
>>  Alex
>> 
>> --
>> Alexander Lorenz
>> http://mapredit.blogspot.com
>> 
>> On Jan 9, 2012, at 2:02 PM, Eli Finkelshteyn wrote:
>> 
>>> A bit more info:
>>> 
>>> When I start up only the namenode by itself, I'm not seeing any errors, but what I am seeing that's really odd is:
>>> 
>>>   2012-01-09 16:48:45,530 INFO org.apache.hadoop.ipc.Server: Starting
>>>   Socket Reader #1 for port 8020
>>>   2012-01-09 16:48:45,531 INFO
>>>   org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics
>>>   with hostName=NameNode, port=8020
>>>   2012-01-09 16:48:45,532 INFO
>>>   org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC
>>>   Metrics with hostName=NameNode, port=8020
>>>   2012-01-09 16:48:45,541 INFO
>>>   org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at:
>>>   localhost.localdomain/127.0.0.1:8020
>>> 
>>> That's despite the fact that doing netstat -a | grep 8020 still returns nothing.  To me, that makes absolutely no sense. I feel like I should be getting an error telling me Namenode did not in fact go up on 8020, but I'm not getting that at all.
>>> 
>>> Eli
>>> 
>>> On 1/9/12 3:22 PM, Idris Ali wrote:
>>>> Hi,
>>>> 
>>>> Looks like problem in starting DFS and MR, can you run 'jps' and see if NN,
>>>> DN, SNN, JT and TT are running,
>>>> 
>>>> also make sure for pseudo-distributed mode, the following entries are
>>>> present:
>>>> 
>>>> 1. In core-site.xml
>>>>  <property>
>>>>     <name>fs.default.name</name>
>>>>     <value>hdfs://localhost:8020</value>
>>>>   </property>
>>>> 
>>>>   <property>
>>>>      <name>hadoop.tmp.dir</name>
>>>>      <value><SOME TMP dir with Read/Write acces not system temp></value>
>>>>   </property>
>>>>   <property>
>>>> 
>>>> 2.  In hdfs-site.xml
>>>> <property>
>>>>     <name>dfs.replication</name>
>>>>     <value>1</value>
>>>>   </property>
>>>>   <property>
>>>>      <name>dfs.permissions</name>
>>>>      <value>false</value>
>>>>   </property>
>>>>   <property>
>>>>      <!-- specify this so that running 'hadoop namenode -format' formats
>>>> the right dir -->
>>>>      <name>dfs.name.dir</name>
>>>>      <value>Local dir with Read/Write access</value>
>>>>   </property>
>>>> 
>>>> 3. In mapred-stie.xml
>>>>   <property>
>>>>     <name>mapred.job.tracker</name>
>>>>     <value>localhost:8021</value>
>>>>   </property>
>>>> 
>>>> Thanks,
>>>> -Idris
>>>> 
>>>> On Tue, Jan 10, 2012 at 1:07 AM, Eli Finkelshteyn<ie...@gmail.com>wrote:
>>>> 
>>>>> Positive. Like I said before, netstat -a | grep 8020 gives me nothing.
>>>>> Even if the firewall was the problem, that should still give me output that
>>>>> the port is listening, but I'd just be unable to hit it from an outside box
>>>>> (I tested this by blocking port 50070, at which point it still showed up in
>>>>> netstat -a, but was inaccessible through http from a remote machine). This
>>>>> problem is something else.
>>>>> 
>>>>> 
>>>>> On 1/9/12 2:31 PM, zGreenfelder wrote:
>>>>> 
>>>>>> On Mon, Jan 9, 2012 at 1:58 PM, Eli Finkelshteyn<ie...@gmail.com>>
>>>>>>  wrote:
>>>>>> 
>>>>>>> More info:
>>>>>>> 
>>>>>>> In the DataNode log, I'm also seeing:
>>>>>>> 
>>>>>>> 2012-01-09 13:06:27,751 INFO org.apache.hadoop.ipc.Client: Retrying
>>>>>>> connect
>>>>>>> to server: localhost/127.0.0.1:8020. Already tried 9 time(s).
>>>>>>> 
>>>>>>> Why would things just not load on port 8020? I feel like all the errors
>>>>>>> I'm
>>>>>>> seeing are caused by this, but I can't see any errors about why this
>>>>>>> occurred in the first place.
>>>>>>> 
>>>>>>>  are you sure there isn't a firewall in place blocking port 8020?
>>>>>> e.g. iptables on the local machines?   if you do
>>>>>> telnet localhost 8020
>>>>>> do you make a connection? if you use lsof and/or netstat can you see
>>>>>> the port open?
>>>>>> if you have root access you can try turning off the firewall with
>>>>>> iptables -F to see if things work without firewall rules.
>>>>>> 
> 


Re: Netstat Shows Port 8020 Doesn't Seem to Listen

Posted by Eli Finkelshteyn <ie...@gmail.com>.
Good call! netstat -anl gives me:
tcp        0      0 ::ffff:127.0.0.1:8020       
:::*                        LISTEN

Now it just looks like nothing is running on 8021. And now I'm really 
confused about why I get no communication over 8020 from the datanode.

Just to reiterate, this definitely is not the firewall, running iptables 
-nvL gives:

...
     0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            
0.0.0.0/0           state NEW tcp dpt:50070
     1    64 ACCEPT     tcp  --  *      *       0.0.0.0/0            
0.0.0.0/0           state NEW tcp dpt:50030
     0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            
0.0.0.0/0           state NEW tcp dpt:8021
     1    64 ACCEPT     tcp  --  *      *       0.0.0.0/0            
0.0.0.0/0           state NEW tcp dpt:8020
...

On 1/9/12 5:08 PM, alo.alt wrote:
> What happen when you try a "telnet localhost 8020"?
> netstat -anl would also useful.
>
> best,
>   Alex
>
> --
> Alexander Lorenz
> http://mapredit.blogspot.com
>
> On Jan 9, 2012, at 2:02 PM, Eli Finkelshteyn wrote:
>
>> A bit more info:
>>
>> When I start up only the namenode by itself, I'm not seeing any errors, but what I am seeing that's really odd is:
>>
>>    2012-01-09 16:48:45,530 INFO org.apache.hadoop.ipc.Server: Starting
>>    Socket Reader #1 for port 8020
>>    2012-01-09 16:48:45,531 INFO
>>    org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics
>>    with hostName=NameNode, port=8020
>>    2012-01-09 16:48:45,532 INFO
>>    org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC
>>    Metrics with hostName=NameNode, port=8020
>>    2012-01-09 16:48:45,541 INFO
>>    org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at:
>>    localhost.localdomain/127.0.0.1:8020
>>
>> That's despite the fact that doing netstat -a | grep 8020 still returns nothing.  To me, that makes absolutely no sense. I feel like I should be getting an error telling me Namenode did not in fact go up on 8020, but I'm not getting that at all.
>>
>> Eli
>>
>> On 1/9/12 3:22 PM, Idris Ali wrote:
>>> Hi,
>>>
>>> Looks like problem in starting DFS and MR, can you run 'jps' and see if NN,
>>> DN, SNN, JT and TT are running,
>>>
>>> also make sure for pseudo-distributed mode, the following entries are
>>> present:
>>>
>>> 1. In core-site.xml
>>>   <property>
>>>      <name>fs.default.name</name>
>>>      <value>hdfs://localhost:8020</value>
>>>    </property>
>>>
>>>    <property>
>>>       <name>hadoop.tmp.dir</name>
>>>       <value><SOME TMP dir with Read/Write acces not system temp></value>
>>>    </property>
>>>    <property>
>>>
>>> 2.  In hdfs-site.xml
>>> <property>
>>>      <name>dfs.replication</name>
>>>      <value>1</value>
>>>    </property>
>>>    <property>
>>>       <name>dfs.permissions</name>
>>>       <value>false</value>
>>>    </property>
>>>    <property>
>>>       <!-- specify this so that running 'hadoop namenode -format' formats
>>> the right dir -->
>>>       <name>dfs.name.dir</name>
>>>       <value>Local dir with Read/Write access</value>
>>>    </property>
>>>
>>> 3. In mapred-stie.xml
>>>    <property>
>>>      <name>mapred.job.tracker</name>
>>>      <value>localhost:8021</value>
>>>    </property>
>>>
>>> Thanks,
>>> -Idris
>>>
>>> On Tue, Jan 10, 2012 at 1:07 AM, Eli Finkelshteyn<ie...@gmail.com>wrote:
>>>
>>>> Positive. Like I said before, netstat -a | grep 8020 gives me nothing.
>>>> Even if the firewall was the problem, that should still give me output that
>>>> the port is listening, but I'd just be unable to hit it from an outside box
>>>> (I tested this by blocking port 50070, at which point it still showed up in
>>>> netstat -a, but was inaccessible through http from a remote machine). This
>>>> problem is something else.
>>>>
>>>>
>>>> On 1/9/12 2:31 PM, zGreenfelder wrote:
>>>>
>>>>> On Mon, Jan 9, 2012 at 1:58 PM, Eli Finkelshteyn<ie...@gmail.com>>
>>>>>   wrote:
>>>>>
>>>>>> More info:
>>>>>>
>>>>>> In the DataNode log, I'm also seeing:
>>>>>>
>>>>>> 2012-01-09 13:06:27,751 INFO org.apache.hadoop.ipc.Client: Retrying
>>>>>> connect
>>>>>> to server: localhost/127.0.0.1:8020. Already tried 9 time(s).
>>>>>>
>>>>>> Why would things just not load on port 8020? I feel like all the errors
>>>>>> I'm
>>>>>> seeing are caused by this, but I can't see any errors about why this
>>>>>> occurred in the first place.
>>>>>>
>>>>>>   are you sure there isn't a firewall in place blocking port 8020?
>>>>> e.g. iptables on the local machines?   if you do
>>>>> telnet localhost 8020
>>>>> do you make a connection? if you use lsof and/or netstat can you see
>>>>> the port open?
>>>>> if you have root access you can try turning off the firewall with
>>>>> iptables -F to see if things work without firewall rules.
>>>>>


Re: Netstat Shows Port 8020 Doesn't Seem to Listen

Posted by "alo.alt" <wg...@googlemail.com>.
What happen when you try a "telnet localhost 8020"?
netstat -anl would also useful.

best,
 Alex 

--
Alexander Lorenz
http://mapredit.blogspot.com

On Jan 9, 2012, at 2:02 PM, Eli Finkelshteyn wrote:

> A bit more info:
> 
> When I start up only the namenode by itself, I'm not seeing any errors, but what I am seeing that's really odd is:
> 
>   2012-01-09 16:48:45,530 INFO org.apache.hadoop.ipc.Server: Starting
>   Socket Reader #1 for port 8020
>   2012-01-09 16:48:45,531 INFO
>   org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics
>   with hostName=NameNode, port=8020
>   2012-01-09 16:48:45,532 INFO
>   org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC
>   Metrics with hostName=NameNode, port=8020
>   2012-01-09 16:48:45,541 INFO
>   org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at:
>   localhost.localdomain/127.0.0.1:8020
> 
> That's despite the fact that doing netstat -a | grep 8020 still returns nothing.  To me, that makes absolutely no sense. I feel like I should be getting an error telling me Namenode did not in fact go up on 8020, but I'm not getting that at all.
> 
> Eli
> 
> On 1/9/12 3:22 PM, Idris Ali wrote:
>> Hi,
>> 
>> Looks like problem in starting DFS and MR, can you run 'jps' and see if NN,
>> DN, SNN, JT and TT are running,
>> 
>> also make sure for pseudo-distributed mode, the following entries are
>> present:
>> 
>> 1. In core-site.xml
>>  <property>
>>     <name>fs.default.name</name>
>>     <value>hdfs://localhost:8020</value>
>>   </property>
>> 
>>   <property>
>>      <name>hadoop.tmp.dir</name>
>>      <value><SOME TMP dir with Read/Write acces not system temp></value>
>>   </property>
>>   <property>
>> 
>> 2.  In hdfs-site.xml
>> <property>
>>     <name>dfs.replication</name>
>>     <value>1</value>
>>   </property>
>>   <property>
>>      <name>dfs.permissions</name>
>>      <value>false</value>
>>   </property>
>>   <property>
>>      <!-- specify this so that running 'hadoop namenode -format' formats
>> the right dir -->
>>      <name>dfs.name.dir</name>
>>      <value>Local dir with Read/Write access</value>
>>   </property>
>> 
>> 3. In mapred-stie.xml
>>   <property>
>>     <name>mapred.job.tracker</name>
>>     <value>localhost:8021</value>
>>   </property>
>> 
>> Thanks,
>> -Idris
>> 
>> On Tue, Jan 10, 2012 at 1:07 AM, Eli Finkelshteyn<ie...@gmail.com>wrote:
>> 
>>> Positive. Like I said before, netstat -a | grep 8020 gives me nothing.
>>> Even if the firewall was the problem, that should still give me output that
>>> the port is listening, but I'd just be unable to hit it from an outside box
>>> (I tested this by blocking port 50070, at which point it still showed up in
>>> netstat -a, but was inaccessible through http from a remote machine). This
>>> problem is something else.
>>> 
>>> 
>>> On 1/9/12 2:31 PM, zGreenfelder wrote:
>>> 
>>>> On Mon, Jan 9, 2012 at 1:58 PM, Eli Finkelshteyn<ie...@gmail.com>>
>>>>  wrote:
>>>> 
>>>>> More info:
>>>>> 
>>>>> In the DataNode log, I'm also seeing:
>>>>> 
>>>>> 2012-01-09 13:06:27,751 INFO org.apache.hadoop.ipc.Client: Retrying
>>>>> connect
>>>>> to server: localhost/127.0.0.1:8020. Already tried 9 time(s).
>>>>> 
>>>>> Why would things just not load on port 8020? I feel like all the errors
>>>>> I'm
>>>>> seeing are caused by this, but I can't see any errors about why this
>>>>> occurred in the first place.
>>>>> 
>>>>>  are you sure there isn't a firewall in place blocking port 8020?
>>>> e.g. iptables on the local machines?   if you do
>>>> telnet localhost 8020
>>>> do you make a connection? if you use lsof and/or netstat can you see
>>>> the port open?
>>>> if you have root access you can try turning off the firewall with
>>>> iptables -F to see if things work without firewall rules.
>>>> 
>>> 
> 


Re: Netstat Shows Port 8020 Doesn't Seem to Listen

Posted by Eli Finkelshteyn <ie...@gmail.com>.
A bit more info:

When I start up only the namenode by itself, I'm not seeing any errors, 
but what I am seeing that's really odd is:

    2012-01-09 16:48:45,530 INFO org.apache.hadoop.ipc.Server: Starting
    Socket Reader #1 for port 8020
    2012-01-09 16:48:45,531 INFO
    org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics
    with hostName=NameNode, port=8020
    2012-01-09 16:48:45,532 INFO
    org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC
    Metrics with hostName=NameNode, port=8020
    2012-01-09 16:48:45,541 INFO
    org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at:
    localhost.localdomain/127.0.0.1:8020

That's despite the fact that doing netstat -a | grep 8020 still returns 
nothing.  To me, that makes absolutely no sense. I feel like I should be 
getting an error telling me Namenode did not in fact go up on 8020, but 
I'm not getting that at all.

Eli

On 1/9/12 3:22 PM, Idris Ali wrote:
> Hi,
>
> Looks like problem in starting DFS and MR, can you run 'jps' and see if NN,
> DN, SNN, JT and TT are running,
>
> also make sure for pseudo-distributed mode, the following entries are
> present:
>
> 1. In core-site.xml
>   <property>
>      <name>fs.default.name</name>
>      <value>hdfs://localhost:8020</value>
>    </property>
>
>    <property>
>       <name>hadoop.tmp.dir</name>
>       <value><SOME TMP dir with Read/Write acces not system temp></value>
>    </property>
>    <property>
>
> 2.  In hdfs-site.xml
> <property>
>      <name>dfs.replication</name>
>      <value>1</value>
>    </property>
>    <property>
>       <name>dfs.permissions</name>
>       <value>false</value>
>    </property>
>    <property>
>       <!-- specify this so that running 'hadoop namenode -format' formats
> the right dir -->
>       <name>dfs.name.dir</name>
>       <value>Local dir with Read/Write access</value>
>    </property>
>
> 3. In mapred-stie.xml
>    <property>
>      <name>mapred.job.tracker</name>
>      <value>localhost:8021</value>
>    </property>
>
> Thanks,
> -Idris
>
> On Tue, Jan 10, 2012 at 1:07 AM, Eli Finkelshteyn<ie...@gmail.com>wrote:
>
>> Positive. Like I said before, netstat -a | grep 8020 gives me nothing.
>> Even if the firewall was the problem, that should still give me output that
>> the port is listening, but I'd just be unable to hit it from an outside box
>> (I tested this by blocking port 50070, at which point it still showed up in
>> netstat -a, but was inaccessible through http from a remote machine). This
>> problem is something else.
>>
>>
>> On 1/9/12 2:31 PM, zGreenfelder wrote:
>>
>>> On Mon, Jan 9, 2012 at 1:58 PM, Eli Finkelshteyn<ie...@gmail.com>>
>>>   wrote:
>>>
>>>> More info:
>>>>
>>>> In the DataNode log, I'm also seeing:
>>>>
>>>> 2012-01-09 13:06:27,751 INFO org.apache.hadoop.ipc.Client: Retrying
>>>> connect
>>>> to server: localhost/127.0.0.1:8020. Already tried 9 time(s).
>>>>
>>>> Why would things just not load on port 8020? I feel like all the errors
>>>> I'm
>>>> seeing are caused by this, but I can't see any errors about why this
>>>> occurred in the first place.
>>>>
>>>>   are you sure there isn't a firewall in place blocking port 8020?
>>> e.g. iptables on the local machines?   if you do
>>> telnet localhost 8020
>>> do you make a connection? if you use lsof and/or netstat can you see
>>> the port open?
>>> if you have root access you can try turning off the firewall with
>>> iptables -F to see if things work without firewall rules.
>>>
>>


Re: Netstat Shows Port 8020 Doesn't Seem to Listen

Posted by Idris Ali <ps...@gmail.com>.
Hi,

Looks like problem in starting DFS and MR, can you run 'jps' and see if NN,
DN, SNN, JT and TT are running,

also make sure for pseudo-distributed mode, the following entries are
present:

1. In core-site.xml
 <property>
    <name>fs.default.name</name>
    <value>hdfs://localhost:8020</value>
  </property>

  <property>
     <name>hadoop.tmp.dir</name>
     <value><SOME TMP dir with Read/Write acces not system temp></value>
  </property>
  <property>

2.  In hdfs-site.xml
<property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>
  <property>
     <name>dfs.permissions</name>
     <value>false</value>
  </property>
  <property>
     <!-- specify this so that running 'hadoop namenode -format' formats
the right dir -->
     <name>dfs.name.dir</name>
     <value>Local dir with Read/Write access</value>
  </property>

3. In mapred-stie.xml
  <property>
    <name>mapred.job.tracker</name>
    <value>localhost:8021</value>
  </property>

Thanks,
-Idris

On Tue, Jan 10, 2012 at 1:07 AM, Eli Finkelshteyn <ie...@gmail.com>wrote:

> Positive. Like I said before, netstat -a | grep 8020 gives me nothing.
> Even if the firewall was the problem, that should still give me output that
> the port is listening, but I'd just be unable to hit it from an outside box
> (I tested this by blocking port 50070, at which point it still showed up in
> netstat -a, but was inaccessible through http from a remote machine). This
> problem is something else.
>
>
> On 1/9/12 2:31 PM, zGreenfelder wrote:
>
>> On Mon, Jan 9, 2012 at 1:58 PM, Eli Finkelshteyn<ie...@gmail.com>>
>>  wrote:
>>
>>> More info:
>>>
>>> In the DataNode log, I'm also seeing:
>>>
>>> 2012-01-09 13:06:27,751 INFO org.apache.hadoop.ipc.Client: Retrying
>>> connect
>>> to server: localhost/127.0.0.1:8020. Already tried 9 time(s).
>>>
>>> Why would things just not load on port 8020? I feel like all the errors
>>> I'm
>>> seeing are caused by this, but I can't see any errors about why this
>>> occurred in the first place.
>>>
>>>  are you sure there isn't a firewall in place blocking port 8020?
>> e.g. iptables on the local machines?   if you do
>> telnet localhost 8020
>> do you make a connection? if you use lsof and/or netstat can you see
>> the port open?
>> if you have root access you can try turning off the firewall with
>> iptables -F to see if things work without firewall rules.
>>
>
>

Re: Netstat Shows Port 8020 Doesn't Seem to Listen

Posted by Eli Finkelshteyn <ie...@gmail.com>.
<value>hdfs://localhost:8020</value>

On 1/9/12 3:14 PM, Harsh J wrote:
> Eli,
>
> What is your fs.default.name set to, in core-site.xml?
>
> On Tue, Jan 10, 2012 at 1:07 AM, Eli Finkelshteyn<ie...@gmail.com>  wrote:
>> Positive. Like I said before, netstat -a | grep 8020 gives me nothing. Even
>> if the firewall was the problem, that should still give me output that the
>> port is listening, but I'd just be unable to hit it from an outside box (I
>> tested this by blocking port 50070, at which point it still showed up in
>> netstat -a, but was inaccessible through http from a remote machine). This
>> problem is something else.
>>
>>
>> On 1/9/12 2:31 PM, zGreenfelder wrote:
>>> On Mon, Jan 9, 2012 at 1:58 PM, Eli Finkelshteyn<ie...@gmail.com>
>>>   wrote:
>>>> More info:
>>>>
>>>> In the DataNode log, I'm also seeing:
>>>>
>>>> 2012-01-09 13:06:27,751 INFO org.apache.hadoop.ipc.Client: Retrying
>>>> connect
>>>> to server: localhost/127.0.0.1:8020. Already tried 9 time(s).
>>>>
>>>> Why would things just not load on port 8020? I feel like all the errors
>>>> I'm
>>>> seeing are caused by this, but I can't see any errors about why this
>>>> occurred in the first place.
>>>>
>>> are you sure there isn't a firewall in place blocking port 8020?
>>> e.g. iptables on the local machines?   if you do
>>> telnet localhost 8020
>>> do you make a connection? if you use lsof and/or netstat can you see
>>> the port open?
>>> if you have root access you can try turning off the firewall with
>>> iptables -F to see if things work without firewall rules.
>>
>
>


Re: Netstat Shows Port 8020 Doesn't Seem to Listen

Posted by Harsh J <ha...@cloudera.com>.
Eli,

What is your fs.default.name set to, in core-site.xml?

On Tue, Jan 10, 2012 at 1:07 AM, Eli Finkelshteyn <ie...@gmail.com> wrote:
> Positive. Like I said before, netstat -a | grep 8020 gives me nothing. Even
> if the firewall was the problem, that should still give me output that the
> port is listening, but I'd just be unable to hit it from an outside box (I
> tested this by blocking port 50070, at which point it still showed up in
> netstat -a, but was inaccessible through http from a remote machine). This
> problem is something else.
>
>
> On 1/9/12 2:31 PM, zGreenfelder wrote:
>>
>> On Mon, Jan 9, 2012 at 1:58 PM, Eli Finkelshteyn<ie...@gmail.com>
>>  wrote:
>>>
>>> More info:
>>>
>>> In the DataNode log, I'm also seeing:
>>>
>>> 2012-01-09 13:06:27,751 INFO org.apache.hadoop.ipc.Client: Retrying
>>> connect
>>> to server: localhost/127.0.0.1:8020. Already tried 9 time(s).
>>>
>>> Why would things just not load on port 8020? I feel like all the errors
>>> I'm
>>> seeing are caused by this, but I can't see any errors about why this
>>> occurred in the first place.
>>>
>> are you sure there isn't a firewall in place blocking port 8020?
>> e.g. iptables on the local machines?   if you do
>> telnet localhost 8020
>> do you make a connection? if you use lsof and/or netstat can you see
>> the port open?
>> if you have root access you can try turning off the firewall with
>> iptables -F to see if things work without firewall rules.
>
>



-- 
Harsh J

Re: Netstat Shows Port 8020 Doesn't Seem to Listen

Posted by Eli Finkelshteyn <ie...@gmail.com>.
Positive. Like I said before, netstat -a | grep 8020 gives me nothing. 
Even if the firewall was the problem, that should still give me output 
that the port is listening, but I'd just be unable to hit it from an 
outside box (I tested this by blocking port 50070, at which point it 
still showed up in netstat -a, but was inaccessible through http from a 
remote machine). This problem is something else.

On 1/9/12 2:31 PM, zGreenfelder wrote:
> On Mon, Jan 9, 2012 at 1:58 PM, Eli Finkelshteyn<ie...@gmail.com>  wrote:
>> More info:
>>
>> In the DataNode log, I'm also seeing:
>>
>> 2012-01-09 13:06:27,751 INFO org.apache.hadoop.ipc.Client: Retrying connect
>> to server: localhost/127.0.0.1:8020. Already tried 9 time(s).
>>
>> Why would things just not load on port 8020? I feel like all the errors I'm
>> seeing are caused by this, but I can't see any errors about why this
>> occurred in the first place.
>>
> are you sure there isn't a firewall in place blocking port 8020?
> e.g. iptables on the local machines?   if you do
> telnet localhost 8020
> do you make a connection? if you use lsof and/or netstat can you see
> the port open?
> if you have root access you can try turning off the firewall with
> iptables -F to see if things work without firewall rules.


Re: Netstat Shows Port 8020 Doesn't Seem to Listen

Posted by zGreenfelder <zg...@gmail.com>.
On Mon, Jan 9, 2012 at 1:58 PM, Eli Finkelshteyn <ie...@gmail.com> wrote:
> More info:
>
> In the DataNode log, I'm also seeing:
>
> 2012-01-09 13:06:27,751 INFO org.apache.hadoop.ipc.Client: Retrying connect
> to server: localhost/127.0.0.1:8020. Already tried 9 time(s).
>
> Why would things just not load on port 8020? I feel like all the errors I'm
> seeing are caused by this, but I can't see any errors about why this
> occurred in the first place.
>

are you sure there isn't a firewall in place blocking port 8020?
e.g. iptables on the local machines?   if you do
telnet localhost 8020
do you make a connection? if you use lsof and/or netstat can you see
the port open?
if you have root access you can try turning off the firewall with
iptables -F to see if things work without firewall rules.
-- 
Even the Magic 8 ball has an opinion on email clients: Outlook not so good.

Re: Netstat Shows Port 8020 Doesn't Seem to Listen

Posted by Eli Finkelshteyn <ie...@gmail.com>.
More info:

In the DataNode log, I'm also seeing:

2012-01-09 13:06:27,751 INFO org.apache.hadoop.ipc.Client: Retrying 
connect to server: localhost/127.0.0.1:8020. Already tried 9 time(s).

Why would things just not load on port 8020? I feel like all the errors 
I'm seeing are caused by this, but I can't see any errors about why this 
occurred in the first place.


On 1/9/12 1:14 PM, Eli Finkelshteyn wrote:
> Hi,
> I've been googling, but haven't been able to find an answer. I'm 
> currently trying to setup Hadoop in pseudo-distributed mode as a first 
> step. I'm using the Cloudera distro and installed everything through 
> YUM on CentOS 5.7. I can run everything just fine from my one node 
> itself (hadoop fs -ls /, test map-red jobs, etc...), but can't get a 
> remote client to be able to connect to it. I'm pretty sure the cause 
> of that has to do with the fact that port 8020 and port 8021 do not 
> seem to be listening (when I do a netstat -a, they don't show up-- all 
> the other Hadoop related ports like 50030 and 50070 do show up). I 
> verified that the firewall allows connections over 8020 and 8021 for 
> tcp, and can connect through my web browser to 50030 and 50070.
>
> Looking at the namenode log, I see the following error which looks 
> suspicious and related to me:
>
>     2012-01-09 12:03:38,000 INFO org.apache.hadoop.ipc.Server: IPC
>     Server listener on 8020: starting
>     2012-01-09 12:03:38,009 INFO org.apache.hadoop.ipc.Server: IPC
>     Server handler 0 on 8020: starting
>     2012-01-09 12:03:39,187 INFO org.apache.hadoop.ipc.Server: IPC
>     Server handler 2 on 8020: starting
>     2012-01-09 12:03:39,188 INFO org.apache.hadoop.ipc.Server: IPC
>     Server handler 3 on 8020: starting
>     2012-01-09 12:03:39,188 INFO org.apache.hadoop.ipc.Server: IPC
>     Server handler 4 on 8020: starting
>     2012-01-09 12:03:39,188 INFO org.apache.hadoop.ipc.Server: IPC
>     Server handler 5 on 8020: starting
>     2012-01-09 12:03:39,188 INFO org.apache.hadoop.ipc.Server: IPC
>     Server handler 6 on 8020: starting
>     2012-01-09 12:03:39,189 INFO org.apache.hadoop.ipc.Server: IPC
>     Server handler 1 on 8020: starting
>     2012-01-09 12:03:39,189 INFO org.apache.hadoop.ipc.Server: IPC
>     Server handler 7 on 8020: starting
>     2012-01-09 12:03:39,189 INFO org.apache.hadoop.ipc.Server: IPC
>     Server handler 8 on 8020: starting
>     2012-01-09 12:03:39,246 INFO org.apache.hadoop.ipc.Server: IPC
>     Server handler 9 on 8020: starting
>     2012-01-09 12:03:39,258 WARN
>     org.apache.hadoop.util.PluginDispatcher: Unable to load
>     dfs.namenode.plugins plugins
>     2012-01-09 12:03:40,254 INFO org.apache.hadoop.ipc.Server: IPC
>     Server handler 8 on 8020, call
>     addBlock(/var/lib/hadoop-0.20/cache/mapred/mapred/system/jobtracker.info,
>     DFSClient_-1779116177, null) from 127.0.0.1:39785: error:
>     java.io.IOException: File
>     /var/lib/hadoop-0.20/cache/mapred/mapred/system/jobtracker.info
>     could only be replicated to 0 nodes, instead of 1
>
> Anyone have any idea what my problem might be?
>
> Cheers,
> Eli