You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by ascend1 <as...@163.com> on 2008/12/19 10:00:50 UTC
Failed to start TaskTracker server
I have made a Hadoop platform on 15 machines recently. NameNode - DataNodes work properly but when I use bin/start-mapred.sh to start MapReduce framework only 3 or 4 TaskTracker could be started properly. All those couldn't be started have the same error.
Here's the log:
2008-12-19 16:16:31,951 INFO org.apache.hadoop.mapred.TaskTracker: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting TaskTracker
STARTUP_MSG: host = msra-5lcd05/172.23.213.80
STARTUP_MSG: args = []
STARTUP_MSG: version = 0.19.0
STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.19 -r 713890; compiled by 'ndaley' on Fri Nov 14 03:12:29 UTC 2008
************************************************************/
2008-12-19 16:16:33,248 INFO org.mortbay.http.HttpServer: Version Jetty/5.1.4
2008-12-19 16:16:33,248 INFO org.mortbay.util.Credential: Checking Resource aliases
2008-12-19 16:16:33,608 INFO org.mortbay.util.Container: Started org.mortbay.jetty.servlet.WebApplicationHandler@e51b2c
2008-12-19 16:16:33,655 INFO org.mortbay.util.Container: Started WebApplicationContext[/static,/static]
2008-12-19 16:16:33,811 INFO org.mortbay.util.Container: Started org.mortbay.jetty.servlet.WebApplicationHandler@edf389
2008-12-19 16:16:33,936 INFO org.mortbay.util.Container: Started WebApplicationContext[/logs,/logs]
2008-12-19 16:16:34,092 INFO org.mortbay.util.Container: Started org.mortbay.jetty.servlet.WebApplicationHandler@17b0998
2008-12-19 16:16:34,092 INFO org.mortbay.util.Container: Started WebApplicationContext[/,/]
2008-12-19 16:16:34,155 WARN org.mortbay.util.ThreadedServer: Failed to start: SocketListener0@0.0.0.0:50060
2008-12-19 16:16:34,155 ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task tracker because java.net.BindException: Address already in use: JVM_Bind
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
at java.net.ServerSocket.bind(ServerSocket.java:319)
at java.net.ServerSocket.<init>(ServerSocket.java:185)
at org.mortbay.util.ThreadedServer.newServerSocket(ThreadedServer.java:391)
at org.mortbay.util.ThreadedServer.open(ThreadedServer.java:477)
at org.mortbay.util.ThreadedServer.start(ThreadedServer.java:503)
at org.mortbay.http.SocketListener.start(SocketListener.java:203)
at org.mortbay.http.HttpServer.doStart(HttpServer.java:761)
at org.mortbay.util.Container.start(Container.java:72)
at org.apache.hadoop.http.HttpServer.start(HttpServer.java:321)
at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:894)
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2698)
2008-12-19 16:16:34,155 INFO org.apache.hadoop.mapred.TaskTracker: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down TaskTracker at msra-5lcd05/172.23.213.80
************************************************************/
Then I use "netstat -an", but port 50060 isn't in the list. "ps -af" also show that no program using 50060. The strange point is that when I repeat "bin/start-mapred.sh" and "bin/stop-mapred.sh" several times, the machines list that could start TaskTracker seems randomly.
Could anybody help me solve this problem?
Re:Re: Failed to start TaskTracker server
Posted by ascend1 <as...@163.com>.
I'll find and have a test. Thanks for your help!
On 2008-12-20，"Sagar Naik" <sn...@attributor.com> wrote：
> - check hadoop-default.xml
>in here u will find all the ports used. Copy the xml-nodes from
>hadoop-default.xml to hadoop-site.xml. Change the port values in
>hadoop-site.xml
>and deploy it on datanodes .
>
>
>Rico wrote:
>> Well the machines are all servers that probably running many services
>> but I have no permission to change or modify other users' programs or
>> settings. Is there any way to change 50060 to other port?
>>
>> Sagar Naik wrote:
>>> Well u have some process which grabs this port and Hadoop is not able
>>> to bind the port
>>> By the time u check, there is a chance that socket connection has
>>> died but was occupied when hadoop processes was attempting
>>>
>>> Check all the processes running on the system
>>> Do any of the processes acquire ports ?
>>>
>>> -Sagar
>>> ascend1 wrote:
>>>> I have made a Hadoop platform on 15 machines recently. NameNode -
>>>> DataNodes work properly but when I use bin/start-mapred.sh to start
>>>> MapReduce framework only 3 or 4 TaskTracker could be started
>>>> properly. All those couldn't be started have the same error.
>>>> Here's the log:
>>>>
>>>> 2008-12-19 16:16:31,951 INFO org.apache.hadoop.mapred.TaskTracker:
>>>> STARTUP_MSG:
>>>> /************************************************************
>>>> STARTUP_MSG: Starting TaskTracker
>>>> STARTUP_MSG: host = msra-5lcd05/172.23.213.80
>>>> STARTUP_MSG: args = []
>>>> STARTUP_MSG: version = 0.19.0
>>>> STARTUP_MSG: build =
>>>> https://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.19 -r
>>>> 713890; compiled by 'ndaley' on Fri Nov 14 03:12:29 UTC 2008
>>>> ************************************************************/
>>>> 2008-12-19 16:16:33,248 INFO org.mortbay.http.HttpServer: Version
>>>> Jetty/5.1.4
>>>> 2008-12-19 16:16:33,248 INFO org.mortbay.util.Credential: Checking
>>>> Resource aliases
>>>> 2008-12-19 16:16:33,608 INFO org.mortbay.util.Container: Started
>>>> org.mortbay.jetty.servlet.WebApplicationHandler@e51b2c
>>>> 2008-12-19 16:16:33,655 INFO org.mortbay.util.Container: Started
>>>> WebApplicationContext[/static,/static]
>>>> 2008-12-19 16:16:33,811 INFO org.mortbay.util.Container: Started
>>>> org.mortbay.jetty.servlet.WebApplicationHandler@edf389
>>>> 2008-12-19 16:16:33,936 INFO org.mortbay.util.Container: Started
>>>> WebApplicationContext[/logs,/logs]
>>>> 2008-12-19 16:16:34,092 INFO org.mortbay.util.Container: Started
>>>> org.mortbay.jetty.servlet.WebApplicationHandler@17b0998
>>>> 2008-12-19 16:16:34,092 INFO org.mortbay.util.Container: Started
>>>> WebApplicationContext[/,/]
>>>> 2008-12-19 16:16:34,155 WARN org.mortbay.util.ThreadedServer: Failed
>>>> to start: SocketListener0@0.0.0.0:50060
>>>> 2008-12-19 16:16:34,155 ERROR org.apache.hadoop.mapred.TaskTracker:
>>>> Can not start task tracker because java.net.BindException: Address
>>>> already in use: JVM_Bind
>>>> at java.net.PlainSocketImpl.socketBind(Native Method)
>>>> at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
>>>> at java.net.ServerSocket.bind(ServerSocket.java:319)
>>>> at java.net.ServerSocket.<init>(ServerSocket.java:185)
>>>> at
>>>> org.mortbay.util.ThreadedServer.newServerSocket(ThreadedServer.java:391)
>>>>
>>>> at org.mortbay.util.ThreadedServer.open(ThreadedServer.java:477)
>>>> at org.mortbay.util.ThreadedServer.start(ThreadedServer.java:503)
>>>> at org.mortbay.http.SocketListener.start(SocketListener.java:203)
>>>> at org.mortbay.http.HttpServer.doStart(HttpServer.java:761)
>>>> at org.mortbay.util.Container.start(Container.java:72)
>>>> at org.apache.hadoop.http.HttpServer.start(HttpServer.java:321)
>>>> at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:894)
>>>> at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2698)
>>>> 2008-12-19 16:16:34,155 INFO org.apache.hadoop.mapred.TaskTracker:
>>>> SHUTDOWN_MSG:
>>>> /************************************************************
>>>> SHUTDOWN_MSG: Shutting down TaskTracker at msra-5lcd05/172.23.213.80
>>>> ************************************************************/
>>>>
>>>> Then I use "netstat -an", but port 50060 isn't in the list. "ps -af"
>>>> also show that no program using 50060. The strange point is that
>>>> when I repeat "bin/start-mapred.sh" and "bin/stop-mapred.sh" several
>>>> times, the machines list that could start TaskTracker seems randomly.
>>>>
>>>> Could anybody help me solve this problem?
>>>
>>>
>>
>>
>
Re: Failed to start TaskTracker server
Posted by Amareshwari Sriramadasu <am...@yahoo-inc.com>.
You can set the configuration property
"mapred.task.tracker.http.address" to 0.0.0.0:0 . If the port is given
as 0, then the server will start on a free port.
Thanks
Amareshwari
Sagar Naik wrote:
>
> - check hadoop-default.xml
> in here u will find all the ports used. Copy the xml-nodes from
> hadoop-default.xml to hadoop-site.xml. Change the port values in
> hadoop-site.xml
> and deploy it on datanodes .
>
>
> Rico wrote:
>> Well the machines are all servers that probably running many services
>> but I have no permission to change or modify other users' programs or
>> settings. Is there any way to change 50060 to other port?
>>
>> Sagar Naik wrote:
>>> Well u have some process which grabs this port and Hadoop is not
>>> able to bind the port
>>> By the time u check, there is a chance that socket connection has
>>> died but was occupied when hadoop processes was attempting
>>>
>>> Check all the processes running on the system
>>> Do any of the processes acquire ports ?
>>>
>>> -Sagar
>>> ascend1 wrote:
>>>> I have made a Hadoop platform on 15 machines recently. NameNode -
>>>> DataNodes work properly but when I use bin/start-mapred.sh to start
>>>> MapReduce framework only 3 or 4 TaskTracker could be started
>>>> properly. All those couldn't be started have the same error.
>>>> Here's the log:
>>>>
>>>> 2008-12-19 16:16:31,951 INFO org.apache.hadoop.mapred.TaskTracker:
>>>> STARTUP_MSG:
>>>> /************************************************************
>>>> STARTUP_MSG: Starting TaskTracker
>>>> STARTUP_MSG: host = msra-5lcd05/172.23.213.80
>>>> STARTUP_MSG: args = []
>>>> STARTUP_MSG: version = 0.19.0
>>>> STARTUP_MSG: build =
>>>> https://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.19
>>>> -r 713890; compiled by 'ndaley' on Fri Nov 14 03:12:29 UTC 2008
>>>> ************************************************************/
>>>> 2008-12-19 16:16:33,248 INFO org.mortbay.http.HttpServer: Version
>>>> Jetty/5.1.4
>>>> 2008-12-19 16:16:33,248 INFO org.mortbay.util.Credential: Checking
>>>> Resource aliases
>>>> 2008-12-19 16:16:33,608 INFO org.mortbay.util.Container: Started
>>>> org.mortbay.jetty.servlet.WebApplicationHandler@e51b2c
>>>> 2008-12-19 16:16:33,655 INFO org.mortbay.util.Container: Started
>>>> WebApplicationContext[/static,/static]
>>>> 2008-12-19 16:16:33,811 INFO org.mortbay.util.Container: Started
>>>> org.mortbay.jetty.servlet.WebApplicationHandler@edf389
>>>> 2008-12-19 16:16:33,936 INFO org.mortbay.util.Container: Started
>>>> WebApplicationContext[/logs,/logs]
>>>> 2008-12-19 16:16:34,092 INFO org.mortbay.util.Container: Started
>>>> org.mortbay.jetty.servlet.WebApplicationHandler@17b0998
>>>> 2008-12-19 16:16:34,092 INFO org.mortbay.util.Container: Started
>>>> WebApplicationContext[/,/]
>>>> 2008-12-19 16:16:34,155 WARN org.mortbay.util.ThreadedServer:
>>>> Failed to start: SocketListener0@0.0.0.0:50060
>>>> 2008-12-19 16:16:34,155 ERROR org.apache.hadoop.mapred.TaskTracker:
>>>> Can not start task tracker because java.net.BindException: Address
>>>> already in use: JVM_Bind
>>>> at java.net.PlainSocketImpl.socketBind(Native Method)
>>>> at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
>>>> at java.net.ServerSocket.bind(ServerSocket.java:319)
>>>> at java.net.ServerSocket.<init>(ServerSocket.java:185)
>>>> at
>>>> org.mortbay.util.ThreadedServer.newServerSocket(ThreadedServer.java:391)
>>>>
>>>> at org.mortbay.util.ThreadedServer.open(ThreadedServer.java:477)
>>>> at org.mortbay.util.ThreadedServer.start(ThreadedServer.java:503)
>>>> at org.mortbay.http.SocketListener.start(SocketListener.java:203)
>>>> at org.mortbay.http.HttpServer.doStart(HttpServer.java:761)
>>>> at org.mortbay.util.Container.start(Container.java:72)
>>>> at org.apache.hadoop.http.HttpServer.start(HttpServer.java:321)
>>>> at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:894)
>>>> at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2698)
>>>> 2008-12-19 16:16:34,155 INFO org.apache.hadoop.mapred.TaskTracker:
>>>> SHUTDOWN_MSG:
>>>> /************************************************************
>>>> SHUTDOWN_MSG: Shutting down TaskTracker at msra-5lcd05/172.23.213.80
>>>> ************************************************************/
>>>>
>>>> Then I use "netstat -an", but port 50060 isn't in the list. "ps
>>>> -af" also show that no program using 50060. The strange point is
>>>> that when I repeat "bin/start-mapred.sh" and "bin/stop-mapred.sh"
>>>> several times, the machines list that could start TaskTracker seems
>>>> randomly.
>>>>
>>>> Could anybody help me solve this problem?
>>>
>>>
>>
>>
>
Re: Failed to start TaskTracker server
Posted by Sagar Naik <sn...@attributor.com>.
- check hadoop-default.xml
in here u will find all the ports used. Copy the xml-nodes from
hadoop-default.xml to hadoop-site.xml. Change the port values in
hadoop-site.xml
and deploy it on datanodes .
Rico wrote:
> Well the machines are all servers that probably running many services
> but I have no permission to change or modify other users' programs or
> settings. Is there any way to change 50060 to other port?
>
> Sagar Naik wrote:
>> Well u have some process which grabs this port and Hadoop is not able
>> to bind the port
>> By the time u check, there is a chance that socket connection has
>> died but was occupied when hadoop processes was attempting
>>
>> Check all the processes running on the system
>> Do any of the processes acquire ports ?
>>
>> -Sagar
>> ascend1 wrote:
>>> I have made a Hadoop platform on 15 machines recently. NameNode -
>>> DataNodes work properly but when I use bin/start-mapred.sh to start
>>> MapReduce framework only 3 or 4 TaskTracker could be started
>>> properly. All those couldn't be started have the same error.
>>> Here's the log:
>>>
>>> 2008-12-19 16:16:31,951 INFO org.apache.hadoop.mapred.TaskTracker:
>>> STARTUP_MSG:
>>> /************************************************************
>>> STARTUP_MSG: Starting TaskTracker
>>> STARTUP_MSG: host = msra-5lcd05/172.23.213.80
>>> STARTUP_MSG: args = []
>>> STARTUP_MSG: version = 0.19.0
>>> STARTUP_MSG: build =
>>> https://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.19 -r
>>> 713890; compiled by 'ndaley' on Fri Nov 14 03:12:29 UTC 2008
>>> ************************************************************/
>>> 2008-12-19 16:16:33,248 INFO org.mortbay.http.HttpServer: Version
>>> Jetty/5.1.4
>>> 2008-12-19 16:16:33,248 INFO org.mortbay.util.Credential: Checking
>>> Resource aliases
>>> 2008-12-19 16:16:33,608 INFO org.mortbay.util.Container: Started
>>> org.mortbay.jetty.servlet.WebApplicationHandler@e51b2c
>>> 2008-12-19 16:16:33,655 INFO org.mortbay.util.Container: Started
>>> WebApplicationContext[/static,/static]
>>> 2008-12-19 16:16:33,811 INFO org.mortbay.util.Container: Started
>>> org.mortbay.jetty.servlet.WebApplicationHandler@edf389
>>> 2008-12-19 16:16:33,936 INFO org.mortbay.util.Container: Started
>>> WebApplicationContext[/logs,/logs]
>>> 2008-12-19 16:16:34,092 INFO org.mortbay.util.Container: Started
>>> org.mortbay.jetty.servlet.WebApplicationHandler@17b0998
>>> 2008-12-19 16:16:34,092 INFO org.mortbay.util.Container: Started
>>> WebApplicationContext[/,/]
>>> 2008-12-19 16:16:34,155 WARN org.mortbay.util.ThreadedServer: Failed
>>> to start: SocketListener0@0.0.0.0:50060
>>> 2008-12-19 16:16:34,155 ERROR org.apache.hadoop.mapred.TaskTracker:
>>> Can not start task tracker because java.net.BindException: Address
>>> already in use: JVM_Bind
>>> at java.net.PlainSocketImpl.socketBind(Native Method)
>>> at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
>>> at java.net.ServerSocket.bind(ServerSocket.java:319)
>>> at java.net.ServerSocket.<init>(ServerSocket.java:185)
>>> at
>>> org.mortbay.util.ThreadedServer.newServerSocket(ThreadedServer.java:391)
>>>
>>> at org.mortbay.util.ThreadedServer.open(ThreadedServer.java:477)
>>> at org.mortbay.util.ThreadedServer.start(ThreadedServer.java:503)
>>> at org.mortbay.http.SocketListener.start(SocketListener.java:203)
>>> at org.mortbay.http.HttpServer.doStart(HttpServer.java:761)
>>> at org.mortbay.util.Container.start(Container.java:72)
>>> at org.apache.hadoop.http.HttpServer.start(HttpServer.java:321)
>>> at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:894)
>>> at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2698)
>>> 2008-12-19 16:16:34,155 INFO org.apache.hadoop.mapred.TaskTracker:
>>> SHUTDOWN_MSG:
>>> /************************************************************
>>> SHUTDOWN_MSG: Shutting down TaskTracker at msra-5lcd05/172.23.213.80
>>> ************************************************************/
>>>
>>> Then I use "netstat -an", but port 50060 isn't in the list. "ps -af"
>>> also show that no program using 50060. The strange point is that
>>> when I repeat "bin/start-mapred.sh" and "bin/stop-mapred.sh" several
>>> times, the machines list that could start TaskTracker seems randomly.
>>>
>>> Could anybody help me solve this problem?
>>
>>
>
>
Re: Failed to start TaskTracker server
Posted by Rico <as...@163.com>.
Well the machines are all servers that probably running many services
but I have no permission to change or modify other users' programs or
settings. Is there any way to change 50060 to other port?
Sagar Naik wrote:
> Well u have some process which grabs this port and Hadoop is not able
> to bind the port
> By the time u check, there is a chance that socket connection has died
> but was occupied when hadoop processes was attempting
>
> Check all the processes running on the system
> Do any of the processes acquire ports ?
>
> -Sagar
> ascend1 wrote:
>> I have made a Hadoop platform on 15 machines recently. NameNode -
>> DataNodes work properly but when I use bin/start-mapred.sh to start
>> MapReduce framework only 3 or 4 TaskTracker could be started
>> properly. All those couldn't be started have the same error.
>> Here's the log:
>>
>> 2008-12-19 16:16:31,951 INFO org.apache.hadoop.mapred.TaskTracker:
>> STARTUP_MSG:
>> /************************************************************
>> STARTUP_MSG: Starting TaskTracker
>> STARTUP_MSG: host = msra-5lcd05/172.23.213.80
>> STARTUP_MSG: args = []
>> STARTUP_MSG: version = 0.19.0
>> STARTUP_MSG: build =
>> https://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.19 -r
>> 713890; compiled by 'ndaley' on Fri Nov 14 03:12:29 UTC 2008
>> ************************************************************/
>> 2008-12-19 16:16:33,248 INFO org.mortbay.http.HttpServer: Version
>> Jetty/5.1.4
>> 2008-12-19 16:16:33,248 INFO org.mortbay.util.Credential: Checking
>> Resource aliases
>> 2008-12-19 16:16:33,608 INFO org.mortbay.util.Container: Started
>> org.mortbay.jetty.servlet.WebApplicationHandler@e51b2c
>> 2008-12-19 16:16:33,655 INFO org.mortbay.util.Container: Started
>> WebApplicationContext[/static,/static]
>> 2008-12-19 16:16:33,811 INFO org.mortbay.util.Container: Started
>> org.mortbay.jetty.servlet.WebApplicationHandler@edf389
>> 2008-12-19 16:16:33,936 INFO org.mortbay.util.Container: Started
>> WebApplicationContext[/logs,/logs]
>> 2008-12-19 16:16:34,092 INFO org.mortbay.util.Container: Started
>> org.mortbay.jetty.servlet.WebApplicationHandler@17b0998
>> 2008-12-19 16:16:34,092 INFO org.mortbay.util.Container: Started
>> WebApplicationContext[/,/]
>> 2008-12-19 16:16:34,155 WARN org.mortbay.util.ThreadedServer: Failed
>> to start: SocketListener0@0.0.0.0:50060
>> 2008-12-19 16:16:34,155 ERROR org.apache.hadoop.mapred.TaskTracker:
>> Can not start task tracker because java.net.BindException: Address
>> already in use: JVM_Bind
>> at java.net.PlainSocketImpl.socketBind(Native Method)
>> at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
>> at java.net.ServerSocket.bind(ServerSocket.java:319)
>> at java.net.ServerSocket.<init>(ServerSocket.java:185)
>> at
>> org.mortbay.util.ThreadedServer.newServerSocket(ThreadedServer.java:391)
>> at org.mortbay.util.ThreadedServer.open(ThreadedServer.java:477)
>> at org.mortbay.util.ThreadedServer.start(ThreadedServer.java:503)
>> at org.mortbay.http.SocketListener.start(SocketListener.java:203)
>> at org.mortbay.http.HttpServer.doStart(HttpServer.java:761)
>> at org.mortbay.util.Container.start(Container.java:72)
>> at org.apache.hadoop.http.HttpServer.start(HttpServer.java:321)
>> at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:894)
>> at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2698)
>> 2008-12-19 16:16:34,155 INFO org.apache.hadoop.mapred.TaskTracker:
>> SHUTDOWN_MSG:
>> /************************************************************
>> SHUTDOWN_MSG: Shutting down TaskTracker at msra-5lcd05/172.23.213.80
>> ************************************************************/
>>
>> Then I use "netstat -an", but port 50060 isn't in the list. "ps -af"
>> also show that no program using 50060. The strange point is that when
>> I repeat "bin/start-mapred.sh" and "bin/stop-mapred.sh" several
>> times, the machines list that could start TaskTracker seems randomly.
>>
>> Could anybody help me solve this problem?
>
>
Re: Failed to start TaskTracker server
Posted by Sagar Naik <sn...@attributor.com>.
Well u have some process which grabs this port and Hadoop is not able to
bind the port
By the time u check, there is a chance that socket connection has died
but was occupied when hadoop processes was attempting
Check all the processes running on the system
Do any of the processes acquire ports ?
-Sagar
ascend1 wrote:
> I have made a Hadoop platform on 15 machines recently. NameNode - DataNodes work properly but when I use bin/start-mapred.sh to start MapReduce framework only 3 or 4 TaskTracker could be started properly. All those couldn't be started have the same error.
> Here's the log:
>
> 2008-12-19 16:16:31,951 INFO org.apache.hadoop.mapred.TaskTracker: STARTUP_MSG:
> /************************************************************
> STARTUP_MSG: Starting TaskTracker
> STARTUP_MSG: host = msra-5lcd05/172.23.213.80
> STARTUP_MSG: args = []
> STARTUP_MSG: version = 0.19.0
> STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.19 -r 713890; compiled by 'ndaley' on Fri Nov 14 03:12:29 UTC 2008
> ************************************************************/
> 2008-12-19 16:16:33,248 INFO org.mortbay.http.HttpServer: Version Jetty/5.1.4
> 2008-12-19 16:16:33,248 INFO org.mortbay.util.Credential: Checking Resource aliases
> 2008-12-19 16:16:33,608 INFO org.mortbay.util.Container: Started org.mortbay.jetty.servlet.WebApplicationHandler@e51b2c
> 2008-12-19 16:16:33,655 INFO org.mortbay.util.Container: Started WebApplicationContext[/static,/static]
> 2008-12-19 16:16:33,811 INFO org.mortbay.util.Container: Started org.mortbay.jetty.servlet.WebApplicationHandler@edf389
> 2008-12-19 16:16:33,936 INFO org.mortbay.util.Container: Started WebApplicationContext[/logs,/logs]
> 2008-12-19 16:16:34,092 INFO org.mortbay.util.Container: Started org.mortbay.jetty.servlet.WebApplicationHandler@17b0998
> 2008-12-19 16:16:34,092 INFO org.mortbay.util.Container: Started WebApplicationContext[/,/]
> 2008-12-19 16:16:34,155 WARN org.mortbay.util.ThreadedServer: Failed to start: SocketListener0@0.0.0.0:50060
> 2008-12-19 16:16:34,155 ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task tracker because java.net.BindException: Address already in use: JVM_Bind
> at java.net.PlainSocketImpl.socketBind(Native Method)
> at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
> at java.net.ServerSocket.bind(ServerSocket.java:319)
> at java.net.ServerSocket.<init>(ServerSocket.java:185)
> at org.mortbay.util.ThreadedServer.newServerSocket(ThreadedServer.java:391)
> at org.mortbay.util.ThreadedServer.open(ThreadedServer.java:477)
> at org.mortbay.util.ThreadedServer.start(ThreadedServer.java:503)
> at org.mortbay.http.SocketListener.start(SocketListener.java:203)
> at org.mortbay.http.HttpServer.doStart(HttpServer.java:761)
> at org.mortbay.util.Container.start(Container.java:72)
> at org.apache.hadoop.http.HttpServer.start(HttpServer.java:321)
> at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:894)
> at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2698)
> 2008-12-19 16:16:34,155 INFO org.apache.hadoop.mapred.TaskTracker: SHUTDOWN_MSG:
> /************************************************************
> SHUTDOWN_MSG: Shutting down TaskTracker at msra-5lcd05/172.23.213.80
> ************************************************************/
>
> Then I use "netstat -an", but port 50060 isn't in the list. "ps -af" also show that no program using 50060. The strange point is that when I repeat "bin/start-mapred.sh" and "bin/stop-mapred.sh" several times, the machines list that could start TaskTracker seems randomly.
>
> Could anybody help me solve this problem?
>