You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by ShiYU Gao <gi...@gmail.com> on 2011/01/16 13:37:07 UTC
I can't use hod to allocate the cluster
Hi ,
I met a strange problem when I used hod to deploying hadoop cluster on
three virtual machines.The torque was running correctly and the pbs_sever
was running on a physical machine, pbs_mom was running on the three virtual
machines. After I executed the " bin/hod allocate -d /home/bjtu/hadoop -n
3" command the program only printed the message " INFO - Cluster Id
142.bjtu1" and paused in there.
then I checked the ringmaster's log I got this :
***********************************************************************************************
[2011-01-16 17:12:33,164] DEBUG/10 (unknown file):0 - Getting service
ID.
[2011-01-16 17:12:33,531] DEBUG/10 (unknown file):0 - Got service ID:
141.bjtu1
[2011-01-16 17:12:33,633] DEBUG/10 (unknown file):0 - Service registry @
http://bjtu1:37181
[2011-01-16 17:12:33,738] DEBUG/10 (unknown file):0 - Returning Hadoop
directory as: /home/bjtu/hadoop-0.18.3
[2011-01-16 17:12:33,826] DEBUG/10 (unknown file):0 - Executing command
/home/bjtu/hadoop-0.18.3/bin/hadoop version to find hadoop version
[2011-01-16 17:12:33,909] CRITICAL/50 (unknown file):0 - the cmd is
/home/bjtu/hadoop-0.18.3/bin/hadoop version
[2011-01-16 17:12:44,680] DEBUG/10 (unknown file):0 - Version from hadoop
command: Hadoop 0.18.3
[2011-01-16 17:12:44,815] DEBUG/10 (unknown file):0 - hdfs desc is @
<hodlib.Common.desc.ServiceDesc instance at 0x945814c>
[2011-01-16 17:12:44,917] DEBUG/10 (unknown file):0 - Using max-connect
value 30
[2011-01-16 17:12:45,004] INFO/20 (unknown file):0 - Twisted interface not
found. Using hodXMLRPCServer.
[2011-01-16 17:12:45,171] DEBUG/10 (unknown file):0 - Ringmaster RPC Server
at 33578
[2011-01-16 17:12:45,246] DEBUG/10 (unknown file):0 - Download not set.
[2011-01-16 17:12:45,315] DEBUG/10 (unknown file):0 - bjtu 141.bjtu1
bjtu-vm63 ringmaster hod
[2011-01-16 17:12:45,803] DEBUG/10 (unknown file):0 - Registered with
serivce registry: http://bjtu1:37181.
[2011-01-16 17:12:45,878] DEBUG/10 (unknown file):0 - Returning Hadoop
directory as: /home/bjtu/hadoop-0.18.3
[2011-01-16 17:12:45,948] DEBUG/10 (unknown file):0 -
hadoopdir=/home/bjtu/hadoop-0.18.3,
java-home=/usr/lib/jvm/java-6-sun-1.6.0.16
[2011-01-16 17:12:46,049] DEBUG/10 (unknown file):0 - Executing command
/home/bjtu/hadoop-0.18.3/bin/hadoop version to find hadoop version
[2011-01-16 17:12:46,125] CRITICAL/50 (unknown file):0 - the cmd is
/home/bjtu/hadoop-0.18.3/bin/hadoop version
[2011-01-16 17:12:46,838] DEBUG/10 (unknown file):0 - getServiceAddr name:
hdfs
[2011-01-16 17:12:46,977] DEBUG/10 (unknown file):0 - getServiceAddr
service: <hodlib.GridServices.hdfs.Hdfs instance at 0x9573cac>
[2011-01-16 17:12:47,121] DEBUG/10 (unknown file):0 - getServiceAddr addr
hdfs: not found
[2011-01-16 17:12:48,264] DEBUG/10 (unknown file):0 - getServiceAddr name:
hdfs
[2011-01-16 17:12:48,365] DEBUG/10 (unknown file):0 - getServiceAddr
service: <hodlib.GridServices.hdfs.Hdfs instance at 0x9573cac>
[2011-01-16 17:12:48,462] DEBUG/10 (unknown file):0 - getServiceAddr addr
hdfs: not found
[2011-01-16 17:12:49,617] DEBUG/10 (unknown file):0 - getServiceAddr name:
hdfs
[2011-01-16 17:12:49,829] DEBUG/10 (unknown file):0 - getServiceAddr
service: <hodlib.GridServices.hdfs.Hdfs instance at 0x9573cac>
[2011-01-16 17:12:49,996] DEBUG/10 (unknown file):0 - getServiceAddr addr
hdfs: not found
[2011-01-16 17:12:51,189] DEBUG/10 (unknown file):0 - getServiceAddr name:
hdfs
[2011-01-16 17:12:51,328] DEBUG/10 (unknown file):0 - getServiceAddr
service: <hodlib.GridServices.hdfs.Hdfs instance at 0x9573cac>
[2011-01-16 17:12:51,457] DEBUG/10 (unknown file):0 - getServiceAddr addr
hdfs: not found
[2011-01-16 17:12:52,661] DEBUG/10 (unknown file):0 - getServiceAddr name:
hdfs
[2011-01-16 17:12:52,871] DEBUG/10 (unknown file):0 - getServiceAddr
service: <hodlib.GridServices.hdfs.Hdfs instance at 0x9573cac>
[2011-01-16 17:12:53,082] DEBUG/10 (unknown file):0 - getServiceAddr addr
hdfs: not found
[2011-01-16 17:12:54,007] DEBUG/10 (unknown file):0 - Version from hadoop
command: Hadoop 0.18.3
[2011-01-16 17:12:54,086] DEBUG/10 (unknown file):0 - starting jt monitor
[2011-01-16 17:12:54,123] DEBUG/10 (unknown file):0 - getServiceAddr name:
mapred
[2011-01-16 17:12:54,227] DEBUG/10 (unknown file):0 - getServiceAddr
service: <hodlib.GridServices.mapred.MapReduce instance at 0x9573d0c>
[2011-01-16 17:12:54,355] DEBUG/10 (unknown file):0 - Entered start method.
[2011-01-16 17:12:54,385] DEBUG/10 (unknown file):0 - getServiceAddr name:
hdfs
[2011-01-16 17:12:54,455] DEBUG/10 (unknown file):0 - getServiceAddr addr
mapred: not found
[2011-01-16 17:12:54,524] DEBUG/10 (unknown file):0 - getServiceAddr
service: <hodlib.GridServices.hdfs.Hdfs instance at 0x9573cac>
[2011-01-16 17:12:54,565] DEBUG/10 (unknown file):0 -
/home/bjtu/hod/bin/hodring --hodring.tarball-retry-initial-time 1.0
--hodring.cmd-retry-initial-time 2.0 --hodring.cmd-retry-interval 2.0
--hodring.service-id 141.bjtu1 --hodring.temp-dir /tmp/hod
--hodring.http-port-range 8000-9000 --hodring.userid bjtu
--hodring.java-home /usr/lib/jvm/java-6-sun-1.6.0.16 --hodring.svcrgy-addr
bjtu1:37181 --hodring.tarball-retry-interval 3.0 --hodring.log-dir
/home/bjtu/hodring/log --hodring.mapred-system-dir-root /mapredsystem
--hodring.xrs-port-range 32768-65536 --hodring.debug 4
--hodring.ringmaster-xrs-addr bjtu-vm63:33578 --hodring.register
[2011-01-16 17:12:54,656] DEBUG/10 (unknown file):0 - getServiceAddr addr
hdfs: not found
[2011-01-16 17:12:54,720] DEBUG/10 (unknown file):0 - pbsdsh command:
/usr/local/bin/pbsdsh /home/bjtu/hod/bin/hodring
--hodring.tarball-retry-initial-time 1.0 --hodring.cmd-retry-initial-time
2.0 --hodring.cmd-retry-interval 2.0 --hodring.service-id 141.bjtu1
--hodring.temp-dir /tmp/hod --hodring.http-port-range 8000-9000
--hodring.userid bjtu --hodring.java-home /usr/lib/jvm/java-6-sun-1.6.0.16
--hodring.svcrgy-addr bjtu1:37181 --hodring.tarball-retry-interval 3.0
--hodring.log-dir /home/bjtu/hodring/log --hodring.mapred-system-dir-root
/mapredsystem --hodring.xrs-port-range 32768-65536 --hodring.debug 4
--hodring.ringmaster-xrs-addr bjtu-vm63:33578 --hodring.register
[2011-01-16 17:12:55,040] DEBUG/10 (unknown file):0 - Returned from
runWorkers.
[2011-01-16 17:12:55,835] DEBUG/10 (unknown file):0 - getServiceAddr name:
hdfs
[2011-01-16 17:12:55,965] DEBUG/10 (unknown file):0 - getServiceAddr
service: <hodlib.GridServices.hdfs.Hdfs instance at 0x9573cac>
[2011-01-16 17:12:56,076] DEBUG/10 (unknown file):0 - getServiceAddr addr
hdfs: not found
[2011-01-16 17:12:57,237] DEBUG/10 (unknown file):0 - getServiceAddr name:
hdfs
[2011-01-16 17:12:57,350] DEBUG/10 (unknown file):0 - getServiceAddr
service: <hodlib.GridServices.hdfs.Hdfs instance at 0x9573cac>
[2011-01-16 17:12:57,486] DEBUG/10 (unknown file):0 - getServiceAddr addr
hdfs: not found
[2011-01-16 17:12:58,637] DEBUG/10 (unknown file):0 - getServiceAddr name:
hdfs
[2011-01-16 17:12:58,783] DEBUG/10 (unknown file):0 - getServiceAddr
service: <hodlib.GridServices.hdfs.Hdfs instance at 0x9573cac>
[2011-01-16 17:12:58,900] DEBUG/10 (unknown file):0 - getServiceAddr addr
hdfs: not found
[2011-01-16 17:13:00,079] DEBUG/10 (unknown file):0 - getServiceAddr name:
hdfs
[2011-01-16 17:13:00,211] DEBUG/10 (unknown file):0 - getServiceAddr
service: <hodlib.GridServices.hdfs.Hdfs instance at 0x9573cac>
[2011-01-16 17:13:00,320] DEBUG/10 (unknown file):0 - getServiceAddr addr
hdfs: not found
[2011-01-16 17:13:01,476] DEBUG/10 (unknown file):0 - getServiceAddr name:
hdfs
[2011-01-16 17:13:01,615] DEBUG/10 (unknown file):0 - getServiceAddr
service: <hodlib.GridServices.hdfs.Hdfs instance at 0x9573cac>
[2011-01-16 17:13:01,749] DEBUG/10 (unknown file):0 - getServiceAddr addr
hdfs: not found
[2011-01-16 17:13:02,915] DEBUG/10 (unknown file):0 - getServiceAddr name:
hdfs
[2011-01-16 17:13:03,039] DEBUG/10 (unknown file):0 - getServiceAddr
service: <hodlib.GridServices.hdfs.Hdfs instance at 0x9573cac>
[2011-01-16 17:13:03,175] DEBUG/10 (unknown file):0 - getServiceAddr addr
hdfs: not found
[2011-01-16 17:13:04,370] DEBUG/10 (unknown file):0 - getServiceAddr name:
hdfs
[2011-01-16 17:13:04,518] DEBUG/10 (unknown file):0 - getServiceAddr
service: <hodlib.GridServices.hdfs.Hdfs instance at 0x9573cac>
[2011-01-16 17:13:04,547] DEBUG/10 (unknown file):0 - getServiceAddr name:
mapred
[2011-01-16 17:13:04,685] DEBUG/10 (unknown file):0 - getServiceAddr
service: <hodlib.GridServices.mapred.MapReduce instance at 0x9573d0c>
[2011-01-16 17:13:04,721] DEBUG/10 (unknown file):0 - getServiceAddr addr
hdfs: not found
[2011-01-16 17:13:04,756] DEBUG/10 (unknown file):0 - getServiceAddr addr
mapred: not found
[2011-01-16 17:13:06,006] DEBUG/10 (unknown file):0 - getServiceAddr name:
hdfs
[2011-01-16 17:13:06,206] DEBUG/10 (unknown file):0 - getServiceAddr
service: <hodlib.GridServices.hdfs.Hdfs instance at 0x9573cac>
[2011-01-16 17:13:06,445] DEBUG/10 (unknown file):0 - getServiceAddr addr
hdfs: not found
[2011-01-16 17:13:06,769] DEBUG/10 (unknown file):0 - RingMaster stop method
invoked.
[2011-01-16 17:13:07,114] DEBUG/10 (unknown file):0 - finding exit code
[2011-01-16 17:13:07,358] DEBUG/10 (unknown file):0 - getServiceAddr name:
hdfs
[2011-01-16 17:13:07,796] DEBUG/10 (unknown file):0 - getServiceAddr
service: <hodlib.GridServices.hdfs.Hdfs instance at 0x9573cac>
[2011-01-16 17:13:07,913] DEBUG/10 (unknown file):0 - getServiceAddr name:
hdfs
[2011-01-16 17:13:08,191] DEBUG/10 (unknown file):0 - getServiceAddr
service: <hodlib.GridServices.hdfs.Hdfs instance at 0x9573cac>
[2011-01-16 17:13:08,255] DEBUG/10 (unknown file):0 - getServiceAddr addr
hdfs: not found
[2011-01-16 17:13:08,580] DEBUG/10 (unknown file):0 - getServiceAddr addr
hdfs: not found
[2011-01-16 17:13:08,710] DEBUG/10 (unknown file):0 - exit code 7
[2011-01-16 17:13:08,919] DEBUG/10 (unknown file):0 - getCommand returning
bjtu-vm37_22707[{'argv': ['namenode', '-format'],
'attrs': {},
'envs': {'HADOOP_ROOT_LOGGER': 'INFO,DRFA'},
'fg': 'true',
'final-attrs': {'dfs.data.dir':
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data,/tmp/hod/2/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data',
'dfs.http.address': 'fillinhostport',
'dfs.name.dir':
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-name',
'fs.default.name': 'fillinhostport',
'hadoop.tmp.dir':
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/hadoop-tmp'},
'name': 'namenode',
'pkgdirs': '/home/bjtu/hadoop-0.18.3',
'program': 'bin/hadoop',
'stdin': 'Y',
'workdirs': ['/tmp/hod/1/bjtu-vm63-16729-2628988258588339',
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn',
'/tmp/hod/2/bjtu-vm63-16729-2628988258588339',
'/tmp/hod/2/bjtu-vm63-16729-2628988258588339/hdfs-nn',
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-name',
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data',
'/tmp/hod/2/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data']},
{'argv': ['namenode'],
'attrs': {},
'envs': {'HADOOP_ROOT_LOGGER': 'INFO,DRFA'},
'final-attrs': {'dfs.data.dir':
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data,/tmp/hod/2/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data',
'dfs.http.address': 'fillinhostport',
'dfs.name.dir':
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-name',
'fs.default.name': 'fillinhostport',
'hadoop.tmp.dir':
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/hadoop-tmp'},
'name': 'namenode',
'pkgdirs': '/home/bjtu/hadoop-0.18.3',
'program': 'bin/hadoop',
'workdirs': ['/tmp/hod/1/bjtu-vm63-16729-2628988258588339',
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn',
'/tmp/hod/2/bjtu-vm63-16729-2628988258588339',
'/tmp/hod/2/bjtu-vm63-16729-2628988258588339/hdfs-nn',
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-name',
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data',
'/tmp/hod/2/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data']}]
[2011-01-16 17:13:09,009] DEBUG/10 (unknown file):0 - stopping ringmaster
instance
[2011-01-16 17:13:09,449] DEBUG/10 (unknown file):0 - getCommand returning
bjtu-vm58_16171[]
[2011-01-16 17:13:09,521] DEBUG/10 (unknown file):0 - Joining the monitoring
thread.
[2011-01-16 17:13:09,655] DEBUG/10 (unknown file):0 - Joined the monitoring
thread.
[2011-01-16 17:13:09,786] DEBUG/10 (unknown file):0 - Cleaned up temporary
dir: /tmp/hod/bjtu.141.bjtu1.ringmaster
[2011-01-16 17:13:09,913] DEBUG/10 (unknown file):0 - RingMaster stop method
invoked.
[2011-01-16 17:13:09,990] DEBUG/10 (unknown file):0 - RingMaster stop method
invoked.
[2011-01-16 17:13:10,059] DEBUG/10 (unknown file):0 - returning from main
*************************************************************************************************
But in the one of the hodring's log i got this:
*********************************************************************************************************
[2011-01-16 17:13:08,814] INFO/20 (unknown file):0 - Starting HOD service:
hodring ...
[2011-01-16 17:13:09,113] DEBUG/10 (unknown file):0 - Ringmaster at
http://bjtu-vm63:33578/
[2011-01-16 17:13:09,141] DEBUG/10 (unknown file):0 - Creating service
registry XML-RPC client.
[2011-01-16 17:13:09,184] DEBUG/10 (unknown file):0 - Creating ringmaster
XML-RPC client.
[2011-01-16 17:13:09,211] DEBUG/10 (unknown file):0 - Did not find a
download address.
[2011-01-16 17:13:09,540] DEBUG/10 (unknown file):0 - Did not get command
list. Waiting for 5.52228860894 seconds.
[2011-01-16 17:13:15,099] DEBUG/10 (unknown file):0 - the increment is 0
[2011-01-16 17:13:15,704] INFO/20 (unknown file):0 - Caught signal 15.
[2011-01-16 17:13:15,744] DEBUG/10 (unknown file):0 - Entered hodring stop.
[2011-01-16 17:13:15,782] DEBUG/10 (unknown file):0 - call hodsvcrgy stop...
[2011-01-16 17:13:15,821] INFO/20 (unknown file):0 - Stopping service...
****************************************************************
But in another hording machine the log is like this:
[2011-01-16 17:12:52,564] INFO/20 (unknown file):0 - Starting HOD service:
hodring ...
[2011-01-16 17:12:52,832] DEBUG/10 (unknown file):0 - Ringmaster at
http://bjtu-vm63:33578/
[2011-01-16 17:12:52,856] DEBUG/10 (unknown file):0 - Creating service
registry XML-RPC client.
[2011-01-16 17:12:52,893] DEBUG/10 (unknown file):0 - Creating ringmaster
XML-RPC client.
[2011-01-16 17:12:52,921] DEBUG/10 (unknown file):0 - Did not find a
download address.
[2011-01-16 17:12:55,279] DEBUG/10 (unknown file):0 - [{'dict': {'argv':
['namenode', '-format'],
'attrs': {},
'envs': {'HADOOP_ROOT_LOGGER': 'INFO,DRFA'},
'fg': 'true',
'final-attrs': {'dfs.data.dir':
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data,/tmp/hod/2/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data',
'dfs.http.address': 'fillinhostport',
'dfs.name.dir':
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-name',
'fs.default.name': 'fillinhostport',
'hadoop.tmp.dir':
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/hadoop-tmp'},
'name': 'namenode',
'pkgdirs': '/home/bjtu/hadoop-0.18.3',
'program': 'bin/hadoop',
'stdin': 'Y',
'workdirs': ['/tmp/hod/1/bjtu-vm63-16729-2628988258588339',
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn',
'/tmp/hod/2/bjtu-vm63-16729-2628988258588339',
'/tmp/hod/2/bjtu-vm63-16729-2628988258588339/hdfs-nn',
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-name',
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data',
'/tmp/hod/2/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data']}},
{'dict': {'argv': ['namenode'],
'attrs': {},
'envs': {'HADOOP_ROOT_LOGGER': 'INFO,DRFA'},
'final-attrs': {'dfs.data.dir':
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data,/tmp/hod/2/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data',
'dfs.http.address': 'fillinhostport',
'dfs.name.dir':
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-name',
'fs.default.name': 'fillinhostport',
'hadoop.tmp.dir':
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/hadoop-tmp'},
'name': 'namenode',
'pkgdirs': '/home/bjtu/hadoop-0.18.3',
'program': 'bin/hadoop',
'workdirs': ['/tmp/hod/1/bjtu-vm63-16729-2628988258588339',
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn',
'/tmp/hod/2/bjtu-vm63-16729-2628988258588339',
'/tmp/hod/2/bjtu-vm63-16729-2628988258588339/hdfs-nn',
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-name',
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data',
'/tmp/hod/2/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data']}}]
[2011-01-16 17:12:55,304] DEBUG/10 (unknown file):0 - In command desc
[2011-01-16 17:12:55,329] DEBUG/10 (unknown file):0 - Done in command desc
[2011-01-16 17:12:55,354] DEBUG/10 (unknown file):0 - Printing dict
[2011-01-16 17:12:55,378] DEBUG/10 (unknown file):0 - In command desc
[2011-01-16 17:12:55,403] DEBUG/10 (unknown file):0 - Done in command desc
[2011-01-16 17:12:55,428] DEBUG/10 (unknown file):0 - Printing dict
[2011-01-16 17:12:55,450] INFO/20 (unknown file):0 - Running hadoop
commands...
[2011-01-16 17:12:55,552] DEBUG/10 (unknown file):0 - {'argv': ['namenode',
'-format'],
'attrs': {},
'envs': {'HADOOP_ROOT_LOGGER': 'INFO,DRFA'},
'fg': 'true',
'final-attrs': {'dfs.data.dir':
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data,/tmp/hod/2/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data',
'dfs.http.address': 'fillinhostport',
'dfs.name.dir':
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-name',
'fs.default.name': 'fillinhostport',
'hadoop.tmp.dir':
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/hadoop-tmp'},
'ignorefailures': False,
'name': 'namenode',
'pkgdirs': '/home/bjtu/hadoop-0.18.3',
'program': 'bin/hadoop',
'stdin': 'Y',
'version': None,
'workdirs': ['/tmp/hod/1/bjtu-vm63-16729-2628988258588339',
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn',
'/tmp/hod/2/bjtu-vm63-16729-2628988258588339',
'/tmp/hod/2/bjtu-vm63-16729-2628988258588339/hdfs-nn',
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-name',
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data',
'/tmp/hod/2/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data']}
[2011-01-16 17:12:55,578] DEBUG/10 (unknown file):0 - mrsysdir is
/mapredsystem/bjtu/mapredsystem/141.bjtu1
[2011-01-16 17:12:55,648] DEBUG/10 (unknown file):0 - _createHadoopSiteXml:
hadoop.tmp.dir
/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/hadoop-tmp
[2011-01-16 17:12:55,683] DEBUG/10 (unknown file):0 - _createHadoopSiteXml:
fs.default.name fillinhostport
[2011-01-16 17:12:55,721] DEBUG/10 (unknown file):0 - Trying to see if port
53357 is available
[2011-01-16 17:12:55,759] DEBUG/10 (unknown file):0 - Yes, port 53357 is
available
[2011-01-16 17:12:55,796] DEBUG/10 (unknown file):0 - Setting hostname to:
bjtu-vm37
[2011-01-16 17:12:55,836] DEBUG/10 (unknown file):0 - _createHadoopSiteXml:
dfs.http.address fillinhostport
[2011-01-16 17:12:55,871] DEBUG/10 (unknown file):0 - Trying to see if port
50386 is available
[2011-01-16 17:12:55,911] DEBUG/10 (unknown file):0 - Yes, port 50386 is
available
[2011-01-16 17:12:55,950] DEBUG/10 (unknown file):0 - Setting hostname to:
bjtu-vm37
[2011-01-16 17:12:55,989] DEBUG/10 (unknown file):0 - _createHadoopSiteXml:
dfs.data.dir
/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data,/tmp/hod/2/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data
[2011-01-16 17:12:56,023] DEBUG/10 (unknown file):0 - _createHadoopSiteXml:
dfs.name.dir /tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-name
[2011-01-16 17:12:56,125] DEBUG/10 (unknown file):0 - created
/tmp/hod/bjtu.141.bjtu1.hodring/0-namenode/confdir/hadoop-site.xml
[2011-01-16 17:12:56,153] DEBUG/10 (unknown file):0 - hadoop log directory:
['/tmp/hod/bjtu.141.bjtu1.hodring/0-namenode/logdir']
[2011-01-16 17:12:56,178] DEBUG/10 (unknown file):0 - This is the packcage
dir /home/bjtu/hadoop-0.18.3
[2011-01-16 17:12:56,289] DEBUG/10 (unknown file):0 - {'argv': ['namenode',
'-format'],
'attrs': {},
'envs': {'HADOOP_ROOT_LOGGER': 'INFO,DRFA'},
'fg': 'true',
'final-attrs': {'dfs.data.dir':
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data,/tmp/hod/2/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data',
'dfs.http.address': 'fillinhostport',
'dfs.name.dir':
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-name',
'fs.default.name': 'fillinhostport',
'hadoop.tmp.dir':
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/hadoop-tmp'},
'ignorefailures': False,
'name': 'namenode',
'pkgdirs': '/home/bjtu/hadoop-0.18.3',
'program': 'bin/hadoop',
'stdin': 'Y',
'version': None,
'workdirs': ['/tmp/hod/1/bjtu-vm63-16729-2628988258588339',
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn',
'/tmp/hod/2/bjtu-vm63-16729-2628988258588339',
'/tmp/hod/2/bjtu-vm63-16729-2628988258588339/hdfs-nn',
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-name',
'/tmp/hod/1/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data',
'/tmp/hod/2/bjtu-vm63-16729-2628988258588339/hdfs-nn/dfs-data']}
[2011-01-16 17:12:56,317] DEBUG/10 (unknown file):0 - Got package dir of
/home/bjtu/hadoop-0.18.3
[2011-01-16 17:12:56,343] DEBUG/10 (unknown file):0 - path:
/home/bjtu/hadoop-0.18.3/bin/hadoop
[2011-01-16 17:12:56,373] INFO/20 (unknown file):0 - {'PBS_O_HOME':
'/home/bjtu', 'PBS_NODENUM': '2', 'ENVIRONMENT': 'BATCH', 'PBS_MOMPORT':
'15003', 'PBS_O_LANG': 'zh_CN.UTF-8', 'HOME': '/home/bjtu', 'PATH':
'/bin:/usr/bin', 'LANG': 'C', 'PBS_ENVIRONMENT': 'PBS_BATCH', 'PBS_VERSION':
'TORQUE-2.5.4', 'HADOOP_ROOT_LOGGER': 'INFO,DRFA', 'PBS_JOBID': '141.bjtu1',
'PBS_O_PATH':
'/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games',
'HADOOP_LOG_DIR': '/tmp/hod/bjtu.141.bjtu1.hodring/0-namenode/logdir',
'PBS_TASKNUM': '4', 'PBS_NUM_NODES': '3', 'PBS_O_INITDIR': '/tmp/',
'PBS_VNODENUM': '2', 'PBS_JOBCOOKIE': 'FA80AE910942DD112660B913626DDFFD',
'PBS_O_HOST': 'bjtu1', 'JAVA_HOME': '/usr/lib/jvm/java-6-sun-1.6.0.16',
'PBS_JOBNAME': 'HOD', 'PBS_O_WORKDIR': '/tmp', 'HADOOP_CONF_DIR':
'/tmp/hod/bjtu.141.bjtu1.hodring/0-namenode/confdir', 'PBS_O_MAIL':
'/var/mail/bjtu', 'PBS_NUM_PPN': '1', 'PBS_O_LOGNAME': 'bjtu',
'PBS_O_SHELL': '/bin/bash', 'OLDPWD': '/tmp', 'PBS_QUEUE': 'batch',
'PBS_O_QUEUE': 'batch', 'PWD': '/home/bjtu/hod/bin', 'PBS_SERVER': 'bjtu1'}
[2011-01-16 17:12:56,402] DEBUG/10 (unknown file):0 - running command:
/home/bjtu/hadoop-0.18.3/bin/hadoop namenode -format
1>/tmp/hod/bjtu.141.bjtu1.hodring/0-namenode/logdir/namenode.out
2>/tmp/hod/bjtu.141.bjtu1.hodring/0-namenode/logdir/namenode.err
[2011-01-16 17:12:56,430] DEBUG/10 (unknown file):0 - hadoop env:
{'PBS_O_HOME': '/home/bjtu', 'PBS_NODENUM': '2', 'ENVIRONMENT': 'BATCH',
'PBS_MOMPORT': '15003', 'PBS_O_LANG': 'zh_CN.UTF-8', 'HOME': '/home/bjtu',
'PATH': '/bin:/usr/bin', 'LANG': 'C', 'PBS_ENVIRONMENT': 'PBS_BATCH',
'PBS_VERSION': 'TORQUE-2.5.4', 'HADOOP_ROOT_LOGGER': 'INFO,DRFA',
'PBS_JOBID': '141.bjtu1', 'PBS_O_PATH':
'/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games',
'HADOOP_LOG_DIR': '/tmp/hod/bjtu.141.bjtu1.hodring/0-namenode/logdir',
'PBS_TASKNUM': '4', 'PBS_NUM_NODES': '3', 'PBS_O_INITDIR': '/tmp/',
'PBS_VNODENUM': '2', 'PBS_JOBCOOKIE': 'FA80AE910942DD112660B913626DDFFD',
'PBS_O_HOST': 'bjtu1', 'JAVA_HOME': '/usr/lib/jvm/java-6-sun-1.6.0.16',
'PBS_JOBNAME': 'HOD', 'PBS_O_WORKDIR': '/tmp', 'HADOOP_CONF_DIR':
'/tmp/hod/bjtu.141.bjtu1.hodring/0-namenode/confdir', 'PBS_O_MAIL':
'/var/mail/bjtu', 'PBS_NUM_PPN': '1', 'PBS_O_LOGNAME': 'bjtu',
'PBS_O_SHELL': '/bin/bash', 'OLDPWD': '/tmp', 'PBS_QUEUE': 'batch',
'PBS_O_QUEUE': 'batch', 'PWD': '/home/bjtu/hod/bin', 'PBS_SERVER': 'bjtu1'}
[2011-01-16 17:12:56,457] DEBUG/10 (unknown file):0 - Command stdout will be
redirected to /tmp/hod/bjtu.141.bjtu1.hodring/0-namenode/logdir/namenode.out
and command stderr to
/tmp/hod/bjtu.141.bjtu1.hodring/0-namenode/logdir/namenode.err
[2011-01-16 17:12:56,756] DEBUG/10 (unknown file):0 - hadoopThread still ==
None ...
[2011-01-16 17:12:56,832] DEBUG/10 (unknown file):0 - hadoop input: Y
[2011-01-16 17:12:56,913] DEBUG/10 (unknown file):0 - isForground: true
[2011-01-16 17:12:57,013] DEBUG/10 (unknown file):0 - Waiting on hadoop to
finish...
********************************************************************************************************************
I don't know why , please help me.
thanks