[jira] Created: (HADOOP-3179) if hadoop.tmp.dir is under your dfs.data.dir, HDFS will silently wipe out your "name" directory

["Brian Karlak (JIRA)" <ji...@apache.org>]

if hadoop.tmp.dir is under your dfs.data.dir, HDFS will silently wipe out your "name" directory
-----------------------------------------------------------------------------------------------

                 Key: HADOOP-3179
                 URL: https://issues.apache.org/jira/browse/HADOOP-3179
             Project: Hadoop Core
          Issue Type: Bug
          Components: dfs
    Affects Versions: 0.16.2
         Environment: gentoo linux on Intel/Dell w/ Sun JDK
            Reporter: Brian Karlak


I used a hadoop-site.xml conf file like:

  <property>
    <name>dfs.data.dir</name>
    <value>/data01/hadoop</value>
    <description>Dirs to store data on.</description>
  </property>
  <property>
    <name>hadoop.tmp.dir</name>
    <value>/data01/hadoop/tmp</value>
    <description>A base for other temporary directories.</description>
  </property>

This file will format the namenode properly.  Upon startup with the bin/start-dfs.sh script, however, the /data01/hadoop/tmp/dfs/name directory is silently wiped out.  This foobars the namenode, but only after the next DFS stop/start cycle.  (see output below)

This is obviously a configuration error first and foremost, but the fact that hadoop silently corrupts itself makes it tricky to track down.

[hid191]$ bin/hadoop namenode -format
08/04/04 18:41:43 INFO dfs.NameNode: STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = hid191.dev01.corp.metaweb.com/127.0.0.1
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 0.16.2
STARTUP_MSG:   build = http://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.16 -r 642481; compiled by 'hadoopqa' on Sat Mar 29 01:59:04 UTC 2008
************************************************************/
08/04/04 18:41:43 INFO fs.FSNamesystem: fsOwner=zenkat,users
08/04/04 18:41:43 INFO fs.FSNamesystem: supergroup=supergroup
08/04/04 18:41:43 INFO fs.FSNamesystem: isPermissionEnabled=true
08/04/04 18:41:43 INFO dfs.Storage: Storage directory /data01/hadoop/tmp/dfs/name has been successfully formatted.
08/04/04 18:41:43 INFO dfs.NameNode: SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at hid191.dev01.corp.metaweb.com/127.0.0.1
************************************************************/
[hid191]$ ls /data01/hadoop/tmp/dfs/name
current  image
[hid191]$ bin/start-dfs.sh 
starting namenode, logging to /data01/hadoop/logs/hadoop-zenkat-namenode-hid191.out
localhost: starting datanode, logging to /data01/hadoop/logs/hadoop-zenkat-datanode-hid191.out
localhost: starting secondarynamenode, logging to /data01/hadoop/logs/hadoop-zenkat-secondarynamenode-hid191.out
[hid191]$ ls /data01/hadoop/tmp/dfs/name
ls: cannot access /data01/hadoop/tmp/dfs/name: No such file or directory




-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.