You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Nguyen Thi Ngoc Huong <hu...@gmail.com> on 2009/08/21 03:33:56 UTC
Location of HBase's database (database' s files) on the hard disk
Hi all,
I am a beginner to HBase. I have some question with Hbase after setup Hbase
and Hadoop.
The first, After setup Hbase and create a new database, I don't know where
is location of HBase's database (database' s files) on the hard disk. At the
first, I think it is on the hbase.rootdir directory, however, when I delete
directory hbase.rootdir, and type the command "list", all of database
exist.
The second, after restart computer and restart hbase, all database of HBase
is lost. Is it always true? Or did I configure wrong? How can i configure
Hbase to save database after restart computer?
--
Nguyễn Thị Ngọc Hương
Re: Location of HBase's database (database' s files) on the hard disk
Posted by Amandeep Khurana <am...@gmail.com>.
Data in hbase tables resides in the memory till it gets flushed to
persistent storage. Anything that was in the memory would probably be lost
unless you are using the write ahead logs (which you would be using by
default in 0.20 if you dont disable it in the puts).
Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz
On Mon, Aug 24, 2009 at 9:43 PM, Nguyen Thi Ngoc Huong
<hu...@gmail.com>wrote:
> How can I restore database when I wasn't stop Hbase and Hadoop when
> restarting my computer?
>
> 2009/8/22 Nguyen Thi Ngoc Huong <hu...@gmail.com>
>
> > ok. This is my error. I wasn't stop Hbase and Hadoop when restarting my
> > computer.
> > Thank you very much.
> >
> > 2009/8/21 Jean-Daniel Cryans <jd...@apache.org>
> >
> > 2 questions:
> >>
> >> Which version of HBase are you using?
> >>
> >> Are you stopping HBase and Hadoop when restarting your computer?
> >>
> >> Thx,
> >>
> >> J-D
> >>
> >> On Fri, Aug 21, 2009 at 5:46 AM, Nguyen Thi Ngoc
> >> Huong<hu...@gmail.com> wrote:
> >> > Thank you very much.
> >> > I deleted everything and configured hadoop.tmp.dir property in
> >> > hadoop-site.xml as follow
> >> > <property>
> >> > <name>hadoop.tmp.dir</name>
> >> > <value>/home/huongntn/hadoop-${user.name}</value>
> >> > <description>A base for other temporary directories.</description>
> >> > </property>
> >> >
> >> > After that, I formated namenode and start-all. When I restarted my
> >> computer
> >> > and typed the command "start-all", hadoop work smoothly. I start hbase
> >> by
> >> > command "./bin/start-hbase.sh" and "./hbase shell"
> >> >
> >> > Now I can't see my database in hbase shell (by command "list")
> although
> >> I
> >> > can see it in Hadoop site manager,
> >> >
> >> >
> >> > 2009/8/21 Amandeep Khurana <am...@gmail.com>
> >> >
> >> >> 1. If you have formatted your namenode before starting the first
> time,
> >> >> thats
> >> >> all thats needed.
> >> >>
> >> >> 2. To start from scratch, delete everything thats there in the
> >> directory
> >> >> where you are pointing your hdfs to; format namenode again; start all
> >> >>
> >> >> 3. If it still doesnt work, look at the namenode logs to see whats
> >> >> happening. Post it here if you cant figure it out.
> >> >>
> >> >>
> >> >> Amandeep Khurana
> >> >> Computer Science Graduate Student
> >> >> University of California, Santa Cruz
> >> >>
> >> >>
> >> >> On Fri, Aug 21, 2009 at 1:30 AM, Nguyen Thi Ngoc Huong
> >> >> <hu...@gmail.com>wrote:
> >> >>
> >> >> > >You dont need to format the namenode everytime.. Just
> >> bin/start-all.sh
> >> >> >
> >> >> > Really? Just bin/start-all.sh, namnode is not started (when I type
> >> >> command
> >> >> > "jps", there are only 5 processes
> >> >> > 3421 SecondaryNameNode
> >> >> > 3492 JobTracker
> >> >> > 3582 TaskTracker
> >> >> > 4031 Jps
> >> >> > 3325 DataNode, there isn't Namenode process)
> >> >> > and certainly, the page http://localhost:50070 is died and
> >> connection
> >> >> from
> >> >> > Hbase to hadoop is died, too
> >> >> >
> >> >> >
> >> >> > 2009/8/21 Amandeep Khurana <am...@gmail.com>
> >> >> >
> >> >> > > On Fri, Aug 21, 2009 at 1:03 AM, Nguyen Thi Ngoc Huong
> >> >> > > <hu...@gmail.com>wrote:
> >> >> > >
> >> >> > > > Thanks you very much. I editted file hbase-site.xml as follow
> >> >> > > >
> >> >> > > > <property>
> >> >> > > > <name>hbase.rootdir</name>
> >> >> > > > <value>hdfs://localhost:54310/hbase</value>
> >> >> > > > <description>The directory shared by region servers.
> >> >> > > > Should be fully-qualified to include the filesystem to use.
> >> >> > > > E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
> >> >> > > > </description>
> >> >> > > > </property>
> >> >> > > >
> >> >> > > > with fs.default.name is hdfs://localhost:54310
> >> >> > > > Now, I can see hbase database in Hadoop site manager (in
> "hbase"
> >> >> > > directory
> >> >> > > > not "tmp" directory in hdfs ).
> >> >> > > > However, when I restart my computer, I must restart hadoop (by
> >> >> command
> >> >> > > > ./bin/hadoop format namenode and ./bin/start all) , restart
> >> hbase,
> >> >> and
> >> >> > my
> >> >> > > > database is lost. What can I do to save my database?
> >> >> > > >
> >> >> > >
> >> >> > > You dont need to format the namenode everytime.. Just
> >> bin/start-all.sh
> >> >> > >
> >> >> > >
> >> >> > >
> >> >> > > >
> >> >> > > > 2009/8/21 Amandeep Khurana <am...@gmail.com>
> >> >> > > >
> >> >> > > > > On Thu, Aug 20, 2009 at 11:46 PM, Nguyen Thi Ngoc Huong <
> >> >> > > > > huongngtn@gmail.com
> >> >> > > > > > wrote:
> >> >> > > > >
> >> >> > > > > > How can I configure the location of the hbase directory? I
> >> >> > configured
> >> >> > > > > > hbase-site.xml as follow:
> >> >> > > > > >
> >> >> > > > > > <property>
> >> >> > > > > > <name>hbase.rootdir</name>
> >> >> > > > > > <value>*file:///temp/hbase-${user.name}/hbase*</value>
> >> >> > > > > > <description>The directory shared by region servers.
> >> >> > > > > > Should be fully-qualified to include the filesystem to
> >> use.
> >> >> > > > > > E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
> >> >> > > > > > </description>
> >> >> > > > > > </property>
> >> >> > > > > >
> >> >> > > > >
> >> >> > > > > Thats the trouble.. Your data is being stored in the temp..
> >> instead
> >> >> > > store
> >> >> > > > > it
> >> >> > > > > in your hdfs.
> >> >> > > > > so the value of the above property would be something like
> >> >> > > > > *hdfs://namenodeserver:port/hbase*
> >> >> > > > >
> >> >> > > > >
> >> >> > > > >
> >> >> > > > > >
> >> >> > > > > > and the log file is
> >> >> > > > > > Not starting HMaster because:
> >> >> > > > > > java.io.IOException: Mkdirs failed to create
> >> >> > > > > > file:/temp/hbase-huongntn/hbase
> >> >> > > > > > at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
> >> >> > > > > > at
> >> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
> >> >> > > > > > at
> >> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
> >> >> > > > > > at
> >> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
> >> >> > > > > > at
> >> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
> >> >> > > > > > at
> >> >> > > org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
> >> >> > > > > > at
> >> >> > org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
> >> >> > > > > > at
> >> >> > org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
> >> >> > > > > > at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
> >> >> > > > > > at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
> >> >> > > > > > at
> >> >> > > org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
> >> >> > > > > > at
> >> >> > org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
> >> >> > > > > > 2009-08-21 13:35:24,163 ERROR
> >> >> > org.apache.hadoop.hbase.master.HMaster:
> >> >> > > > Can
> >> >> > > > > > not start master
> >> >> > > > > > java.io.IOException: Mkdirs failed to create
> >> >> > > > > > file:/temp/hbase-huongntn/hbase
> >> >> > > > > > at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
> >> >> > > > > > at
> >> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
> >> >> > > > > > at
> >> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
> >> >> > > > > > at
> >> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
> >> >> > > > > > at
> >> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
> >> >> > > > > > at
> >> >> > > org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
> >> >> > > > > > at
> >> >> > org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
> >> >> > > > > > at
> >> >> > org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
> >> >> > > > > > at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
> >> >> > > > > > at
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
> >> >> > > > > > at
> >> >> > > org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
> >> >> > > > > > at
> >> >> > org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > > 2009/8/21 Amandeep Khurana <am...@gmail.com>
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > > > You configure the location of the hbase directory in the
> >> >> > > > hbase-site.xml
> >> >> > > > > > >
> >> >> > > > > > > The data being lost could have multilple reasons. To rule
> >> out
> >> >> the
> >> >> > > > > > > basic one - where have you pointed the hdfs to store
> data?
> >> If
> >> >> its
> >> >> > > > > > > going into /tmp, you'll lose data everytime the tmp
> cleaner
> >> >> comes
> >> >> > > > into
> >> >> > > > > > > action.
> >> >> > > > > > >
> >> >> > > > > > > On 8/20/09, Nguyen Thi Ngoc Huong <hu...@gmail.com>
> >> wrote:
> >> >> > > > > > > > Hi all,
> >> >> > > > > > > > I am a beginner to HBase. I have some question with
> Hbase
> >> >> after
> >> >> > > > setup
> >> >> > > > > > > Hbase
> >> >> > > > > > > > and Hadoop.
> >> >> > > > > > > >
> >> >> > > > > > > > The first, After setup Hbase and create a new database,
> I
> >> >> don't
> >> >> > > > know
> >> >> > > > > > > where
> >> >> > > > > > > > is location of HBase's database (database' s files) on
> >> the
> >> >> hard
> >> >> > > > disk.
> >> >> > > > > > At
> >> >> > > > > > > the
> >> >> > > > > > > > first, I think it is on the hbase.rootdir directory,
> >> however,
> >> >> > > when
> >> >> > > > I
> >> >> > > > > > > delete
> >> >> > > > > > > > directory hbase.rootdir, and type the command "list",
> all
> >> of
> >> >> > > > database
> >> >> > > > > > > > exist.
> >> >> > > > > > > >
> >> >> > > > > > > > The second, after restart computer and restart hbase,
> all
> >> >> > > database
> >> >> > > > of
> >> >> > > > > > > HBase
> >> >> > > > > > > > is lost. Is it always true? Or did I configure wrong?
> How
> >> can
> >> >> i
> >> >> > > > > > configure
> >> >> > > > > > > > Hbase to save database after restart computer?
> >> >> > > > > > > >
> >> >> > > > > > > > --
> >> >> > > > > > > > Nguyễn Thị Ngọc Hương
> >> >> > > > > > > >
> >> >> > > > > > >
> >> >> > > > > > >
> >> >> > > > > > > --
> >> >> > > > > > >
> >> >> > > > > > >
> >> >> > > > > > > Amandeep Khurana
> >> >> > > > > > > Computer Science Graduate Student
> >> >> > > > > > > University of California, Santa Cruz
> >> >> > > > > > >
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > > --
> >> >> > > > > > Nguyễn Thị Ngọc Hương
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > > --
> >> >> > > > Nguyễn Thị Ngọc Hương
> >> >> > > >
> >> >> > >
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Nguyễn Thị Ngọc Hương
> >> >> >
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Nguyễn Thị Ngọc Hương
> >> >
> >>
> >
> >
> >
> > --
> > Nguyễn Thị Ngọc Hương
> >
>
>
>
> --
> Nguyễn Thị Ngọc Hương
>
Re: Location of HBase's database (database' s files) on the hard disk
Posted by Nguyen Thi Ngoc Huong <hu...@gmail.com>.
How can I restore database when I wasn't stop Hbase and Hadoop when
restarting my computer?
2009/8/22 Nguyen Thi Ngoc Huong <hu...@gmail.com>
> ok. This is my error. I wasn't stop Hbase and Hadoop when restarting my
> computer.
> Thank you very much.
>
> 2009/8/21 Jean-Daniel Cryans <jd...@apache.org>
>
> 2 questions:
>>
>> Which version of HBase are you using?
>>
>> Are you stopping HBase and Hadoop when restarting your computer?
>>
>> Thx,
>>
>> J-D
>>
>> On Fri, Aug 21, 2009 at 5:46 AM, Nguyen Thi Ngoc
>> Huong<hu...@gmail.com> wrote:
>> > Thank you very much.
>> > I deleted everything and configured hadoop.tmp.dir property in
>> > hadoop-site.xml as follow
>> > <property>
>> > <name>hadoop.tmp.dir</name>
>> > <value>/home/huongntn/hadoop-${user.name}</value>
>> > <description>A base for other temporary directories.</description>
>> > </property>
>> >
>> > After that, I formated namenode and start-all. When I restarted my
>> computer
>> > and typed the command "start-all", hadoop work smoothly. I start hbase
>> by
>> > command "./bin/start-hbase.sh" and "./hbase shell"
>> >
>> > Now I can't see my database in hbase shell (by command "list") although
>> I
>> > can see it in Hadoop site manager,
>> >
>> >
>> > 2009/8/21 Amandeep Khurana <am...@gmail.com>
>> >
>> >> 1. If you have formatted your namenode before starting the first time,
>> >> thats
>> >> all thats needed.
>> >>
>> >> 2. To start from scratch, delete everything thats there in the
>> directory
>> >> where you are pointing your hdfs to; format namenode again; start all
>> >>
>> >> 3. If it still doesnt work, look at the namenode logs to see whats
>> >> happening. Post it here if you cant figure it out.
>> >>
>> >>
>> >> Amandeep Khurana
>> >> Computer Science Graduate Student
>> >> University of California, Santa Cruz
>> >>
>> >>
>> >> On Fri, Aug 21, 2009 at 1:30 AM, Nguyen Thi Ngoc Huong
>> >> <hu...@gmail.com>wrote:
>> >>
>> >> > >You dont need to format the namenode everytime.. Just
>> bin/start-all.sh
>> >> >
>> >> > Really? Just bin/start-all.sh, namnode is not started (when I type
>> >> command
>> >> > "jps", there are only 5 processes
>> >> > 3421 SecondaryNameNode
>> >> > 3492 JobTracker
>> >> > 3582 TaskTracker
>> >> > 4031 Jps
>> >> > 3325 DataNode, there isn't Namenode process)
>> >> > and certainly, the page http://localhost:50070 is died and
>> connection
>> >> from
>> >> > Hbase to hadoop is died, too
>> >> >
>> >> >
>> >> > 2009/8/21 Amandeep Khurana <am...@gmail.com>
>> >> >
>> >> > > On Fri, Aug 21, 2009 at 1:03 AM, Nguyen Thi Ngoc Huong
>> >> > > <hu...@gmail.com>wrote:
>> >> > >
>> >> > > > Thanks you very much. I editted file hbase-site.xml as follow
>> >> > > >
>> >> > > > <property>
>> >> > > > <name>hbase.rootdir</name>
>> >> > > > <value>hdfs://localhost:54310/hbase</value>
>> >> > > > <description>The directory shared by region servers.
>> >> > > > Should be fully-qualified to include the filesystem to use.
>> >> > > > E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
>> >> > > > </description>
>> >> > > > </property>
>> >> > > >
>> >> > > > with fs.default.name is hdfs://localhost:54310
>> >> > > > Now, I can see hbase database in Hadoop site manager (in "hbase"
>> >> > > directory
>> >> > > > not "tmp" directory in hdfs ).
>> >> > > > However, when I restart my computer, I must restart hadoop (by
>> >> command
>> >> > > > ./bin/hadoop format namenode and ./bin/start all) , restart
>> hbase,
>> >> and
>> >> > my
>> >> > > > database is lost. What can I do to save my database?
>> >> > > >
>> >> > >
>> >> > > You dont need to format the namenode everytime.. Just
>> bin/start-all.sh
>> >> > >
>> >> > >
>> >> > >
>> >> > > >
>> >> > > > 2009/8/21 Amandeep Khurana <am...@gmail.com>
>> >> > > >
>> >> > > > > On Thu, Aug 20, 2009 at 11:46 PM, Nguyen Thi Ngoc Huong <
>> >> > > > > huongngtn@gmail.com
>> >> > > > > > wrote:
>> >> > > > >
>> >> > > > > > How can I configure the location of the hbase directory? I
>> >> > configured
>> >> > > > > > hbase-site.xml as follow:
>> >> > > > > >
>> >> > > > > > <property>
>> >> > > > > > <name>hbase.rootdir</name>
>> >> > > > > > <value>*file:///temp/hbase-${user.name}/hbase*</value>
>> >> > > > > > <description>The directory shared by region servers.
>> >> > > > > > Should be fully-qualified to include the filesystem to
>> use.
>> >> > > > > > E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
>> >> > > > > > </description>
>> >> > > > > > </property>
>> >> > > > > >
>> >> > > > >
>> >> > > > > Thats the trouble.. Your data is being stored in the temp..
>> instead
>> >> > > store
>> >> > > > > it
>> >> > > > > in your hdfs.
>> >> > > > > so the value of the above property would be something like
>> >> > > > > *hdfs://namenodeserver:port/hbase*
>> >> > > > >
>> >> > > > >
>> >> > > > >
>> >> > > > > >
>> >> > > > > > and the log file is
>> >> > > > > > Not starting HMaster because:
>> >> > > > > > java.io.IOException: Mkdirs failed to create
>> >> > > > > > file:/temp/hbase-huongntn/hbase
>> >> > > > > > at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
>> >> > > > > > at
>> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
>> >> > > > > > at
>> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
>> >> > > > > > at
>> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
>> >> > > > > > at
>> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
>> >> > > > > > at
>> >> > > org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
>> >> > > > > > at
>> >> > org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
>> >> > > > > > at
>> >> > org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
>> >> > > > > > at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
>> >> > > > > > at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
>> >> > > > > > at
>> >> > > org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
>> >> > > > > > at
>> >> > org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
>> >> > > > > > 2009-08-21 13:35:24,163 ERROR
>> >> > org.apache.hadoop.hbase.master.HMaster:
>> >> > > > Can
>> >> > > > > > not start master
>> >> > > > > > java.io.IOException: Mkdirs failed to create
>> >> > > > > > file:/temp/hbase-huongntn/hbase
>> >> > > > > > at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
>> >> > > > > > at
>> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
>> >> > > > > > at
>> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
>> >> > > > > > at
>> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
>> >> > > > > > at
>> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
>> >> > > > > > at
>> >> > > org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
>> >> > > > > > at
>> >> > org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
>> >> > > > > > at
>> >> > org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
>> >> > > > > > at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
>> >> > > > > > at
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
>> >> > > > > > at
>> >> > > org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
>> >> > > > > > at
>> >> > org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
>> >> > > > > >
>> >> > > > > >
>> >> > > > > > 2009/8/21 Amandeep Khurana <am...@gmail.com>
>> >> > > > > >
>> >> > > > > >
>> >> > > > > >
>> >> > > > > > > You configure the location of the hbase directory in the
>> >> > > > hbase-site.xml
>> >> > > > > > >
>> >> > > > > > > The data being lost could have multilple reasons. To rule
>> out
>> >> the
>> >> > > > > > > basic one - where have you pointed the hdfs to store data?
>> If
>> >> its
>> >> > > > > > > going into /tmp, you'll lose data everytime the tmp cleaner
>> >> comes
>> >> > > > into
>> >> > > > > > > action.
>> >> > > > > > >
>> >> > > > > > > On 8/20/09, Nguyen Thi Ngoc Huong <hu...@gmail.com>
>> wrote:
>> >> > > > > > > > Hi all,
>> >> > > > > > > > I am a beginner to HBase. I have some question with Hbase
>> >> after
>> >> > > > setup
>> >> > > > > > > Hbase
>> >> > > > > > > > and Hadoop.
>> >> > > > > > > >
>> >> > > > > > > > The first, After setup Hbase and create a new database, I
>> >> don't
>> >> > > > know
>> >> > > > > > > where
>> >> > > > > > > > is location of HBase's database (database' s files) on
>> the
>> >> hard
>> >> > > > disk.
>> >> > > > > > At
>> >> > > > > > > the
>> >> > > > > > > > first, I think it is on the hbase.rootdir directory,
>> however,
>> >> > > when
>> >> > > > I
>> >> > > > > > > delete
>> >> > > > > > > > directory hbase.rootdir, and type the command "list", all
>> of
>> >> > > > database
>> >> > > > > > > > exist.
>> >> > > > > > > >
>> >> > > > > > > > The second, after restart computer and restart hbase, all
>> >> > > database
>> >> > > > of
>> >> > > > > > > HBase
>> >> > > > > > > > is lost. Is it always true? Or did I configure wrong? How
>> can
>> >> i
>> >> > > > > > configure
>> >> > > > > > > > Hbase to save database after restart computer?
>> >> > > > > > > >
>> >> > > > > > > > --
>> >> > > > > > > > Nguyễn Thị Ngọc Hương
>> >> > > > > > > >
>> >> > > > > > >
>> >> > > > > > >
>> >> > > > > > > --
>> >> > > > > > >
>> >> > > > > > >
>> >> > > > > > > Amandeep Khurana
>> >> > > > > > > Computer Science Graduate Student
>> >> > > > > > > University of California, Santa Cruz
>> >> > > > > > >
>> >> > > > > >
>> >> > > > > >
>> >> > > > > >
>> >> > > > > > --
>> >> > > > > > Nguyễn Thị Ngọc Hương
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > > --
>> >> > > > Nguyễn Thị Ngọc Hương
>> >> > > >
>> >> > >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Nguyễn Thị Ngọc Hương
>> >> >
>> >>
>> >
>> >
>> >
>> > --
>> > Nguyễn Thị Ngọc Hương
>> >
>>
>
>
>
> --
> Nguyễn Thị Ngọc Hương
>
--
Nguyễn Thị Ngọc Hương
Re: Location of HBase's database (database' s files) on the hard disk
Posted by Jean-Daniel Cryans <jd...@apache.org>.
2 questions:
Which version of HBase are you using?
Are you stopping HBase and Hadoop when restarting your computer?
Thx,
J-D
On Fri, Aug 21, 2009 at 5:46 AM, Nguyen Thi Ngoc
Huong<hu...@gmail.com> wrote:
> Thank you very much.
> I deleted everything and configured hadoop.tmp.dir property in
> hadoop-site.xml as follow
> <property>
> <name>hadoop.tmp.dir</name>
> <value>/home/huongntn/hadoop-${user.name}</value>
> <description>A base for other temporary directories.</description>
> </property>
>
> After that, I formated namenode and start-all. When I restarted my computer
> and typed the command "start-all", hadoop work smoothly. I start hbase by
> command "./bin/start-hbase.sh" and "./hbase shell"
>
> Now I can't see my database in hbase shell (by command "list") although I
> can see it in Hadoop site manager,
>
>
> 2009/8/21 Amandeep Khurana <am...@gmail.com>
>
>> 1. If you have formatted your namenode before starting the first time,
>> thats
>> all thats needed.
>>
>> 2. To start from scratch, delete everything thats there in the directory
>> where you are pointing your hdfs to; format namenode again; start all
>>
>> 3. If it still doesnt work, look at the namenode logs to see whats
>> happening. Post it here if you cant figure it out.
>>
>>
>> Amandeep Khurana
>> Computer Science Graduate Student
>> University of California, Santa Cruz
>>
>>
>> On Fri, Aug 21, 2009 at 1:30 AM, Nguyen Thi Ngoc Huong
>> <hu...@gmail.com>wrote:
>>
>> > >You dont need to format the namenode everytime.. Just bin/start-all.sh
>> >
>> > Really? Just bin/start-all.sh, namnode is not started (when I type
>> command
>> > "jps", there are only 5 processes
>> > 3421 SecondaryNameNode
>> > 3492 JobTracker
>> > 3582 TaskTracker
>> > 4031 Jps
>> > 3325 DataNode, there isn't Namenode process)
>> > and certainly, the page http://localhost:50070 is died and connection
>> from
>> > Hbase to hadoop is died, too
>> >
>> >
>> > 2009/8/21 Amandeep Khurana <am...@gmail.com>
>> >
>> > > On Fri, Aug 21, 2009 at 1:03 AM, Nguyen Thi Ngoc Huong
>> > > <hu...@gmail.com>wrote:
>> > >
>> > > > Thanks you very much. I editted file hbase-site.xml as follow
>> > > >
>> > > > <property>
>> > > > <name>hbase.rootdir</name>
>> > > > <value>hdfs://localhost:54310/hbase</value>
>> > > > <description>The directory shared by region servers.
>> > > > Should be fully-qualified to include the filesystem to use.
>> > > > E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
>> > > > </description>
>> > > > </property>
>> > > >
>> > > > with fs.default.name is hdfs://localhost:54310
>> > > > Now, I can see hbase database in Hadoop site manager (in "hbase"
>> > > directory
>> > > > not "tmp" directory in hdfs ).
>> > > > However, when I restart my computer, I must restart hadoop (by
>> command
>> > > > ./bin/hadoop format namenode and ./bin/start all) , restart hbase,
>> and
>> > my
>> > > > database is lost. What can I do to save my database?
>> > > >
>> > >
>> > > You dont need to format the namenode everytime.. Just bin/start-all.sh
>> > >
>> > >
>> > >
>> > > >
>> > > > 2009/8/21 Amandeep Khurana <am...@gmail.com>
>> > > >
>> > > > > On Thu, Aug 20, 2009 at 11:46 PM, Nguyen Thi Ngoc Huong <
>> > > > > huongngtn@gmail.com
>> > > > > > wrote:
>> > > > >
>> > > > > > How can I configure the location of the hbase directory? I
>> > configured
>> > > > > > hbase-site.xml as follow:
>> > > > > >
>> > > > > > <property>
>> > > > > > <name>hbase.rootdir</name>
>> > > > > > <value>*file:///temp/hbase-${user.name}/hbase*</value>
>> > > > > > <description>The directory shared by region servers.
>> > > > > > Should be fully-qualified to include the filesystem to use.
>> > > > > > E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
>> > > > > > </description>
>> > > > > > </property>
>> > > > > >
>> > > > >
>> > > > > Thats the trouble.. Your data is being stored in the temp.. instead
>> > > store
>> > > > > it
>> > > > > in your hdfs.
>> > > > > so the value of the above property would be something like
>> > > > > *hdfs://namenodeserver:port/hbase*
>> > > > >
>> > > > >
>> > > > >
>> > > > > >
>> > > > > > and the log file is
>> > > > > > Not starting HMaster because:
>> > > > > > java.io.IOException: Mkdirs failed to create
>> > > > > > file:/temp/hbase-huongntn/hbase
>> > > > > > at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
>> > > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
>> > > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
>> > > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
>> > > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
>> > > > > > at
>> > > org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
>> > > > > > at
>> > org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
>> > > > > > at
>> > org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
>> > > > > > at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
>> > > > > > at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
>> > > > > > at
>> > > org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
>> > > > > > at
>> > org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
>> > > > > > 2009-08-21 13:35:24,163 ERROR
>> > org.apache.hadoop.hbase.master.HMaster:
>> > > > Can
>> > > > > > not start master
>> > > > > > java.io.IOException: Mkdirs failed to create
>> > > > > > file:/temp/hbase-huongntn/hbase
>> > > > > > at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
>> > > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
>> > > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
>> > > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
>> > > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
>> > > > > > at
>> > > org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
>> > > > > > at
>> > org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
>> > > > > > at
>> > org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
>> > > > > > at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
>> > > > > > at
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
>> > > > > > at
>> > > org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
>> > > > > > at
>> > org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
>> > > > > >
>> > > > > >
>> > > > > > 2009/8/21 Amandeep Khurana <am...@gmail.com>
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > > You configure the location of the hbase directory in the
>> > > > hbase-site.xml
>> > > > > > >
>> > > > > > > The data being lost could have multilple reasons. To rule out
>> the
>> > > > > > > basic one - where have you pointed the hdfs to store data? If
>> its
>> > > > > > > going into /tmp, you'll lose data everytime the tmp cleaner
>> comes
>> > > > into
>> > > > > > > action.
>> > > > > > >
>> > > > > > > On 8/20/09, Nguyen Thi Ngoc Huong <hu...@gmail.com> wrote:
>> > > > > > > > Hi all,
>> > > > > > > > I am a beginner to HBase. I have some question with Hbase
>> after
>> > > > setup
>> > > > > > > Hbase
>> > > > > > > > and Hadoop.
>> > > > > > > >
>> > > > > > > > The first, After setup Hbase and create a new database, I
>> don't
>> > > > know
>> > > > > > > where
>> > > > > > > > is location of HBase's database (database' s files) on the
>> hard
>> > > > disk.
>> > > > > > At
>> > > > > > > the
>> > > > > > > > first, I think it is on the hbase.rootdir directory, however,
>> > > when
>> > > > I
>> > > > > > > delete
>> > > > > > > > directory hbase.rootdir, and type the command "list", all of
>> > > > database
>> > > > > > > > exist.
>> > > > > > > >
>> > > > > > > > The second, after restart computer and restart hbase, all
>> > > database
>> > > > of
>> > > > > > > HBase
>> > > > > > > > is lost. Is it always true? Or did I configure wrong? How can
>> i
>> > > > > > configure
>> > > > > > > > Hbase to save database after restart computer?
>> > > > > > > >
>> > > > > > > > --
>> > > > > > > > Nguyễn Thị Ngọc Hương
>> > > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > --
>> > > > > > >
>> > > > > > >
>> > > > > > > Amandeep Khurana
>> > > > > > > Computer Science Graduate Student
>> > > > > > > University of California, Santa Cruz
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > --
>> > > > > > Nguyễn Thị Ngọc Hương
>> > > > > >
>> > > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > Nguyễn Thị Ngọc Hương
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Nguyễn Thị Ngọc Hương
>> >
>>
>
>
>
> --
> Nguyễn Thị Ngọc Hương
>
Re: Location of HBase's database (database' s files) on the hard disk
Posted by Nguyen Thi Ngoc Huong <hu...@gmail.com>.
Thank you very much.
I deleted everything and configured hadoop.tmp.dir property in
hadoop-site.xml as follow
<property>
<name>hadoop.tmp.dir</name>
<value>/home/huongntn/hadoop-${user.name}</value>
<description>A base for other temporary directories.</description>
</property>
After that, I formated namenode and start-all. When I restarted my computer
and typed the command "start-all", hadoop work smoothly. I start hbase by
command "./bin/start-hbase.sh" and "./hbase shell"
Now I can't see my database in hbase shell (by command "list") although I
can see it in Hadoop site manager,
2009/8/21 Amandeep Khurana <am...@gmail.com>
> 1. If you have formatted your namenode before starting the first time,
> thats
> all thats needed.
>
> 2. To start from scratch, delete everything thats there in the directory
> where you are pointing your hdfs to; format namenode again; start all
>
> 3. If it still doesnt work, look at the namenode logs to see whats
> happening. Post it here if you cant figure it out.
>
>
> Amandeep Khurana
> Computer Science Graduate Student
> University of California, Santa Cruz
>
>
> On Fri, Aug 21, 2009 at 1:30 AM, Nguyen Thi Ngoc Huong
> <hu...@gmail.com>wrote:
>
> > >You dont need to format the namenode everytime.. Just bin/start-all.sh
> >
> > Really? Just bin/start-all.sh, namnode is not started (when I type
> command
> > "jps", there are only 5 processes
> > 3421 SecondaryNameNode
> > 3492 JobTracker
> > 3582 TaskTracker
> > 4031 Jps
> > 3325 DataNode, there isn't Namenode process)
> > and certainly, the page http://localhost:50070 is died and connection
> from
> > Hbase to hadoop is died, too
> >
> >
> > 2009/8/21 Amandeep Khurana <am...@gmail.com>
> >
> > > On Fri, Aug 21, 2009 at 1:03 AM, Nguyen Thi Ngoc Huong
> > > <hu...@gmail.com>wrote:
> > >
> > > > Thanks you very much. I editted file hbase-site.xml as follow
> > > >
> > > > <property>
> > > > <name>hbase.rootdir</name>
> > > > <value>hdfs://localhost:54310/hbase</value>
> > > > <description>The directory shared by region servers.
> > > > Should be fully-qualified to include the filesystem to use.
> > > > E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
> > > > </description>
> > > > </property>
> > > >
> > > > with fs.default.name is hdfs://localhost:54310
> > > > Now, I can see hbase database in Hadoop site manager (in "hbase"
> > > directory
> > > > not "tmp" directory in hdfs ).
> > > > However, when I restart my computer, I must restart hadoop (by
> command
> > > > ./bin/hadoop format namenode and ./bin/start all) , restart hbase,
> and
> > my
> > > > database is lost. What can I do to save my database?
> > > >
> > >
> > > You dont need to format the namenode everytime.. Just bin/start-all.sh
> > >
> > >
> > >
> > > >
> > > > 2009/8/21 Amandeep Khurana <am...@gmail.com>
> > > >
> > > > > On Thu, Aug 20, 2009 at 11:46 PM, Nguyen Thi Ngoc Huong <
> > > > > huongngtn@gmail.com
> > > > > > wrote:
> > > > >
> > > > > > How can I configure the location of the hbase directory? I
> > configured
> > > > > > hbase-site.xml as follow:
> > > > > >
> > > > > > <property>
> > > > > > <name>hbase.rootdir</name>
> > > > > > <value>*file:///temp/hbase-${user.name}/hbase*</value>
> > > > > > <description>The directory shared by region servers.
> > > > > > Should be fully-qualified to include the filesystem to use.
> > > > > > E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
> > > > > > </description>
> > > > > > </property>
> > > > > >
> > > > >
> > > > > Thats the trouble.. Your data is being stored in the temp.. instead
> > > store
> > > > > it
> > > > > in your hdfs.
> > > > > so the value of the above property would be something like
> > > > > *hdfs://namenodeserver:port/hbase*
> > > > >
> > > > >
> > > > >
> > > > > >
> > > > > > and the log file is
> > > > > > Not starting HMaster because:
> > > > > > java.io.IOException: Mkdirs failed to create
> > > > > > file:/temp/hbase-huongntn/hbase
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
> > > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
> > > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
> > > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
> > > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
> > > > > > at
> > > org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
> > > > > > at
> > org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
> > > > > > at
> > org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
> > > > > > at
> > > org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
> > > > > > at
> > org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
> > > > > > 2009-08-21 13:35:24,163 ERROR
> > org.apache.hadoop.hbase.master.HMaster:
> > > > Can
> > > > > > not start master
> > > > > > java.io.IOException: Mkdirs failed to create
> > > > > > file:/temp/hbase-huongntn/hbase
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
> > > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
> > > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
> > > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
> > > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
> > > > > > at
> > > org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
> > > > > > at
> > org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
> > > > > > at
> > org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
> > > > > > at
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
> > > > > > at
> > > org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
> > > > > > at
> > org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
> > > > > >
> > > > > >
> > > > > > 2009/8/21 Amandeep Khurana <am...@gmail.com>
> > > > > >
> > > > > >
> > > > > >
> > > > > > > You configure the location of the hbase directory in the
> > > > hbase-site.xml
> > > > > > >
> > > > > > > The data being lost could have multilple reasons. To rule out
> the
> > > > > > > basic one - where have you pointed the hdfs to store data? If
> its
> > > > > > > going into /tmp, you'll lose data everytime the tmp cleaner
> comes
> > > > into
> > > > > > > action.
> > > > > > >
> > > > > > > On 8/20/09, Nguyen Thi Ngoc Huong <hu...@gmail.com> wrote:
> > > > > > > > Hi all,
> > > > > > > > I am a beginner to HBase. I have some question with Hbase
> after
> > > > setup
> > > > > > > Hbase
> > > > > > > > and Hadoop.
> > > > > > > >
> > > > > > > > The first, After setup Hbase and create a new database, I
> don't
> > > > know
> > > > > > > where
> > > > > > > > is location of HBase's database (database' s files) on the
> hard
> > > > disk.
> > > > > > At
> > > > > > > the
> > > > > > > > first, I think it is on the hbase.rootdir directory, however,
> > > when
> > > > I
> > > > > > > delete
> > > > > > > > directory hbase.rootdir, and type the command "list", all of
> > > > database
> > > > > > > > exist.
> > > > > > > >
> > > > > > > > The second, after restart computer and restart hbase, all
> > > database
> > > > of
> > > > > > > HBase
> > > > > > > > is lost. Is it always true? Or did I configure wrong? How can
> i
> > > > > > configure
> > > > > > > > Hbase to save database after restart computer?
> > > > > > > >
> > > > > > > > --
> > > > > > > > Nguyễn Thị Ngọc Hương
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > >
> > > > > > >
> > > > > > > Amandeep Khurana
> > > > > > > Computer Science Graduate Student
> > > > > > > University of California, Santa Cruz
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Nguyễn Thị Ngọc Hương
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Nguyễn Thị Ngọc Hương
> > > >
> > >
> >
> >
> >
> > --
> > Nguyễn Thị Ngọc Hương
> >
>
--
Nguyễn Thị Ngọc Hương
Re: Location of HBase's database (database' s files) on the hard disk
Posted by Amandeep Khurana <am...@gmail.com>.
1. If you have formatted your namenode before starting the first time, thats
all thats needed.
2. To start from scratch, delete everything thats there in the directory
where you are pointing your hdfs to; format namenode again; start all
3. If it still doesnt work, look at the namenode logs to see whats
happening. Post it here if you cant figure it out.
Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz
On Fri, Aug 21, 2009 at 1:30 AM, Nguyen Thi Ngoc Huong
<hu...@gmail.com>wrote:
> >You dont need to format the namenode everytime.. Just bin/start-all.sh
>
> Really? Just bin/start-all.sh, namnode is not started (when I type command
> "jps", there are only 5 processes
> 3421 SecondaryNameNode
> 3492 JobTracker
> 3582 TaskTracker
> 4031 Jps
> 3325 DataNode, there isn't Namenode process)
> and certainly, the page http://localhost:50070 is died and connection from
> Hbase to hadoop is died, too
>
>
> 2009/8/21 Amandeep Khurana <am...@gmail.com>
>
> > On Fri, Aug 21, 2009 at 1:03 AM, Nguyen Thi Ngoc Huong
> > <hu...@gmail.com>wrote:
> >
> > > Thanks you very much. I editted file hbase-site.xml as follow
> > >
> > > <property>
> > > <name>hbase.rootdir</name>
> > > <value>hdfs://localhost:54310/hbase</value>
> > > <description>The directory shared by region servers.
> > > Should be fully-qualified to include the filesystem to use.
> > > E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
> > > </description>
> > > </property>
> > >
> > > with fs.default.name is hdfs://localhost:54310
> > > Now, I can see hbase database in Hadoop site manager (in "hbase"
> > directory
> > > not "tmp" directory in hdfs ).
> > > However, when I restart my computer, I must restart hadoop (by command
> > > ./bin/hadoop format namenode and ./bin/start all) , restart hbase, and
> my
> > > database is lost. What can I do to save my database?
> > >
> >
> > You dont need to format the namenode everytime.. Just bin/start-all.sh
> >
> >
> >
> > >
> > > 2009/8/21 Amandeep Khurana <am...@gmail.com>
> > >
> > > > On Thu, Aug 20, 2009 at 11:46 PM, Nguyen Thi Ngoc Huong <
> > > > huongngtn@gmail.com
> > > > > wrote:
> > > >
> > > > > How can I configure the location of the hbase directory? I
> configured
> > > > > hbase-site.xml as follow:
> > > > >
> > > > > <property>
> > > > > <name>hbase.rootdir</name>
> > > > > <value>*file:///temp/hbase-${user.name}/hbase*</value>
> > > > > <description>The directory shared by region servers.
> > > > > Should be fully-qualified to include the filesystem to use.
> > > > > E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
> > > > > </description>
> > > > > </property>
> > > > >
> > > >
> > > > Thats the trouble.. Your data is being stored in the temp.. instead
> > store
> > > > it
> > > > in your hdfs.
> > > > so the value of the above property would be something like
> > > > *hdfs://namenodeserver:port/hbase*
> > > >
> > > >
> > > >
> > > > >
> > > > > and the log file is
> > > > > Not starting HMaster because:
> > > > > java.io.IOException: Mkdirs failed to create
> > > > > file:/temp/hbase-huongntn/hbase
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
> > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
> > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
> > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
> > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
> > > > > at
> > org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
> > > > > at
> org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
> > > > > at
> org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
> > > > > at
> > org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
> > > > > at
> org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
> > > > > 2009-08-21 13:35:24,163 ERROR
> org.apache.hadoop.hbase.master.HMaster:
> > > Can
> > > > > not start master
> > > > > java.io.IOException: Mkdirs failed to create
> > > > > file:/temp/hbase-huongntn/hbase
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
> > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
> > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
> > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
> > > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
> > > > > at
> > org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
> > > > > at
> org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
> > > > > at
> org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
> > > > > at
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
> > > > > at
> > org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
> > > > > at
> org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
> > > > >
> > > > >
> > > > > 2009/8/21 Amandeep Khurana <am...@gmail.com>
> > > > >
> > > > >
> > > > >
> > > > > > You configure the location of the hbase directory in the
> > > hbase-site.xml
> > > > > >
> > > > > > The data being lost could have multilple reasons. To rule out the
> > > > > > basic one - where have you pointed the hdfs to store data? If its
> > > > > > going into /tmp, you'll lose data everytime the tmp cleaner comes
> > > into
> > > > > > action.
> > > > > >
> > > > > > On 8/20/09, Nguyen Thi Ngoc Huong <hu...@gmail.com> wrote:
> > > > > > > Hi all,
> > > > > > > I am a beginner to HBase. I have some question with Hbase after
> > > setup
> > > > > > Hbase
> > > > > > > and Hadoop.
> > > > > > >
> > > > > > > The first, After setup Hbase and create a new database, I don't
> > > know
> > > > > > where
> > > > > > > is location of HBase's database (database' s files) on the hard
> > > disk.
> > > > > At
> > > > > > the
> > > > > > > first, I think it is on the hbase.rootdir directory, however,
> > when
> > > I
> > > > > > delete
> > > > > > > directory hbase.rootdir, and type the command "list", all of
> > > database
> > > > > > > exist.
> > > > > > >
> > > > > > > The second, after restart computer and restart hbase, all
> > database
> > > of
> > > > > > HBase
> > > > > > > is lost. Is it always true? Or did I configure wrong? How can i
> > > > > configure
> > > > > > > Hbase to save database after restart computer?
> > > > > > >
> > > > > > > --
> > > > > > > Nguyễn Thị Ngọc Hương
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > >
> > > > > > Amandeep Khurana
> > > > > > Computer Science Graduate Student
> > > > > > University of California, Santa Cruz
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Nguyễn Thị Ngọc Hương
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Nguyễn Thị Ngọc Hương
> > >
> >
>
>
>
> --
> Nguyễn Thị Ngọc Hương
>
Re: Location of HBase's database (database' s files) on the hard disk
Posted by Nguyen Thi Ngoc Huong <hu...@gmail.com>.
>You dont need to format the namenode everytime.. Just bin/start-all.sh
Really? Just bin/start-all.sh, namnode is not started (when I type command
"jps", there are only 5 processes
3421 SecondaryNameNode
3492 JobTracker
3582 TaskTracker
4031 Jps
3325 DataNode, there isn't Namenode process)
and certainly, the page http://localhost:50070 is died and connection from
Hbase to hadoop is died, too
2009/8/21 Amandeep Khurana <am...@gmail.com>
> On Fri, Aug 21, 2009 at 1:03 AM, Nguyen Thi Ngoc Huong
> <hu...@gmail.com>wrote:
>
> > Thanks you very much. I editted file hbase-site.xml as follow
> >
> > <property>
> > <name>hbase.rootdir</name>
> > <value>hdfs://localhost:54310/hbase</value>
> > <description>The directory shared by region servers.
> > Should be fully-qualified to include the filesystem to use.
> > E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
> > </description>
> > </property>
> >
> > with fs.default.name is hdfs://localhost:54310
> > Now, I can see hbase database in Hadoop site manager (in "hbase"
> directory
> > not "tmp" directory in hdfs ).
> > However, when I restart my computer, I must restart hadoop (by command
> > ./bin/hadoop format namenode and ./bin/start all) , restart hbase, and my
> > database is lost. What can I do to save my database?
> >
>
> You dont need to format the namenode everytime.. Just bin/start-all.sh
>
>
>
> >
> > 2009/8/21 Amandeep Khurana <am...@gmail.com>
> >
> > > On Thu, Aug 20, 2009 at 11:46 PM, Nguyen Thi Ngoc Huong <
> > > huongngtn@gmail.com
> > > > wrote:
> > >
> > > > How can I configure the location of the hbase directory? I configured
> > > > hbase-site.xml as follow:
> > > >
> > > > <property>
> > > > <name>hbase.rootdir</name>
> > > > <value>*file:///temp/hbase-${user.name}/hbase*</value>
> > > > <description>The directory shared by region servers.
> > > > Should be fully-qualified to include the filesystem to use.
> > > > E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
> > > > </description>
> > > > </property>
> > > >
> > >
> > > Thats the trouble.. Your data is being stored in the temp.. instead
> store
> > > it
> > > in your hdfs.
> > > so the value of the above property would be something like
> > > *hdfs://namenodeserver:port/hbase*
> > >
> > >
> > >
> > > >
> > > > and the log file is
> > > > Not starting HMaster because:
> > > > java.io.IOException: Mkdirs failed to create
> > > > file:/temp/hbase-huongntn/hbase
> > > > at
> > > >
> > >
> >
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
> > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
> > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
> > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
> > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
> > > > at
> org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
> > > > at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
> > > > at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
> > > > at
> > > >
> > >
> >
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
> > > > at
> > > >
> > >
> >
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
> > > > at
> org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
> > > > at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
> > > > 2009-08-21 13:35:24,163 ERROR org.apache.hadoop.hbase.master.HMaster:
> > Can
> > > > not start master
> > > > java.io.IOException: Mkdirs failed to create
> > > > file:/temp/hbase-huongntn/hbase
> > > > at
> > > >
> > >
> >
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
> > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
> > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
> > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
> > > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
> > > > at
> org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
> > > > at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
> > > > at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
> > > > at
> > > >
> > >
> >
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
> > > > at
> > > >
> > >
> >
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
> > > > at
> org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
> > > > at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
> > > >
> > > >
> > > > 2009/8/21 Amandeep Khurana <am...@gmail.com>
> > > >
> > > >
> > > >
> > > > > You configure the location of the hbase directory in the
> > hbase-site.xml
> > > > >
> > > > > The data being lost could have multilple reasons. To rule out the
> > > > > basic one - where have you pointed the hdfs to store data? If its
> > > > > going into /tmp, you'll lose data everytime the tmp cleaner comes
> > into
> > > > > action.
> > > > >
> > > > > On 8/20/09, Nguyen Thi Ngoc Huong <hu...@gmail.com> wrote:
> > > > > > Hi all,
> > > > > > I am a beginner to HBase. I have some question with Hbase after
> > setup
> > > > > Hbase
> > > > > > and Hadoop.
> > > > > >
> > > > > > The first, After setup Hbase and create a new database, I don't
> > know
> > > > > where
> > > > > > is location of HBase's database (database' s files) on the hard
> > disk.
> > > > At
> > > > > the
> > > > > > first, I think it is on the hbase.rootdir directory, however,
> when
> > I
> > > > > delete
> > > > > > directory hbase.rootdir, and type the command "list", all of
> > database
> > > > > > exist.
> > > > > >
> > > > > > The second, after restart computer and restart hbase, all
> database
> > of
> > > > > HBase
> > > > > > is lost. Is it always true? Or did I configure wrong? How can i
> > > > configure
> > > > > > Hbase to save database after restart computer?
> > > > > >
> > > > > > --
> > > > > > Nguyễn Thị Ngọc Hương
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > >
> > > > > Amandeep Khurana
> > > > > Computer Science Graduate Student
> > > > > University of California, Santa Cruz
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Nguyễn Thị Ngọc Hương
> > > >
> > >
> >
> >
> >
> > --
> > Nguyễn Thị Ngọc Hương
> >
>
--
Nguyễn Thị Ngọc Hương
Re: Location of HBase's database (database' s files) on the hard disk
Posted by Amandeep Khurana <am...@gmail.com>.
On Fri, Aug 21, 2009 at 1:03 AM, Nguyen Thi Ngoc Huong
<hu...@gmail.com>wrote:
> Thanks you very much. I editted file hbase-site.xml as follow
>
> <property>
> <name>hbase.rootdir</name>
> <value>hdfs://localhost:54310/hbase</value>
> <description>The directory shared by region servers.
> Should be fully-qualified to include the filesystem to use.
> E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
> </description>
> </property>
>
> with fs.default.name is hdfs://localhost:54310
> Now, I can see hbase database in Hadoop site manager (in "hbase" directory
> not "tmp" directory in hdfs ).
> However, when I restart my computer, I must restart hadoop (by command
> ./bin/hadoop format namenode and ./bin/start all) , restart hbase, and my
> database is lost. What can I do to save my database?
>
You dont need to format the namenode everytime.. Just bin/start-all.sh
>
> 2009/8/21 Amandeep Khurana <am...@gmail.com>
>
> > On Thu, Aug 20, 2009 at 11:46 PM, Nguyen Thi Ngoc Huong <
> > huongngtn@gmail.com
> > > wrote:
> >
> > > How can I configure the location of the hbase directory? I configured
> > > hbase-site.xml as follow:
> > >
> > > <property>
> > > <name>hbase.rootdir</name>
> > > <value>*file:///temp/hbase-${user.name}/hbase*</value>
> > > <description>The directory shared by region servers.
> > > Should be fully-qualified to include the filesystem to use.
> > > E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
> > > </description>
> > > </property>
> > >
> >
> > Thats the trouble.. Your data is being stored in the temp.. instead store
> > it
> > in your hdfs.
> > so the value of the above property would be something like
> > *hdfs://namenodeserver:port/hbase*
> >
> >
> >
> > >
> > > and the log file is
> > > Not starting HMaster because:
> > > java.io.IOException: Mkdirs failed to create
> > > file:/temp/hbase-huongntn/hbase
> > > at
> > >
> >
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
> > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
> > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
> > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
> > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
> > > at org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
> > > at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
> > > at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
> > > at
> > >
> >
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
> > > at
> > >
> >
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
> > > at org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
> > > at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
> > > 2009-08-21 13:35:24,163 ERROR org.apache.hadoop.hbase.master.HMaster:
> Can
> > > not start master
> > > java.io.IOException: Mkdirs failed to create
> > > file:/temp/hbase-huongntn/hbase
> > > at
> > >
> >
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
> > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
> > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
> > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
> > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
> > > at org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
> > > at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
> > > at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
> > > at
> > >
> >
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
> > > at
> > >
> >
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
> > > at org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
> > > at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
> > >
> > >
> > > 2009/8/21 Amandeep Khurana <am...@gmail.com>
> > >
> > >
> > >
> > > > You configure the location of the hbase directory in the
> hbase-site.xml
> > > >
> > > > The data being lost could have multilple reasons. To rule out the
> > > > basic one - where have you pointed the hdfs to store data? If its
> > > > going into /tmp, you'll lose data everytime the tmp cleaner comes
> into
> > > > action.
> > > >
> > > > On 8/20/09, Nguyen Thi Ngoc Huong <hu...@gmail.com> wrote:
> > > > > Hi all,
> > > > > I am a beginner to HBase. I have some question with Hbase after
> setup
> > > > Hbase
> > > > > and Hadoop.
> > > > >
> > > > > The first, After setup Hbase and create a new database, I don't
> know
> > > > where
> > > > > is location of HBase's database (database' s files) on the hard
> disk.
> > > At
> > > > the
> > > > > first, I think it is on the hbase.rootdir directory, however, when
> I
> > > > delete
> > > > > directory hbase.rootdir, and type the command "list", all of
> database
> > > > > exist.
> > > > >
> > > > > The second, after restart computer and restart hbase, all database
> of
> > > > HBase
> > > > > is lost. Is it always true? Or did I configure wrong? How can i
> > > configure
> > > > > Hbase to save database after restart computer?
> > > > >
> > > > > --
> > > > > Nguyễn Thị Ngọc Hương
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > >
> > > > Amandeep Khurana
> > > > Computer Science Graduate Student
> > > > University of California, Santa Cruz
> > > >
> > >
> > >
> > >
> > > --
> > > Nguyễn Thị Ngọc Hương
> > >
> >
>
>
>
> --
> Nguyễn Thị Ngọc Hương
>
Re: Location of HBase's database (database' s files) on the hard disk
Posted by Nguyen Thi Ngoc Huong <hu...@gmail.com>.
Thanks you very much. I editted file hbase-site.xml as follow
<property>
<name>hbase.rootdir</name>
<value>hdfs://localhost:54310/hbase</value>
<description>The directory shared by region servers.
Should be fully-qualified to include the filesystem to use.
E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
</description>
</property>
with fs.default.name is hdfs://localhost:54310
Now, I can see hbase database in Hadoop site manager (in "hbase" directory
not "tmp" directory in hdfs ).
However, when I restart my computer, I must restart hadoop (by command
./bin/hadoop format namenode and ./bin/start all) , restart hbase, and my
database is lost. What can I do to save my database?
2009/8/21 Amandeep Khurana <am...@gmail.com>
> On Thu, Aug 20, 2009 at 11:46 PM, Nguyen Thi Ngoc Huong <
> huongngtn@gmail.com
> > wrote:
>
> > How can I configure the location of the hbase directory? I configured
> > hbase-site.xml as follow:
> >
> > <property>
> > <name>hbase.rootdir</name>
> > <value>*file:///temp/hbase-${user.name}/hbase*</value>
> > <description>The directory shared by region servers.
> > Should be fully-qualified to include the filesystem to use.
> > E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
> > </description>
> > </property>
> >
>
> Thats the trouble.. Your data is being stored in the temp.. instead store
> it
> in your hdfs.
> so the value of the above property would be something like
> *hdfs://namenodeserver:port/hbase*
>
>
>
> >
> > and the log file is
> > Not starting HMaster because:
> > java.io.IOException: Mkdirs failed to create
> > file:/temp/hbase-huongntn/hbase
> > at
> >
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
> > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
> > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
> > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
> > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
> > at org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
> > at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
> > at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
> > at
> >
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
> > at
> >
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
> > at org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
> > at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
> > 2009-08-21 13:35:24,163 ERROR org.apache.hadoop.hbase.master.HMaster: Can
> > not start master
> > java.io.IOException: Mkdirs failed to create
> > file:/temp/hbase-huongntn/hbase
> > at
> >
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
> > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
> > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
> > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
> > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
> > at org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
> > at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
> > at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
> > at
> >
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
> > at
> >
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
> > at org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
> > at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
> >
> >
> > 2009/8/21 Amandeep Khurana <am...@gmail.com>
> >
> >
> >
> > > You configure the location of the hbase directory in the hbase-site.xml
> > >
> > > The data being lost could have multilple reasons. To rule out the
> > > basic one - where have you pointed the hdfs to store data? If its
> > > going into /tmp, you'll lose data everytime the tmp cleaner comes into
> > > action.
> > >
> > > On 8/20/09, Nguyen Thi Ngoc Huong <hu...@gmail.com> wrote:
> > > > Hi all,
> > > > I am a beginner to HBase. I have some question with Hbase after setup
> > > Hbase
> > > > and Hadoop.
> > > >
> > > > The first, After setup Hbase and create a new database, I don't know
> > > where
> > > > is location of HBase's database (database' s files) on the hard disk.
> > At
> > > the
> > > > first, I think it is on the hbase.rootdir directory, however, when I
> > > delete
> > > > directory hbase.rootdir, and type the command "list", all of database
> > > > exist.
> > > >
> > > > The second, after restart computer and restart hbase, all database of
> > > HBase
> > > > is lost. Is it always true? Or did I configure wrong? How can i
> > configure
> > > > Hbase to save database after restart computer?
> > > >
> > > > --
> > > > Nguyễn Thị Ngọc Hương
> > > >
> > >
> > >
> > > --
> > >
> > >
> > > Amandeep Khurana
> > > Computer Science Graduate Student
> > > University of California, Santa Cruz
> > >
> >
> >
> >
> > --
> > Nguyễn Thị Ngọc Hương
> >
>
--
Nguyễn Thị Ngọc Hương
Re: Location of HBase's database (database' s files) on the hard disk
Posted by Amandeep Khurana <am...@gmail.com>.
On Thu, Aug 20, 2009 at 11:46 PM, Nguyen Thi Ngoc Huong <huongngtn@gmail.com
> wrote:
> How can I configure the location of the hbase directory? I configured
> hbase-site.xml as follow:
>
> <property>
> <name>hbase.rootdir</name>
> <value>*file:///temp/hbase-${user.name}/hbase*</value>
> <description>The directory shared by region servers.
> Should be fully-qualified to include the filesystem to use.
> E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
> </description>
> </property>
>
Thats the trouble.. Your data is being stored in the temp.. instead store it
in your hdfs.
so the value of the above property would be something like
*hdfs://namenodeserver:port/hbase*
>
> and the log file is
> Not starting HMaster because:
> java.io.IOException: Mkdirs failed to create
> file:/temp/hbase-huongntn/hbase
> at
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
> at org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
> at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
> at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
> at org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
> 2009-08-21 13:35:24,163 ERROR org.apache.hadoop.hbase.master.HMaster: Can
> not start master
> java.io.IOException: Mkdirs failed to create
> file:/temp/hbase-huongntn/hbase
> at
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
> at org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
> at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
> at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
> at org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
> at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
>
>
> 2009/8/21 Amandeep Khurana <am...@gmail.com>
>
>
>
> > You configure the location of the hbase directory in the hbase-site.xml
> >
> > The data being lost could have multilple reasons. To rule out the
> > basic one - where have you pointed the hdfs to store data? If its
> > going into /tmp, you'll lose data everytime the tmp cleaner comes into
> > action.
> >
> > On 8/20/09, Nguyen Thi Ngoc Huong <hu...@gmail.com> wrote:
> > > Hi all,
> > > I am a beginner to HBase. I have some question with Hbase after setup
> > Hbase
> > > and Hadoop.
> > >
> > > The first, After setup Hbase and create a new database, I don't know
> > where
> > > is location of HBase's database (database' s files) on the hard disk.
> At
> > the
> > > first, I think it is on the hbase.rootdir directory, however, when I
> > delete
> > > directory hbase.rootdir, and type the command "list", all of database
> > > exist.
> > >
> > > The second, after restart computer and restart hbase, all database of
> > HBase
> > > is lost. Is it always true? Or did I configure wrong? How can i
> configure
> > > Hbase to save database after restart computer?
> > >
> > > --
> > > Nguyễn Thị Ngọc Hương
> > >
> >
> >
> > --
> >
> >
> > Amandeep Khurana
> > Computer Science Graduate Student
> > University of California, Santa Cruz
> >
>
>
>
> --
> Nguyễn Thị Ngọc Hương
>
Re: Location of HBase's database (database' s files) on the hard disk
Posted by Nguyen Thi Ngoc Huong <hu...@gmail.com>.
How can I configure the location of the hbase directory? I configured
hbase-site.xml as follow:
<property>
<name>hbase.rootdir</name>
<value>file:///temp/hbase-${user.name}/hbase</value>
<description>The directory shared by region servers.
Should be fully-qualified to include the filesystem to use.
E.g: hdfs://NAMENODE_SERVER:PORT/HBASE_ROOTDIR
</description>
</property>
and the log file is
Not starting HMaster because:
java.io.IOException: Mkdirs failed to create file:/temp/hbase-huongntn/hbase
at
org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
at org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
at
org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
at
org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
at org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
2009-08-21 13:35:24,163 ERROR org.apache.hadoop.hbase.master.HMaster: Can
not start master
java.io.IOException: Mkdirs failed to create file:/temp/hbase-huongntn/hbase
at
org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:358)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:487)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:468)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:375)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:367)
at org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:141)
at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:210)
at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:156)
at
org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:96)
at
org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:78)
at org.apache.hadoop.hbase.master.HMaster.doMain(HMaster.java:1013)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1057)
2009/8/21 Amandeep Khurana <am...@gmail.com>
> You configure the location of the hbase directory in the hbase-site.xml
>
> The data being lost could have multilple reasons. To rule out the
> basic one - where have you pointed the hdfs to store data? If its
> going into /tmp, you'll lose data everytime the tmp cleaner comes into
> action.
>
> On 8/20/09, Nguyen Thi Ngoc Huong <hu...@gmail.com> wrote:
> > Hi all,
> > I am a beginner to HBase. I have some question with Hbase after setup
> Hbase
> > and Hadoop.
> >
> > The first, After setup Hbase and create a new database, I don't know
> where
> > is location of HBase's database (database' s files) on the hard disk. At
> the
> > first, I think it is on the hbase.rootdir directory, however, when I
> delete
> > directory hbase.rootdir, and type the command "list", all of database
> > exist.
> >
> > The second, after restart computer and restart hbase, all database of
> HBase
> > is lost. Is it always true? Or did I configure wrong? How can i configure
> > Hbase to save database after restart computer?
> >
> > --
> > Nguyễn Thị Ngọc Hương
> >
>
>
> --
>
>
> Amandeep Khurana
> Computer Science Graduate Student
> University of California, Santa Cruz
>
--
Nguyễn Thị Ngọc Hương
Re: Location of HBase's database (database' s files) on the hard disk
Posted by Amandeep Khurana <am...@gmail.com>.
You configure the location of the hbase directory in the hbase-site.xml
The data being lost could have multilple reasons. To rule out the
basic one - where have you pointed the hdfs to store data? If its
going into /tmp, you'll lose data everytime the tmp cleaner comes into
action.
On 8/20/09, Nguyen Thi Ngoc Huong <hu...@gmail.com> wrote:
> Hi all,
> I am a beginner to HBase. I have some question with Hbase after setup Hbase
> and Hadoop.
>
> The first, After setup Hbase and create a new database, I don't know where
> is location of HBase's database (database' s files) on the hard disk. At the
> first, I think it is on the hbase.rootdir directory, however, when I delete
> directory hbase.rootdir, and type the command "list", all of database
> exist.
>
> The second, after restart computer and restart hbase, all database of HBase
> is lost. Is it always true? Or did I configure wrong? How can i configure
> Hbase to save database after restart computer?
>
> --
> Nguyễn Thị Ngọc Hương
>
--
Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz