You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Mohit <mo...@huawei.com> on 2010/12/15 06:41:10 UTC
Difference between logs present under /hbase/.logs and
/hbase/testTable/6674094/.logs directory
Hello HBase Users,
I explored the root directory of hbase on dfs which is /hbase, under it I
found two logs folders /hbase/.logs and a log directory under each table, in
this case testTable.
/hbase/testTable/6674094/.logs.
As per HBase storage structure, data is written to write-ahead-log, OR Hlog
and then to memstore for each hstores(directly proportional to column
families) and after that to dfs, such that each Hregion has it's own
instance of Hlog, so what is this log directory /hbase/.logs which during
startup of Hmaster is checked for the presence of log files for uninformed
splits.
So I'm bit confused, what is the purpose of each, and how they fit in.
-Mohit
****************************************************************************
***********
This e-mail and attachments contain confidential information from HUAWEI,
which is intended only for the person or entity whose address is listed
above. Any use of the information contained herein in any way (including,
but not limited to, total or partial disclosure, reproduction, or
dissemination) by persons other than the intended recipient's) is
prohibited. If you receive this e-mail in error, please notify the sender by
phone or email immediately and delete it!
RE: Difference between logs present under /hbase/.logs and
/hbase/testTable/6674094/.logs directory
Posted by Mohit <mo...@huawei.com>.
Hi Lars,
Thanks for the inputs.
Yes I can see in the master log file while creating HRegionInfo object.
New Hlog /hbase/testTable/6674094/.logs/hlog.dat.123456789
followed by log statement ,
creating region testTable,,458236390,encoded=6674094
-Mohit
****************************************************************************
***********
This e-mail and attachments contain confidential information from HUAWEI,
which is intended only for the person or entity whose address is listed
above. Any use of the information contained herein in any way (including,
but not limited to, total or partial disclosure, reproduction, or
dissemination) by persons other than the intended recipient's) is
prohibited. If you receive this e-mail in error, please notify the sender by
phone or email immediately and delete it!
-----Original Message-----
From: Lars George [mailto:lars.george@gmail.com]
Sent: Wednesday, December 15, 2010 12:19 PM
To: user@hbase.apache.org; mohitsikri@huawei.com
Subject: Re: Difference between logs present under /hbase/.logs and
/hbase/testTable/6674094/.logs directory
Hi Mohit,
The one under /hbase/.logs is the one per region server. It is split
in case there is a region server crash and put into
/hbase/<tablename>/<regionhash>/.oldlogs before the region is
redeployed.
Are you sure you saw a .logs underneath the region directory or was it
in fact a .oldlogs?
Lars
On Wed, Dec 15, 2010 at 6:41 AM, Mohit <mo...@huawei.com> wrote:
> Hello HBase Users,
>
>
>
> I explored the root directory of hbase on dfs which is /hbase, under it I
> found two logs folders /hbase/.logs and a log directory under each table,
in
> this case testTable.
>
> /hbase/testTable/6674094/.logs.
>
>
>
> As per HBase storage structure, data is written to write-ahead-log, OR
Hlog
> and then to memstore for each hstores(directly proportional to column
> families) and after that to dfs, such that each Hregion has it's own
> instance of Hlog, so what is this log directory /hbase/.logs which during
> startup of Hmaster is checked for the presence of log files for uninformed
> splits.
>
>
>
> So I'm bit confused, what is the purpose of each, and how they fit in.
>
>
>
> -Mohit
>
>
****************************************************************************
> ***********
> This e-mail and attachments contain confidential information from HUAWEI,
> which is intended only for the person or entity whose address is listed
> above. Any use of the information contained herein in any way (including,
> but not limited to, total or partial disclosure, reproduction, or
> dissemination) by persons other than the intended recipient's) is
> prohibited. If you receive this e-mail in error, please notify the sender
by
> phone or email immediately and delete it!
>
>
>
>
Re: Difference between logs present under /hbase/.logs and
/hbase/testTable/6674094/.logs directory
Posted by Lars George <la...@gmail.com>.
Hi Mohit,
The one under /hbase/.logs is the one per region server. It is split
in case there is a region server crash and put into
/hbase/<tablename>/<regionhash>/.oldlogs before the region is
redeployed.
Are you sure you saw a .logs underneath the region directory or was it
in fact a .oldlogs?
Lars
On Wed, Dec 15, 2010 at 6:41 AM, Mohit <mo...@huawei.com> wrote:
> Hello HBase Users,
>
>
>
> I explored the root directory of hbase on dfs which is /hbase, under it I
> found two logs folders /hbase/.logs and a log directory under each table, in
> this case testTable.
>
> /hbase/testTable/6674094/.logs.
>
>
>
> As per HBase storage structure, data is written to write-ahead-log, OR Hlog
> and then to memstore for each hstores(directly proportional to column
> families) and after that to dfs, such that each Hregion has it's own
> instance of Hlog, so what is this log directory /hbase/.logs which during
> startup of Hmaster is checked for the presence of log files for uninformed
> splits.
>
>
>
> So I'm bit confused, what is the purpose of each, and how they fit in.
>
>
>
> -Mohit
>
> ****************************************************************************
> ***********
> This e-mail and attachments contain confidential information from HUAWEI,
> which is intended only for the person or entity whose address is listed
> above. Any use of the information contained herein in any way (including,
> but not limited to, total or partial disclosure, reproduction, or
> dissemination) by persons other than the intended recipient's) is
> prohibited. If you receive this e-mail in error, please notify the sender by
> phone or email immediately and delete it!
>
>
>
>