You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "zhuobin zheng (Jira)" <ji...@apache.org> on 2019/12/19 12:58:00 UTC
[jira] [Updated] (HBASE-23598) There are too much small WAL File
[ https://issues.apache.org/jira/browse/HBASE-23598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
zhuobin zheng updated HBASE-23598:
----------------------------------
Attachment: wals
Component/s: wal
Affects Version/s: 2.2.2
Description:
I found 10W + WAL files in my 400-scale hbase cluster. Too many WAL files will cause the cluster and recover very slowly when cluster crash completely . (In the split log step) (because too many WAL files will cause too many ZK requests). By default, WAL files start to roll when they reach HDFS Block Size (256M In My Case) * 0.95. But I found that there are many small files (0-100M) in the WAL directory. When I look at the code , I found that when I configured multiwal (I configured 4 WALs for each RS), as long as a single WAL file reached HDFS Block Size (256M In My Case) * 0.95, all WAL files would scroll, so it caused a lot of WAL small files.
I tried to modify the code to solve the problem (making each WAL scroll independently). Although this change is very small, I am not sure if such a change will cause other problems, currently being tested ...
Environment:
hbase version: cdh5-1.2.0_5.14.4
hbase.wal.provider: multiwal
hbase.wal.regiongrouping.numgroups: 4
The wals file shows 100+ wal files in wal-3 , and some of them has very small size
Summary: There are too much small WAL File (was: There are too much small WAL)
Remaining Estimate: 168h
Original Estimate: 168h
> There are too much small WAL File
> ---------------------------------
>
> Key: HBASE-23598
> URL: https://issues.apache.org/jira/browse/HBASE-23598
> Project: HBase
> Issue Type: Improvement
> Components: wal
> Affects Versions: 1.3.6, 2.2.2
> Environment: hbase version: cdh5-1.2.0_5.14.4
> hbase.wal.provider: multiwal
> hbase.wal.regiongrouping.numgroups: 4
> The wals file shows 100+ wal files in wal-3 , and some of them has very small size
> Reporter: zhuobin zheng
> Priority: Major
> Attachments: wals
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> I found 10W + WAL files in my 400-scale hbase cluster. Too many WAL files will cause the cluster and recover very slowly when cluster crash completely . (In the split log step) (because too many WAL files will cause too many ZK requests). By default, WAL files start to roll when they reach HDFS Block Size (256M In My Case) * 0.95. But I found that there are many small files (0-100M) in the WAL directory. When I look at the code , I found that when I configured multiwal (I configured 4 WALs for each RS), as long as a single WAL file reached HDFS Block Size (256M In My Case) * 0.95, all WAL files would scroll, so it caused a lot of WAL small files.
> I tried to modify the code to solve the problem (making each WAL scroll independently). Although this change is very small, I am not sure if such a change will cause other problems, currently being tested ...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)