You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org> on 2011/08/11 08:59:28 UTC
[jira] [Commented] (HBASE-4192) Optimize HLog for Throughput Using
Delayed RPCs
[ https://issues.apache.org/jira/browse/HBASE-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082967#comment-13082967 ]
jiraposter@reviews.apache.org commented on HBASE-4192:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1463/
-----------------------------------------------------------
Review request for hbase.
Summary
-------
Changes:
1. Add hbase.region.wal.batchentries configuration parameter. If this is
enabled, batch entries to the HLog in a queue.
2. Use delayed RPCs for sync requests when aggresive batching is enabled.
This frees up RPC handler threads for the duration of the sync.
3. Pass the RPC server instance all the way to down to HLog. This is needed
to find out the current remote call, mark it as delayed, and finally complete
it when the sync is done.
4. Use the region read-write consistency control to avoid exposing to
RegionScanners edits which have not yet been synced.
5. Change a few tests which directly create HRegions or HLogs. The
rpcServers passed in are null, HLog falls back to classic RPCs when it has no
knowledge of the RPC server.
6. Add TestBatchEntriesLogRolling, which is identical to TestLogRolling,
except that it uses aggressive batching. I'm not sure how to add tests that
verify the same functionality but don't duplicate code, suggestion are
welcome.
The new parameter is disabled by default.
This addresses bug HBASE-4192.
https://issues.apache.org/jira/browse/HBASE-4192
Diffs
-----
src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java 6fb1da7
src/main/java/org/apache/hadoop/hbase/master/HMaster.java a00b93d
src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 2f86f04
src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 83ff7b2
src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java f22fb6e
src/main/java/org/apache/hadoop/hbase/regionserver/ReadWriteConsistencyControl.java 8ec53d3
src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 887f736
src/main/java/org/apache/hadoop/hbase/util/HMerge.java 9f0499e
src/main/java/org/apache/hadoop/hbase/util/MetaUtils.java af8d734
src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java bf8004b
src/test/java/org/apache/hadoop/hbase/TestScanMultipleVersions.java d7baba4
src/test/java/org/apache/hadoop/hbase/coprocessor/TestCoprocessorInterface.java 36816e8
src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestBatchEntriesLogRolling.java PRE-CREATION
src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLogSplit.java de28418
src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALActionsListener.java dc43eb2
src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java 381ac90
Diff: https://reviews.apache.org/r/1463/diff
Testing
-------
All unit tests run with aggressive batching turned on and off.
Thanks,
Vlad
> Optimize HLog for Throughput Using Delayed RPCs
> -----------------------------------------------
>
> Key: HBASE-4192
> URL: https://issues.apache.org/jira/browse/HBASE-4192
> Project: HBase
> Issue Type: New Feature
> Components: wal
> Affects Versions: 0.92.0
> Reporter: Vlad Dogaru
> Priority: Minor
>
> Introduce a new HLog configuration parameter (batchEntries) for more aggressive batching of appends. If this is enabled, HLog appends are not written to the HLog writer immediately, but batched and written either periodically or when a sync is requested. Because sync times become larger, they use delayed RPCs to free up RPC handler threads.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira