[hbase] branch HBASE-21879 updated: HBASE-22483 It's better to use 65KB as the default buffer size in ByteBuffAllocator (#279)

[op...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

openinx pushed a commit to branch HBASE-21879
in repository https://gitbox.apache.org/repos/asf/hbase.git


The following commit(s) were added to refs/heads/HBASE-21879 by this push:
     new e2c8e65  HBASE-22483 It's better to use 65KB as the default buffer size in ByteBuffAllocator (#279)
e2c8e65 is described below

commit e2c8e65cdfe5dde6f0a2336d48f104439a327aad
Author: openinx <op...@gmail.com>
AuthorDate: Fri May 31 17:53:00 2019 +0800

    HBASE-22483 It's better to use 65KB as the default buffer size in ByteBuffAllocator (#279)
---
 .../apache/hadoop/hbase/io/ByteBuffAllocator.java    | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/hbase-common/src/main/java/org/apache/hadoop/hbase/io/ByteBuffAllocator.java b/hbase-common/src/main/java/org/apache/hadoop/hbase/io/ByteBuffAllocator.java
index 75a4699..c85675b 100644
--- a/hbase-common/src/main/java/org/apache/hadoop/hbase/io/ByteBuffAllocator.java
+++ b/hbase-common/src/main/java/org/apache/hadoop/hbase/io/ByteBuffAllocator.java
@@ -71,8 +71,24 @@ public class ByteBuffAllocator {
   public static final String MAX_BUFFER_COUNT_KEY = "hbase.ipc.server.allocator.max.buffer.count";
 
   public static final String BUFFER_SIZE_KEY = "hbase.ipc.server.allocator.buffer.size";
-  // 64 KB. Making it same as the chunk size what we will write/read to/from the socket channel.
-  public static final int DEFAULT_BUFFER_SIZE = 64 * 1024;
+
+  /**
+   * There're some reasons why better to choose 65KB(rather than 64KB) as the default buffer size:
+   * <p>
+   * 1. Almost all of the data blocks have the block size: 64KB + delta, whose delta is very small,
+   * depends on the size of lastKeyValue. If we set buffer.size=64KB, then each block will be
+   * allocated as a MultiByteBuff: one 64KB DirectByteBuffer and delta bytes HeapByteBuffer, the
+   * HeapByteBuffer will increase the GC pressure. Ideally, we should let the data block to be
+   * allocated as a SingleByteBuff, it has simpler data structure, faster access speed, less heap
+   * usage.
+   * <p>
+   * 2. Since the blocks are MultiByteBuff when using buffer.size=64KB, so we have to calculate the
+   * checksum by an temp heap copying (see HBASE-21917), while if it's a SingleByteBuff, we can
+   * speed the checksum by calling the hadoop' checksum in native lib, which is more faster.
+   * <p>
+   * For performance comparison, please see HBASE-22483.
+   */
+  public static final int DEFAULT_BUFFER_SIZE = 65 * 1024;
 
   public static final String MIN_ALLOCATE_SIZE_KEY =
       "hbase.ipc.server.reservoir.minimal.allocating.size";