[hbase] branch branch-2 updated: HBASE-27166 WAL value compression minor improvements (#4584)

[ap...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

apurtell pushed a commit to branch branch-2
in repository https://gitbox.apache.org/repos/asf/hbase.git


The following commit(s) were added to refs/heads/branch-2 by this push:
     new e4d2895cadf HBASE-27166 WAL value compression minor improvements (#4584)
e4d2895cadf is described below

commit e4d2895cadf021465719dcd7c9178ce72e3f75c2
Author: Andrew Purtell <ap...@apache.org>
AuthorDate: Tue Jun 28 12:49:25 2022 -0700

    HBASE-27166 WAL value compression minor improvements (#4584)
    
    A larger IO buffer for absorbing WALCodec writes can improve the compression
    ratio of larger values, because the compressor will be given a larger internal
    buffer over which there will be more match opportunities. Does not impact the
    ability to read existing written files.
    
    Also, reset the BAOS internal buffer on the way out of compress() so potential
    large-ish buffers do not linger on the heap longer than necessary.
    
    Signed-off-by: Duo Zhang <zh...@apache.org>
---
 .../apache/hadoop/hbase/regionserver/wal/CompressionContext.java  | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/CompressionContext.java b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/CompressionContext.java
index e626d9e14a8..0ac0ce0d4fb 100644
--- a/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/CompressionContext.java
+++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/CompressionContext.java
@@ -71,7 +71,7 @@ public class CompressionContext {
    */
   static class ValueCompressor {
 
-    static final int IO_BUFFER_SIZE = 4096;
+    static final int IO_BUFFER_SIZE = 64 * 1024; // bigger buffer improves large edit compress ratio
 
     private final Compression.Algorithm algorithm;
     private Compressor compressor;
@@ -97,12 +97,12 @@ public class CompressionContext {
           compressor = algorithm.getCompressor();
         }
         compressedOut = algorithm.createCompressionStream(lowerOut, compressor, IO_BUFFER_SIZE);
-      } else {
-        lowerOut.reset();
       }
       compressedOut.write(valueArray, valueOffset, valueLength);
       compressedOut.flush();
-      return lowerOut.toByteArray();
+      final byte[] compressed = lowerOut.toByteArray();
+      lowerOut.reset(); // Reset now to minimize the overhead of keeping around the BAOS
+      return compressed;
     }
 
     public int decompress(InputStream in, int inLength, byte[] outArray, int outOffset,