[GitHub] [hbase] Apache9 commented on pull request #4675: HBASE-27264 Add options to consider compressed size when delimiting blocks during hfile writes

[GitBox <gi...@apache.org>]

Apache9 commented on PR #4675:
URL: https://github.com/apache/hbase/pull/4675#issuecomment-1209216982

   I think this is a useful feature. When implementing an in-house LSM tree based storage system, I used to make use of the compression rates for the already written blocks to predicate the compression rate of the next block, to determine whether we should finish a block.
   
   I think the approach here is also OK, compressing once when we reach the default block size to predicate the compression rate.
   
   Maybe we could introduce something like a plugin to predicate the compression rate? The default implementation just always returns 1, and we could introduce different algorithms to predicate the compression rate of the current block size.
   
   What do you guys think? For me, the option names are a bit confusing... What does 'size.limit.compressed' mean?
   
   Thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org