[GitHub] [hbase] apurtell edited a comment on pull request #3748: HBASE-26353 Support loadable dictionaries in hbase-compression-zstd

[GitBox <gi...@apache.org>]

apurtell edited a comment on pull request #3748:
URL: https://github.com/apache/hbase/pull/3748#issuecomment-942711578


   > Haven't read the code yet, but is it possible to copy the dict into the hbase storage so it is controlled by us?
   
   @Apache9 I was thinking about writing the dictionary used to compress values in an HFile or WAL into the HFile or WAL in the metadata section, but there would need to be format extensions to the WAL. Hopefully there can be some re-use of meta blocks for HFiles. But this raises questions. There should be some way for a codec to read and write metadata, but we don't have API support for that. I would consider it future work, but definitely of interest.
   
   Otherwise I think the current scheme is ok. The operator is already in charge of their table schema and compression codec dependencies (like deployment of native link libraries). This is an incremental responsibility... if you put a compression dictionary attribute into your schema, don't lose the dictionary. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hbase.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org