You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2020/10/15 16:20:00 UTC

[GitHub] [hudi] bvaradar commented on issue #2066: [SUPPORT] Hudi is increasing the storage size big time

bvaradar commented on issue #2066:
URL: https://github.com/apache/hudi/issues/2066#issuecomment-709436915


   THis looks fine to me. Just wondering instead of default implementation of ComplexKeyGenerator which concatenates all the fields, can you try implementing a key-generator which generates md5/sha checkums to study the behavior. Assuming the original length of the record key is large and there is pattern to exploit compression, the checksum approach could perform better.  Also, see if you can try gzip as opposed to snappy.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org