[GitHub] [incubator-doris] imay opened a new issue #1837: Reduce the number of rowset of a table who has HLL column

[GitBox <gi...@apache.org>]

imay opened a new issue #1837: Reduce the number of rowset of a table who has HLL column
URL: https://github.com/apache/incubator-doris/issues/1837
 
 
   When we load data to table who contains HLL columns. Doris will generate many small rowsets whose size is about 100KB, then there will be two many small files.
   
   Because a Memtable is 100MB in size and a HLL column is 16KB, a rowset can only contain thousands of rows of data. However, in the import process, in fact, the HLL column does not use such a large amount of data, generally only a few items, and does not need 16KB. So we need to optimize the HLLContext so that the memory used by it is reduced, which increases the size of the Rowset and thus the number of rowsets.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org