[GitHub] [pinot] Jackie-Jiang commented on issue #10691: Unable to create Star-Tree index for DISTINCTCOUNTBITMAP

["Jackie-Jiang (via GitHub)" <gi...@apache.org>]

Jackie-Jiang commented on issue #10691:
URL: https://github.com/apache/pinot/issues/10691#issuecomment-1524212793

   Is `active_riders` already serialized `RoaringBitmap`? 
   I think it throws exception because star-tree builder is still using the V2 raw forward index, where the chunk size is calculated based on the largest entry. If the largest bitmap size is larger than 2M, then the chunk size will be greater than 2GB.
   
   To fix this, we want to allow star-tree to switch to higher version of raw forward index. Further more, we might want to consider making V4 the default raw forward index version, and deprecate the old versions from the builder side.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org