[GitHub] [incubator-pinot] richardstartin commented on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator

[GitBox <gi...@apache.org>]

richardstartin commented on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739076353


   Regarding the test failure, this is because the bitmap is being run length encoded incrementally. It becomes deterministic when setting
   
   ```java
   _bitmapWriters[i] = RoaringBitmapWriter.bufferWriter().runCompress(false).get();
   ```
   
   To make the test deterministic, RLE could be disabled, but this is bad for file size. Or the off-heap index creator could run-optimise its bitmaps, but that's bad for speed compared to doing it incrementally. Alternatively the test could be modified to check that the bitmaps are equal after memory mapping. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org