You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2020/12/04 22:43:47 UTC

[GitHub] [incubator-pinot] richardstartin opened a new pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator

richardstartin opened a new pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320


   ## Description
   This PR uses some more efficient APIs from RoaringBitmap.
   - `RoaringBitmapWriter` is optimised for writing row ids into the bitmap sequentially. This class has a few options to control memory usage, but I've used its defaults.
   - A `RoaringBitmap` can be serialized to a `ByteBuffer`, which is generally a lot faster than a `DataOutput`. However, this requires mapping a file outside of the control of `PinotByteBuffer`. The buffer also has to be big endian for backward compatibility with `DataOutput`.
   
   I haven't run any benchmarks on this and am judging this on having used (and created) the same APIs for a similar problem. I tried to run the pinot-perf benchmarks but ran into errors with missing files. I would appreciate some help validating the benefits this change brings.
   
   ## Upgrade Notes
   Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion)
   * [No]
   
   Does this PR fix a zero-downtime upgrade introduced earlier?
   * [ No]
   
   Does this PR otherwise need attention when creating release notes?
   * [ No]
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] siddharthteotia commented on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739128938


   Can you follow one of the perf test classes (module pinot-perf , e.g BenchmarkPinotDataBuffer, BenchmarkDictionary etc) and get some numbers?
   
   Also, you can write a simple test function that can be called while simultaneously profiling with YourKit/VisualVM. This can help reveal how the cpu and memory usage changes with this code.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739096511


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=h1) Report
   > Merging [#6320](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=desc) (e3ec220) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **increase** coverage by `7.67%`.
   > The diff coverage is `68.40%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6320/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #6320      +/-   ##
   ==========================================
   + Coverage   66.44%   74.12%   +7.67%     
   ==========================================
     Files        1075     1281     +206     
     Lines       54773    61823    +7050     
     Branches     8168     8986     +818     
   ==========================================
   + Hits        36396    45825    +9429     
   + Misses      15700    13036    -2664     
   - Partials     2677     2962     +285     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration | `45.71% <52.01%> (?)` | |
   | unittests | `65.36% <45.12%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `52.83% <0.00%> (-13.84%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `44.44% <0.00%> (-4.40%)` | :arrow_down: |
   | [...not/common/assignment/InstancePartitionsUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXNzaWdubWVudC9JbnN0YW5jZVBhcnRpdGlvbnNVdGlscy5qYXZh) | `78.57% <ø> (+5.40%)` | :arrow_up: |
   | [.../apache/pinot/common/exception/QueryException.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZXhjZXB0aW9uL1F1ZXJ5RXhjZXB0aW9uLmphdmE=) | `90.27% <ø> (+5.55%)` | :arrow_up: |
   | [...pinot/common/function/AggregationFunctionType.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vQWdncmVnYXRpb25GdW5jdGlvblR5cGUuamF2YQ==) | `98.27% <ø> (-1.73%)` | :arrow_down: |
   | [.../pinot/common/function/DateTimePatternHandler.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRGF0ZVRpbWVQYXR0ZXJuSGFuZGxlci5qYXZh) | `83.33% <ø> (ø)` | |
   | [...ot/common/function/FunctionDefinitionRegistry.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRnVuY3Rpb25EZWZpbml0aW9uUmVnaXN0cnkuamF2YQ==) | `88.88% <ø> (+44.44%)` | :arrow_up: |
   | [...org/apache/pinot/common/function/FunctionInfo.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRnVuY3Rpb25JbmZvLmphdmE=) | `100.00% <ø> (ø)` | |
   | ... and [1067 more](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=footer). Last update [41a7722...e3ec220](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739241103


   This PR allocates about the same amount as before
   
   before
   ```
   
   Benchmark                                                                           (assignment)  (cardinality)  (numDocs)  Mode  Cnt           Score           Error   Units
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN             10    1000000    ss    5           0.031 ±         0.009    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN             10    1000000    ss    5          11.935 ±         1.702  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN             10    1000000    ss    5     7164704.000 ±       355.929    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN             10    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN             10   10000000    ss    5           0.408 ±         0.019    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN             10   10000000    ss    5          62.589 ±         5.249  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN             10   10000000    ss    5    71375131.200 ±     56512.643    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN             10   10000000    ss    5          70.263 ±       370.500  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN             10   10000000    ss    5    81788928.000 ± 431258859.992    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN             10   10000000    ss    5           0.190 ±         1.632  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN             10   10000000    ss    5      217683.200 ±   1874318.001    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN             10   10000000    ss    5           2.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN             10   10000000    ss    5          16.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN             10  100000000    ss    5           4.193 ±         0.372    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN             10  100000000    ss    5         122.075 ±        12.717  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN             10  100000000    ss    5   712415345.600 ±     55691.481    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN             10  100000000    ss    5         120.923 ±        70.361  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN             10  100000000    ss    5   706320793.600 ± 422412916.188    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN             10  100000000    ss    5           1.682 ±         0.744  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN             10  100000000    ss    5     9832512.000 ±   4758932.013    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN             10  100000000    ss    5          18.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN             10  100000000    ss    5          35.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN           1000    1000000    ss    5           0.092 ±         0.014    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN           1000    1000000    ss    5          22.304 ±         0.194  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN           1000    1000000    ss    5    14236214.400 ±      1270.764    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN           1000    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN           1000   10000000    ss    5           1.010 ±         0.033    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN           1000   10000000    ss    5          76.099 ±         1.422  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN           1000   10000000    ss    5   139322380.800 ±     46579.496    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN           1000   10000000    ss    5          83.271 ±       179.254  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN           1000   10000000    ss    5   152672665.600 ± 328639423.512    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN           1000   10000000    ss    5           0.046 ±         0.399  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN           1000   10000000    ss    5       84628.800 ±    728679.490    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN           1000   10000000    ss    5           4.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN           1000   10000000    ss    5          29.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN           1000  100000000    ss    5          11.633 ±         0.451    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN           1000  100000000    ss    5         101.363 ±         2.794  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN           1000  100000000    ss    5  1385939961.600 ±     55294.403    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN           1000  100000000    ss    5         102.172 ±        30.699  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN           1000  100000000    ss    5  1396703232.000 ± 398076377.621    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN           1000  100000000    ss    5           0.011 ±         0.052  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN           1000  100000000    ss    5      145384.000 ±    710672.251    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN           1000  100000000    ss    5          37.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN           1000  100000000    ss    5          37.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN          10000    1000000    ss    5           0.150 ±         0.045    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN          10000    1000000    ss    5          46.513 ±         4.210  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN          10000    1000000    ss    5    33470916.800 ±     45678.008    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN          10000    1000000    ss    5          41.258 ±       355.241  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN          10000    1000000    ss    5    29779558.400 ± 256410978.784    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN          10000    1000000    ss    5           1.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN          10000    1000000    ss    5          10.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN          10000   10000000    ss    5           1.738 ±         0.386    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN          10000   10000000    ss    5         125.776 ±        12.578  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN          10000   10000000    ss    5   317429830.400 ±     64562.976    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN          10000   10000000    ss    5         110.973 ±       164.658  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN          10000   10000000    ss    5   279340646.400 ± 400867868.240    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN          10000   10000000    ss    5           0.143 ±         0.755  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN          10000   10000000    ss    5      354760.000 ±   1874217.875    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN          10000   10000000    ss    5           6.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN          10000   10000000    ss    5          57.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN          10000  100000000    ss    5          17.156 ±         0.148    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN          10000  100000000    ss    5         160.044 ±         1.436  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN          10000  100000000    ss    5  3116823352.000 ±     45545.903    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN          10000  100000000    ss    5         159.645 ±        26.123  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN          10000  100000000    ss    5  3108818124.800 ± 482114724.008    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN          10000  100000000    ss    5           0.026 ±         0.009  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN          10000  100000000    ss    5      497857.600 ±    172824.399    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN          10000  100000000    ss    5          68.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN          10000  100000000    ss    5          73.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM             10    1000000    ss    5           0.020 ±         0.002    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM             10    1000000    ss    5           2.138 ±         0.034  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM             10    1000000    ss    5     1184286.400 ±       378.164    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM             10    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM             10   10000000    ss    5           0.211 ±         0.001    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM             10   10000000    ss    5           9.535 ±         0.231  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM             10   10000000    ss    5     7565811.200 ±       502.228    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM             10   10000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM             10  100000000    ss    5           2.326 ±         0.034    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM             10  100000000    ss    5          20.862 ±         0.237  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM             10  100000000    ss    5    71689880.000 ±     56609.543    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM             10  100000000    ss    5          20.966 ±       110.550  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM             10  100000000    ss    5    72142028.800 ± 380384094.171    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space       SORTED_UNIFORM             10  100000000    ss    5           0.015 ±         0.127  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm  SORTED_UNIFORM             10  100000000    ss    5       50654.400 ±    436149.660    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM             10  100000000    ss    5           2.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM             10  100000000    ss    5          15.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM           1000    1000000    ss    5           0.074 ±         0.004    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM           1000    1000000    ss    5          12.163 ±         0.138  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM           1000    1000000    ss    5     7452222.400 ±      1019.460    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM           1000    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM           1000   10000000    ss    5           0.275 ±         0.013    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM           1000   10000000    ss    5          52.103 ±         1.439  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM           1000   10000000    ss    5    50582414.400 ±     45991.882    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM           1000   10000000    ss    5          42.028 ±       361.875  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM           1000   10000000    ss    5    41104179.200 ± 353919379.167    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM           1000   10000000    ss    5           1.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM           1000   10000000    ss    5           7.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM           1000  100000000    ss    5           2.099 ±         0.251    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM           1000  100000000    ss    5          35.099 ±         0.579  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM           1000  100000000    ss    5   114648668.800 ±     55423.632    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM           1000  100000000    ss    5          33.538 ±       117.892  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM           1000  100000000    ss    5   109471334.400 ± 384807165.034    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space       SORTED_UNIFORM           1000  100000000    ss    5           0.026 ±         0.223  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm  SORTED_UNIFORM           1000  100000000    ss    5       83841.600 ±    721901.461    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM           1000  100000000    ss    5           3.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM           1000  100000000    ss    5          22.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM          10000    1000000    ss    5           0.093 ±         0.011    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM          10000    1000000    ss    5          20.878 ±         1.225  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM          10000    1000000    ss    5    13544881.600 ±       191.637    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM          10000    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM          10000   10000000    ss    5           0.886 ±         0.244    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM          10000   10000000    ss    5          46.010 ±         1.042  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM          10000   10000000    ss    5    74304364.800 ±     46037.614    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM          10000   10000000    ss    5          24.963 ±       214.936  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM          10000   10000000    ss    5    40684748.800 ± 350307956.931    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM          10000   10000000    ss    5           1.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM          10000   10000000    ss    5           9.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM          10000  100000000    ss    5           3.826 ±         0.052    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM          10000  100000000    ss    5         100.180 ±         1.713  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM          10000  100000000    ss    5   505538409.600 ±     96799.515    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM          10000  100000000    ss    5          99.990 ±        79.926  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM          10000  100000000    ss    5   504574771.200 ± 402985207.107    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space       SORTED_UNIFORM          10000  100000000    ss    5           0.329 ±         2.728  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm  SORTED_UNIFORM          10000  100000000    ss    5     1669273.600 ±  13844467.735    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM          10000  100000000    ss    5          13.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM          10000  100000000    ss    5          28.000                      ms
   ```
   
   after
   ```
   Benchmark                                                                           (assignment)  (cardinality)  (numDocs)  Mode  Cnt           Score           Error   Units
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN             10    1000000    ss    5           0.029 ±         0.022    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN             10    1000000    ss    5          12.403 ±         1.117  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN             10    1000000    ss    5     7146006.400 ±     30985.905    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN             10    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN             10   10000000    ss    5           0.188 ±         0.026    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN             10   10000000    ss    5          87.108 ±         4.467  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN             10   10000000    ss    5    71108766.400 ±     78261.812    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN             10   10000000    ss    5          95.241 ±       502.203  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN             10   10000000    ss    5    78014054.400 ± 411385221.551    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN             10   10000000    ss    5           1.101 ±         9.482  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN             10   10000000    ss    5      899424.000 ±   7744311.890    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN             10   10000000    ss    5           2.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN             10   10000000    ss    5          25.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN             10  100000000    ss    5           1.947 ±         0.116    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN             10  100000000    ss    5         208.005 ±         7.371  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN             10  100000000    ss    5   709829240.000 ±     51497.263    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN             10  100000000    ss    5         201.399 ±        76.815  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN             10  100000000    ss    5   687026995.200 ± 247098882.679    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen                 ROUND_ROBIN             10  100000000    ss    5           2.224 ±        17.872  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen.norm            ROUND_ROBIN             10  100000000    ss    5     7549747.200 ±  60592304.093    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN             10  100000000    ss    5           4.788 ±        16.708  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN             10  100000000    ss    5    16357785.600 ±  57215688.521    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN             10  100000000    ss    5          17.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN             10  100000000    ss    5         256.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN           1000    1000000    ss    5           0.040 ±         0.023    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN           1000    1000000    ss    5          20.222 ±         1.683  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN           1000    1000000    ss    5    11859302.400 ±     30898.399    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN           1000    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN           1000   10000000    ss    5           0.293 ±         0.080    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN           1000   10000000    ss    5         123.802 ±        11.035  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN           1000   10000000    ss    5   113905795.200 ±     74048.230    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN           1000   10000000    ss    5         108.134 ±       386.562  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN           1000   10000000    ss    5   100663296.000 ± 360192977.778    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN           1000   10000000    ss    5           6.363 ±        34.666  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN           1000   10000000    ss    5     5872025.600 ±  32124407.719    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN           1000   10000000    ss    5           3.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN           1000   10000000    ss    5          35.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN           1000  100000000    ss    5           2.898 ±         0.114    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN           1000  100000000    ss    5         255.212 ±         6.003  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN           1000  100000000    ss    5  1129770475.200 ±    124788.799    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN           1000  100000000    ss    5         257.389 ±        62.089  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN           1000  100000000    ss    5  1139592396.800 ± 287647003.971    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen                 ROUND_ROBIN           1000  100000000    ss    5           6.228 ±        32.860  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen.norm            ROUND_ROBIN           1000  100000000    ss    5    27472691.200 ± 144918864.230    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN           1000  100000000    ss    5           7.259 ±        12.981  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN           1000  100000000    ss    5    32086425.600 ±  56930036.769    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN           1000  100000000    ss    5          24.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN           1000  100000000    ss    5        1023.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN          10000    1000000    ss    5           0.066 ±         0.037    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN          10000    1000000    ss    5          72.601 ±         4.420  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN          10000    1000000    ss    5    45097956.800 ±     48245.104    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN          10000    1000000    ss    5          74.815 ±       644.181  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN          10000    1000000    ss    5    47395635.200 ± 408090712.713    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN          10000    1000000    ss    5           1.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN          10000    1000000    ss    5          14.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN          10000   10000000    ss    5           0.557 ±         0.267    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN          10000   10000000    ss    5         186.074 ±        47.551  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN          10000   10000000    ss    5   223440820.800 ±     78325.490    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN          10000   10000000    ss    5         138.524 ±       337.002  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN          10000   10000000    ss    5   171127603.200 ± 416904247.274    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN          10000   10000000    ss    5           1.756 ±        15.119  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN          10000   10000000    ss    5     2097152.000 ±  18057111.182    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN          10000   10000000    ss    5           5.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN          10000   10000000    ss    5         298.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN          10000  100000000    ss    5           5.260 ±         0.325    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN          10000  100000000    ss    5         310.016 ±        14.954  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN          10000  100000000    ss    5  2143068185.600 ±     64550.566    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN          10000  100000000    ss    5         306.636 ±        42.750  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN          10000  100000000    ss    5  2120220672.000 ± 339646533.455    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN          10000  100000000    ss    5           8.017 ±         4.673  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN          10000  100000000    ss    5    55364812.800 ±  30537345.918    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN          10000  100000000    ss    5          39.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN          10000  100000000    ss    5        4087.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM             10    1000000    ss    5           0.017 ±         0.006    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM             10    1000000    ss    5           2.146 ±         0.059  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM             10    1000000    ss    5     1175939.200 ±     30372.447    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM             10    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM             10   10000000    ss    5           0.155 ±         0.001    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM             10   10000000    ss    5          10.358 ±         0.065  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM             10   10000000    ss    5     7563580.800 ±     30697.759    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM             10   10000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM             10  100000000    ss    5           1.548 ±         0.023    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM             10  100000000    ss    5          28.135 ±         0.251  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM             10  100000000    ss    5    71579300.800 ±     78187.548    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM             10  100000000    ss    5          28.278 ±       149.101  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM             10  100000000    ss    5    72142028.800 ± 380384094.171    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space       SORTED_UNIFORM             10  100000000    ss    5           0.028 ±         0.239  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm  SORTED_UNIFORM             10  100000000    ss    5       70888.000 ±    610367.059    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM             10  100000000    ss    5           2.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM             10  100000000    ss    5          17.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM           1000    1000000    ss    5           0.023 ±         0.032    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM           1000    1000000    ss    5          13.814 ±         1.193  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM           1000    1000000    ss    5     7669910.400 ±    300898.435    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM           1000    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM           1000   10000000    ss    5           0.181 ±         0.037    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM           1000   10000000    ss    5          66.605 ±         4.336  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM           1000   10000000    ss    5    50803308.800 ±     76359.327    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM           1000   10000000    ss    5          62.270 ±       536.163  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM           1000   10000000    ss    5    47395635.200 ± 408090712.713    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM           1000   10000000    ss    5           1.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM           1000   10000000    ss    5           8.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM           1000  100000000    ss    5           1.568 ±         0.020    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM           1000  100000000    ss    5          44.731 ±         0.340  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM           1000  100000000    ss    5   114796513.600 ±     73178.185    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM           1000  100000000    ss    5          43.084 ±       151.447  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM           1000  100000000    ss    5   110729625.600 ± 389230235.896    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space       SORTED_UNIFORM           1000  100000000    ss    5           0.032 ±         0.275  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm  SORTED_UNIFORM           1000  100000000    ss    5       82238.400 ±    708097.426    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM           1000  100000000    ss    5           3.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM           1000  100000000    ss    5          24.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM          10000    1000000    ss    5           0.034 ±         0.037    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM          10000    1000000    ss    5          27.006 ±         2.034  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM          10000    1000000    ss    5    15331587.200 ±     30511.819    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM          10000    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM          10000   10000000    ss    5           0.186 ±         0.016    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM          10000   10000000    ss    5          99.771 ±         3.199  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM          10000   10000000    ss    5    76239598.400 ±     76412.193    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM          10000   10000000    ss    5          53.525 ±       460.870  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM          10000   10000000    ss    5    41523609.600 ± 357530801.403    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space       SORTED_UNIFORM          10000   10000000    ss    5           0.062 ±         0.531  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm  SORTED_UNIFORM          10000   10000000    ss    5       47827.200 ±    411806.616    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM          10000   10000000    ss    5           1.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM          10000   10000000    ss    5           9.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM          10000  100000000    ss    5           1.736 ±         0.045    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM          10000  100000000    ss    5         184.429 ±         5.794  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM          10000  100000000    ss    5   507845612.800 ±     59214.207    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM          10000  100000000    ss    5         171.683 ±        16.667  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM          10000  100000000    ss    5   472698060.800 ±  32099022.960    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space       SORTED_UNIFORM          10000  100000000    ss    5           0.901 ±         4.227  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm  SORTED_UNIFORM          10000  100000000    ss    5     2492544.000 ±  11744141.868    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM          10000  100000000    ss    5          10.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM          10000  100000000    ss    5          49.000                      ms
   ```
   
   Allocation can be reduced and get a speed gain by guessing the approximate sizes of the bitmaps, and by reusing a single writer.
   
   latest commit
   ```
   Benchmark                                          (assignment)  (cardinality)  (numDocs)  Mode  Cnt  Score   Error  Units
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN             10    1000000    ss    5  0.021 ± 0.038   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN             10   10000000    ss    5  0.175 ± 0.015   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN             10  100000000    ss    5  1.780 ± 0.117   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN           1000    1000000    ss    5  0.024 ± 0.003   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN           1000   10000000    ss    5  0.243 ± 0.035   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN           1000  100000000    ss    5  2.711 ± 0.219   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN          10000    1000000    ss    5  0.052 ± 0.024   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN          10000   10000000    ss    5  0.539 ± 0.107   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN          10000  100000000    ss    5  4.500 ± 0.246   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM             10    1000000    ss    5  0.016 ± 0.003   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM             10   10000000    ss    5  0.148 ± 0.005   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM             10  100000000    ss    5  1.412 ± 0.026   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM           1000    1000000    ss    5  0.021 ± 0.026   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM           1000   10000000    ss    5  0.165 ± 0.019   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM           1000  100000000    ss    5  1.437 ± 0.021   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM          10000    1000000    ss    5  0.022 ± 0.005   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM          10000   10000000    ss    5  0.172 ± 0.022   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM          10000  100000000    ss    5  1.581 ± 0.095  s/op
   ```
   
   ```
   Benchmark                                                                           (assignment)  (cardinality)  (numDocs)  Mode  Cnt           Score           Error   Units
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN             10    1000000    ss    5           0.017 ±         0.002    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN             10    1000000    ss    5          12.633 ±         0.155  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN             10    1000000    ss    5     6986539.200 ±     30442.010    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN             10    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN             10   10000000    ss    5           0.176 ±         0.012    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN             10   10000000    ss    5          87.350 ±         1.202  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN             10   10000000    ss    5    69659446.400 ±     52302.495    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN             10   10000000    ss    5         110.528 ±       582.795  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN             10   10000000    ss    5    88499814.400 ± 466642710.250    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN             10   10000000    ss    5           2.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN             10   10000000    ss    5          12.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN             10  100000000    ss    5           1.783 ±         0.104    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN             10  100000000    ss    5         214.505 ±         7.603  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN             10  100000000    ss    5   695378540.800 ±    139270.627    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN             10  100000000    ss    5         214.143 ±       104.774  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN             10  100000000    ss    5   694576742.400 ± 352155335.840    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen                 ROUND_ROBIN             10  100000000    ss    5           1.321 ±        11.370  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen.norm            ROUND_ROBIN             10  100000000    ss    5     4256358.400 ±  36648529.462    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN             10  100000000    ss    5           3.111 ±         7.794  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN             10  100000000    ss    5    10067219.200 ±  25155045.841    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN             10  100000000    ss    5          23.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN             10  100000000    ss    5         275.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN           1000    1000000    ss    5           0.025 ±         0.022    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN           1000    1000000    ss    5          14.770 ±         1.072  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN           1000    1000000    ss    5     8325694.400 ±     30481.121    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN           1000    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN           1000   10000000    ss    5           0.244 ±         0.040    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN           1000   10000000    ss    5          92.415 ±         4.679  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN           1000   10000000    ss    5    80292582.400 ±     51274.216    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN           1000   10000000    ss    5          69.550 ±       366.944  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN           1000   10000000    ss    5    60817408.000 ± 320787006.897    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN           1000   10000000    ss    5           2.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN           1000   10000000    ss    5          22.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN           1000  100000000    ss    5           2.429 ±         0.248    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN           1000  100000000    ss    5         200.418 ±        13.277  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN           1000  100000000    ss    5   787876115.200 ±    256741.280    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN           1000  100000000    ss    5         193.777 ±       130.177  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN           1000  100000000    ss    5   762943897.600 ± 544097645.991    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen                 ROUND_ROBIN           1000  100000000    ss    5           1.467 ±        12.630  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen.norm            ROUND_ROBIN           1000  100000000    ss    5     5872025.600 ±  50559911.310    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN           1000  100000000    ss    5           4.343 ±         9.498  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN           1000  100000000    ss    5    17196646.400 ±  38474784.725    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN           1000  100000000    ss    5          23.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN           1000  100000000    ss    5         746.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN          10000    1000000    ss    5           0.054 ±         0.025    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN          10000    1000000    ss    5          29.477 ±         2.118  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN          10000    1000000    ss    5    17577993.600 ±     48406.649    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN          10000    1000000    ss    5          30.144 ±       259.553  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN          10000    1000000    ss    5    18035507.200 ± 155291156.165    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN          10000    1000000    ss    5           1.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN          10000    1000000    ss    5          12.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN          10000   10000000    ss    5           0.481 ±         0.050    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN          10000   10000000    ss    5         205.551 ±        10.013  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN          10000   10000000    ss    5   230199414.400 ±    124257.456    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN          10000   10000000    ss    5         200.550 ±       150.026  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN          10000   10000000    ss    5   224395264.000 ± 162413652.372    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen                 ROUND_ROBIN          10000   10000000    ss    5           1.853 ±        15.957  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen.norm            ROUND_ROBIN          10000   10000000    ss    5     2097152.000 ±  18057111.182    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN          10000   10000000    ss    5          11.056 ±        41.589  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN          10000   10000000    ss    5    12379680.000 ±  46553344.432    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN          10000   10000000    ss    5          11.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN          10000   10000000    ss    5         304.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN          10000  100000000    ss    5           4.505 ±         0.168    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN          10000  100000000    ss    5         220.361 ±         6.174  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN          10000  100000000    ss    5  1348977696.000 ±     48374.015    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN          10000  100000000    ss    5         220.473 ±        89.136  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN          10000  100000000    ss    5  1350146457.600 ± 565202002.425    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen                 ROUND_ROBIN          10000  100000000    ss    5           0.719 ±         6.193  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen.norm            ROUND_ROBIN          10000  100000000    ss    5     4404019.200 ±  37919933.482    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN          10000  100000000    ss    5           5.615 ±         5.115  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN          10000  100000000    ss    5    34393292.800 ±  31587043.999    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN          10000  100000000    ss    5          36.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN          10000  100000000    ss    5        2728.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM             10    1000000    ss    5           0.022 ±         0.037    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM             10    1000000    ss    5           2.066 ±         0.233  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM             10    1000000    ss    5     1148363.200 ±     30594.344    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM             10    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM             10   10000000    ss    5           0.148 ±         0.003    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM             10   10000000    ss    5          10.223 ±         0.105  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM             10   10000000    ss    5     7390347.200 ±     30613.010    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM             10   10000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM             10  100000000    ss    5           1.410 ±         0.015    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM             10  100000000    ss    5          29.172 ±         0.183  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM             10  100000000    ss    5    70052542.400 ±    137755.129    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM             10  100000000    ss    5          29.999 ±       158.176  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM             10  100000000    ss    5    72142028.800 ± 380384094.171    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space       SORTED_UNIFORM             10  100000000    ss    5           0.034 ±         0.293  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm  SORTED_UNIFORM             10  100000000    ss    5       81944.000 ±    705562.553    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM             10  100000000    ss    5           2.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM             10  100000000    ss    5          16.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM           1000    1000000    ss    5           0.025 ±         0.040    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM           1000    1000000    ss    5          12.373 ±         1.158  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM           1000    1000000    ss    5     6932864.000 ±     30927.743    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM           1000    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM           1000   10000000    ss    5           0.164 ±         0.015    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM           1000   10000000    ss    5          66.973 ±         1.648  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM           1000   10000000    ss    5    49529336.000 ±     76261.103    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM           1000   10000000    ss    5          55.023 ±       473.763  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM           1000   10000000    ss    5    41104179.200 ± 353919379.167    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM           1000   10000000    ss    5           1.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM           1000   10000000    ss    5           8.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM           1000  100000000    ss    5           1.434 ±         0.018    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM           1000  100000000    ss    5          46.177 ±         0.422  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM           1000  100000000    ss    5   112162459.200 ±     73253.706    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM           1000  100000000    ss    5          44.479 ±       156.352  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM           1000  100000000    ss    5   108213043.200 ± 380384094.171    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space       SORTED_UNIFORM           1000  100000000    ss    5           0.033 ±         0.282  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm  SORTED_UNIFORM           1000  100000000    ss    5       80251.200 ±    686689.981    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM           1000  100000000    ss    5           3.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM           1000  100000000    ss    5          23.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM          10000    1000000    ss    5           0.023 ±         0.011    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM          10000    1000000    ss    5          20.115 ±         0.796  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM          10000    1000000    ss    5    11170392.000 ±     30879.694    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM          10000    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM          10000   10000000    ss    5           0.167 ±         0.016    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM          10000   10000000    ss    5          91.921 ±         7.404  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM          10000   10000000    ss    5    69227684.800 ±     52043.344    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM          10000   10000000    ss    5         114.528 ±       604.014  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM          10000   10000000    ss    5    87241523.200 ± 459999369.696    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM          10000   10000000    ss    5           2.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM          10000   10000000    ss    5          17.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM          10000  100000000    ss    5           1.603 ±         0.123    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM          10000  100000000    ss    5         190.115 ±         9.171  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM          10000  100000000    ss    5   495109988.800 ±    101261.939    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM          10000  100000000    ss    5         197.630 ±       142.159  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM          10000  100000000    ss    5   514641100.800 ± 368630518.221    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space       SORTED_UNIFORM          10000  100000000    ss    5           0.842 ±         4.357  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm  SORTED_UNIFORM          10000  100000000    ss    5     2221792.000 ±  11637986.706    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM          10000  100000000    ss    5          11.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM          10000  100000000    ss    5          49.000                      ms
   
   ```
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] codecov-io commented on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
codecov-io commented on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739096511


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=h1) Report
   > Merging [#6320](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=desc) (0a7af36) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **decrease** coverage by `1.10%`.
   > The diff coverage is `45.12%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6320/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #6320      +/-   ##
   ==========================================
   - Coverage   66.44%   65.34%   -1.11%     
   ==========================================
     Files        1075     1281     +206     
     Lines       54773    61813    +7040     
     Branches     8168     8986     +818     
   ==========================================
   + Hits        36396    40393    +3997     
   - Misses      15700    18532    +2832     
   - Partials     2677     2888     +211     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `65.34% <45.12%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...e/pinot/broker/api/resources/PinotBrokerDebug.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYXBpL3Jlc291cmNlcy9QaW5vdEJyb2tlckRlYnVnLmphdmE=) | `0.00% <0.00%> (-79.32%)` | :arrow_down: |
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `71.42% <ø> (-28.58%)` | :arrow_down: |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `33.96% <0.00%> (-32.71%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ava/org/apache/pinot/client/AbstractResultSet.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Fic3RyYWN0UmVzdWx0U2V0LmphdmE=) | `66.66% <0.00%> (+9.52%)` | :arrow_up: |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `35.55% <0.00%> (-13.29%)` | :arrow_down: |
   | [...inot/client/JsonAsyncHttpPinotClientTransport.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0pzb25Bc3luY0h0dHBQaW5vdENsaWVudFRyYW5zcG9ydC5qYXZh) | `10.90% <0.00%> (-51.10%)` | :arrow_down: |
   | [...not/common/assignment/InstancePartitionsUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXNzaWdubWVudC9JbnN0YW5jZVBhcnRpdGlvbnNVdGlscy5qYXZh) | `73.80% <ø> (+0.63%)` | :arrow_up: |
   | [.../apache/pinot/common/exception/QueryException.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZXhjZXB0aW9uL1F1ZXJ5RXhjZXB0aW9uLmphdmE=) | `90.27% <ø> (+5.55%)` | :arrow_up: |
   | [...pinot/common/function/AggregationFunctionType.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vQWdncmVnYXRpb25GdW5jdGlvblR5cGUuamF2YQ==) | `98.27% <ø> (-1.73%)` | :arrow_down: |
   | ... and [1128 more](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=footer). Last update [41a7722...0a7af36](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739096511


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=h1) Report
   > Merging [#6320](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=desc) (ba99103) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **decrease** coverage by `1.10%`.
   > The diff coverage is `45.12%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6320/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #6320      +/-   ##
   ==========================================
   - Coverage   66.44%   65.34%   -1.11%     
   ==========================================
     Files        1075     1281     +206     
     Lines       54773    61813    +7040     
     Branches     8168     8986     +818     
   ==========================================
   + Hits        36396    40392    +3996     
   - Misses      15700    18529    +2829     
   - Partials     2677     2892     +215     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `65.34% <45.12%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...e/pinot/broker/api/resources/PinotBrokerDebug.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYXBpL3Jlc291cmNlcy9QaW5vdEJyb2tlckRlYnVnLmphdmE=) | `0.00% <0.00%> (-79.32%)` | :arrow_down: |
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `71.42% <ø> (-28.58%)` | :arrow_down: |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `33.96% <0.00%> (-32.71%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ava/org/apache/pinot/client/AbstractResultSet.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Fic3RyYWN0UmVzdWx0U2V0LmphdmE=) | `66.66% <0.00%> (+9.52%)` | :arrow_up: |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `35.55% <0.00%> (-13.29%)` | :arrow_down: |
   | [...inot/client/JsonAsyncHttpPinotClientTransport.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0pzb25Bc3luY0h0dHBQaW5vdENsaWVudFRyYW5zcG9ydC5qYXZh) | `10.90% <0.00%> (-51.10%)` | :arrow_down: |
   | [...not/common/assignment/InstancePartitionsUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXNzaWdubWVudC9JbnN0YW5jZVBhcnRpdGlvbnNVdGlscy5qYXZh) | `73.80% <ø> (+0.63%)` | :arrow_up: |
   | [.../apache/pinot/common/exception/QueryException.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZXhjZXB0aW9uL1F1ZXJ5RXhjZXB0aW9uLmphdmE=) | `90.27% <ø> (+5.55%)` | :arrow_up: |
   | [...pinot/common/function/AggregationFunctionType.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vQWdncmVnYXRpb25GdW5jdGlvblR5cGUuamF2YQ==) | `98.27% <ø> (-1.73%)` | :arrow_down: |
   | ... and [1129 more](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=footer). Last update [41a7722...18d06e3](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r536890866



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +188,49 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {

Review comment:
       From the code, I feel the benefit of serializing to `ByteBuffer` is to avoid the reverseBytes caused by BE to LE. How could it cause 20x performance difference? (Or the output stream is not buffered maybe?)




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r536458126



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +188,49 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      // map the offsets buffer
+      final int startOfBitmaps = (_cardinality + 1) * Integer.BYTES;
+      int bitmapOffset = startOfBitmaps;
+      offsetBuffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, bitmapOffset).order(LITTLE_ENDIAN);
+      offsetBuffer.putInt(reverseBytes(bitmapOffset));
+      MutableRoaringBitmap[] bitmaps = new MutableRoaringBitmap[_cardinality];

Review comment:
       This is a allocation, but it's offset against the 16KB in stream buffers no longer allocated, and is a lot smaller than the bitmaps being materialised here anyway. This pattern where bitmaps are allocated just to obtain their serialized bytes could probably be supported by roaringbitmap without actually creating the intermediate bitmaps without much complication.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r536911160



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +188,51 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      // map the offsets buffer
+      final int startOfBitmaps = (_cardinality + 1) * Integer.BYTES;
+      int bitmapOffset = startOfBitmaps;
+      offsetBuffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, bitmapOffset).order(LITTLE_ENDIAN);
+      offsetBuffer.putInt(reverseBytes(bitmapOffset));
+      RoaringBitmap[] bitmaps = new RoaringBitmap[_cardinality];
+      RoaringBitmapWriter<RoaringBitmap> writer = RoaringBitmapWriter.writer()
+              .initialCapacity(((_nextDocId - 1) >>> 16) / _cardinality).get();

Review comment:
       This is just a heuristic that the keys of the bitmap will be evenly distributed across the dictionary ids. It won't always be perfect, but is never that bad. I can provide more justification.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r537835291



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OnHeapBitmapInvertedIndexCreator.java
##########
@@ -18,69 +18,85 @@
  */
 package org.apache.pinot.core.segment.creator.impl.inv;
 
-import com.google.common.base.Preconditions;
-import java.io.BufferedOutputStream;
-import java.io.DataOutputStream;
+
 import java.io.File;
-import java.io.FileOutputStream;
 import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.ByteBuffer;
+import java.nio.ByteOrder;
+import java.nio.channels.FileChannel;
+
 import org.apache.commons.io.FileUtils;
 import org.apache.pinot.core.segment.creator.DictionaryBasedInvertedIndexCreator;
 import org.apache.pinot.core.segment.creator.impl.V1Constants;
-import org.roaringbitmap.buffer.MutableRoaringBitmap;
+import org.apache.pinot.core.util.CleanerUtil;
+import org.roaringbitmap.RoaringBitmap;
+import org.roaringbitmap.RoaringBitmapWriter;
 
 
 /**
  * Implementation of {@link DictionaryBasedInvertedIndexCreator} that uses on-heap memory.
  */
 public final class OnHeapBitmapInvertedIndexCreator implements DictionaryBasedInvertedIndexCreator {
   private final File _invertedIndexFile;
-  private final MutableRoaringBitmap[] _bitmaps;
+  private final RoaringBitmapWriter<RoaringBitmap>[] _bitmapWriters;
   private int _nextDocId;
 
+  @SuppressWarnings("unchecked")
   public OnHeapBitmapInvertedIndexCreator(File indexDir, String columnName, int cardinality) {
     _invertedIndexFile = new File(indexDir, columnName + V1Constants.Indexes.BITMAP_INVERTED_INDEX_FILE_EXTENSION);
-    _bitmaps = new MutableRoaringBitmap[cardinality];
+    _bitmapWriters = new RoaringBitmapWriter[cardinality];
     for (int i = 0; i < cardinality; i++) {
-      _bitmaps[i] = new MutableRoaringBitmap();
+      _bitmapWriters[i] = RoaringBitmapWriter.writer().runCompress(false).get();
     }
   }
 
   @Override
   public void add(int dictId) {
-    _bitmaps[dictId].add(_nextDocId++);
+    _bitmapWriters[dictId].add(_nextDocId++);
   }
 
   @Override
   public void add(int[] dictIds, int length) {
     for (int i = 0; i < length; i++) {
-      _bitmaps[dictIds[i]].add(_nextDocId);
+      _bitmapWriters[dictIds[i]].add(_nextDocId);
     }
     _nextDocId++;
   }
 
   @Override
   public void seal()
       throws IOException {
-    try (DataOutputStream out = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)))) {
+    // calculate file size

Review comment:
       this comment is now misleading, will remove it




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] kishoreg commented on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
kishoreg commented on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739308749


   This looks promising both from performance and memory allocation. @Jackie-Jiang can you please look into this?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] Jackie-Jiang commented on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-740193290


   I'm working on a feature which requires extracting the common part of the on-heap and off-heap inverted index creator. Can I add a commit to this PR?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r537847693



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +184,45 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      // map the offsets buffer
+      int startOfBitmaps = (_cardinality + 1) * Integer.BYTES;
+      offsetBuffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, startOfBitmaps)
+              .order(ByteOrder.BIG_ENDIAN);
+      offsetBuffer.putInt(startOfBitmaps);
+      bitmapBuffer = channel.map(FileChannel.MapMode.READ_WRITE, startOfBitmaps, Integer.MAX_VALUE - startOfBitmaps)
+              .order(ByteOrder.LITTLE_ENDIAN);
+      RoaringBitmapWriter<RoaringBitmap> writer = RoaringBitmapWriter.writer().get();

Review comment:
       The bitmap could be dense (when cardinality is very low), but won't be contiguous. What is the old behavior? We don't explicitly call `runOptimize()` on the `MutableRoaringBitmap`, will it perform RLE automatically? If not, we can disable it here to keep the same behavior and have better performance.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r536915214



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OnHeapBitmapInvertedIndexCreator.java
##########
@@ -19,68 +19,85 @@
 package org.apache.pinot.core.segment.creator.impl.inv;
 
 import com.google.common.base.Preconditions;
-import java.io.BufferedOutputStream;
-import java.io.DataOutputStream;
+
+
 import java.io.File;
-import java.io.FileOutputStream;
 import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.ByteBuffer;
+import java.nio.channels.FileChannel;
+
 import org.apache.commons.io.FileUtils;
 import org.apache.pinot.core.segment.creator.DictionaryBasedInvertedIndexCreator;
 import org.apache.pinot.core.segment.creator.impl.V1Constants;
+import org.apache.pinot.core.util.CleanerUtil;
+import org.roaringbitmap.RoaringBitmapWriter;
 import org.roaringbitmap.buffer.MutableRoaringBitmap;
 
+import static java.nio.ByteOrder.LITTLE_ENDIAN;
+
 
 /**
  * Implementation of {@link DictionaryBasedInvertedIndexCreator} that uses on-heap memory.
  */
 public final class OnHeapBitmapInvertedIndexCreator implements DictionaryBasedInvertedIndexCreator {
   private final File _invertedIndexFile;
-  private final MutableRoaringBitmap[] _bitmaps;
+  private final RoaringBitmapWriter<MutableRoaringBitmap>[] _bitmapWriters;
   private int _nextDocId;
 
+  @SuppressWarnings("unchecked")
   public OnHeapBitmapInvertedIndexCreator(File indexDir, String columnName, int cardinality) {
     _invertedIndexFile = new File(indexDir, columnName + V1Constants.Indexes.BITMAP_INVERTED_INDEX_FILE_EXTENSION);
-    _bitmaps = new MutableRoaringBitmap[cardinality];
+    _bitmapWriters = new RoaringBitmapWriter[cardinality];
     for (int i = 0; i < cardinality; i++) {
-      _bitmaps[i] = new MutableRoaringBitmap();
+      _bitmapWriters[i] = RoaringBitmapWriter.bufferWriter().get();
     }
   }
 
   @Override
   public void add(int dictId) {
-    _bitmaps[dictId].add(_nextDocId++);
+    _bitmapWriters[dictId].add(_nextDocId++);
   }
 
   @Override
   public void add(int[] dictIds, int length) {
     for (int i = 0; i < length; i++) {
-      _bitmaps[dictIds[i]].add(_nextDocId);
+      _bitmapWriters[dictIds[i]].add(_nextDocId);
     }
     _nextDocId++;
   }
 
   @Override
   public void seal()
       throws IOException {
-    try (DataOutputStream out = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)))) {
+    // calculate file size
+    int size = (_bitmapWriters.length + 1) * Integer.BYTES;
+    for (RoaringBitmapWriter<MutableRoaringBitmap> writer : _bitmapWriters) {
+      size += writer.get().serializedSizeInBytes();
+      // Check for int overflow
+      Preconditions.checkState(size > 0, "Inverted index file: %s exceeds 2GB limit", _invertedIndexFile);
+    }
+    ByteBuffer buffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      buffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, size).order(LITTLE_ENDIAN);

Review comment:
       Not sure I completely follow. You are writing big endian bytes by doing reverseBytes() anyway. So why not just choose the byte order as BE. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-740213456


   > I'm working on a feature which requires extracting the common part of the on-heap and off-heap inverted index creator. Can I add a commit to this PR?
   
   Since this is on a fork, I'm not sure if that's possible. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739096511


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=h1) Report
   > Merging [#6320](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=desc) (03de308) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **decrease** coverage by `1.07%`.
   > The diff coverage is `45.12%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6320/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #6320      +/-   ##
   ==========================================
   - Coverage   66.44%   65.37%   -1.08%     
   ==========================================
     Files        1075     1281     +206     
     Lines       54773    61823    +7050     
     Branches     8168     8986     +818     
   ==========================================
   + Hits        36396    40417    +4021     
   - Misses      15700    18515    +2815     
   - Partials     2677     2891     +214     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `65.37% <45.12%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...e/pinot/broker/api/resources/PinotBrokerDebug.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYXBpL3Jlc291cmNlcy9QaW5vdEJyb2tlckRlYnVnLmphdmE=) | `0.00% <0.00%> (-79.32%)` | :arrow_down: |
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `71.42% <ø> (-28.58%)` | :arrow_down: |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `33.96% <0.00%> (-32.71%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ava/org/apache/pinot/client/AbstractResultSet.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Fic3RyYWN0UmVzdWx0U2V0LmphdmE=) | `66.66% <0.00%> (+9.52%)` | :arrow_up: |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `35.55% <0.00%> (-13.29%)` | :arrow_down: |
   | [...inot/client/JsonAsyncHttpPinotClientTransport.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0pzb25Bc3luY0h0dHBQaW5vdENsaWVudFRyYW5zcG9ydC5qYXZh) | `10.90% <0.00%> (-51.10%)` | :arrow_down: |
   | [...not/common/assignment/InstancePartitionsUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXNzaWdubWVudC9JbnN0YW5jZVBhcnRpdGlvbnNVdGlscy5qYXZh) | `73.80% <ø> (+0.63%)` | :arrow_up: |
   | [.../apache/pinot/common/exception/QueryException.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZXhjZXB0aW9uL1F1ZXJ5RXhjZXB0aW9uLmphdmE=) | `90.27% <ø> (+5.55%)` | :arrow_up: |
   | [...pinot/common/function/AggregationFunctionType.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vQWdncmVnYXRpb25GdW5jdGlvblR5cGUuamF2YQ==) | `98.27% <ø> (-1.73%)` | :arrow_down: |
   | ... and [1128 more](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=footer). Last update [41a7722...e3ec220](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r536565284



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +188,49 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {

Review comment:
       Two places.
   
   - We found that serializing to `ByteBuffer` could be up to 20x faster than to `DataOutput`, which was why the API was added. 
   - If the bitmaps being materialised prior to serialisation below get reasonably large, using `RoaringBitmapWriter` improves the efficiency of construction. This is because when you call `RoaringBitmap.add`, it does a binary search in the high 16 bits of the value to figure out where to put the low 16 bits, but `RoaringBitmapWriter` assumes all adds are appends and optimises for it.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r536917433



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OnHeapBitmapInvertedIndexCreator.java
##########
@@ -19,68 +19,85 @@
 package org.apache.pinot.core.segment.creator.impl.inv;
 
 import com.google.common.base.Preconditions;
-import java.io.BufferedOutputStream;
-import java.io.DataOutputStream;
+
+
 import java.io.File;
-import java.io.FileOutputStream;
 import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.ByteBuffer;
+import java.nio.channels.FileChannel;
+
 import org.apache.commons.io.FileUtils;
 import org.apache.pinot.core.segment.creator.DictionaryBasedInvertedIndexCreator;
 import org.apache.pinot.core.segment.creator.impl.V1Constants;
+import org.apache.pinot.core.util.CleanerUtil;
+import org.roaringbitmap.RoaringBitmapWriter;
 import org.roaringbitmap.buffer.MutableRoaringBitmap;
 
+import static java.nio.ByteOrder.LITTLE_ENDIAN;
+
 
 /**
  * Implementation of {@link DictionaryBasedInvertedIndexCreator} that uses on-heap memory.
  */
 public final class OnHeapBitmapInvertedIndexCreator implements DictionaryBasedInvertedIndexCreator {
   private final File _invertedIndexFile;
-  private final MutableRoaringBitmap[] _bitmaps;
+  private final RoaringBitmapWriter<MutableRoaringBitmap>[] _bitmapWriters;
   private int _nextDocId;
 
+  @SuppressWarnings("unchecked")
   public OnHeapBitmapInvertedIndexCreator(File indexDir, String columnName, int cardinality) {
     _invertedIndexFile = new File(indexDir, columnName + V1Constants.Indexes.BITMAP_INVERTED_INDEX_FILE_EXTENSION);
-    _bitmaps = new MutableRoaringBitmap[cardinality];
+    _bitmapWriters = new RoaringBitmapWriter[cardinality];
     for (int i = 0; i < cardinality; i++) {
-      _bitmaps[i] = new MutableRoaringBitmap();
+      _bitmapWriters[i] = RoaringBitmapWriter.bufferWriter().get();
     }
   }
 
   @Override
   public void add(int dictId) {
-    _bitmaps[dictId].add(_nextDocId++);
+    _bitmapWriters[dictId].add(_nextDocId++);
   }
 
   @Override
   public void add(int[] dictIds, int length) {
     for (int i = 0; i < length; i++) {
-      _bitmaps[dictIds[i]].add(_nextDocId);
+      _bitmapWriters[dictIds[i]].add(_nextDocId);
     }
     _nextDocId++;
   }
 
   @Override
   public void seal()
       throws IOException {
-    try (DataOutputStream out = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)))) {
+    // calculate file size
+    int size = (_bitmapWriters.length + 1) * Integer.BYTES;
+    for (RoaringBitmapWriter<MutableRoaringBitmap> writer : _bitmapWriters) {
+      size += writer.get().serializedSizeInBytes();
+      // Check for int overflow
+      Preconditions.checkState(size > 0, "Inverted index file: %s exceeds 2GB limit", _invertedIndexFile);
+    }
+    ByteBuffer buffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      buffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, size).order(LITTLE_ENDIAN);

Review comment:
       In any case, the code in question was changed so that the offsets part of the file is big endian anyway, and a little endian buffer is provided to `RoaringBitmap.serialize`




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r537811099



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +184,45 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      // map the offsets buffer
+      int startOfBitmaps = (_cardinality + 1) * Integer.BYTES;
+      offsetBuffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, startOfBitmaps)
+              .order(ByteOrder.BIG_ENDIAN);
+      offsetBuffer.putInt(startOfBitmaps);
+      bitmapBuffer = channel.map(FileChannel.MapMode.READ_WRITE, startOfBitmaps, Integer.MAX_VALUE - startOfBitmaps)
+              .order(ByteOrder.LITTLE_ENDIAN);
+      RoaringBitmapWriter<RoaringBitmap> writer = RoaringBitmapWriter.writer().get();

Review comment:
       If that's the case, currently this will try to RLE each container when it's appended to the bitmap. Would you prefer I remove that when I reformat the file?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r536520136



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OnHeapBitmapInvertedIndexCreator.java
##########
@@ -19,68 +19,85 @@
 package org.apache.pinot.core.segment.creator.impl.inv;
 
 import com.google.common.base.Preconditions;
-import java.io.BufferedOutputStream;
-import java.io.DataOutputStream;
+
+
 import java.io.File;
-import java.io.FileOutputStream;
 import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.ByteBuffer;
+import java.nio.channels.FileChannel;
+
 import org.apache.commons.io.FileUtils;
 import org.apache.pinot.core.segment.creator.DictionaryBasedInvertedIndexCreator;
 import org.apache.pinot.core.segment.creator.impl.V1Constants;
+import org.apache.pinot.core.util.CleanerUtil;
+import org.roaringbitmap.RoaringBitmapWriter;
 import org.roaringbitmap.buffer.MutableRoaringBitmap;
 
+import static java.nio.ByteOrder.LITTLE_ENDIAN;
+
 
 /**
  * Implementation of {@link DictionaryBasedInvertedIndexCreator} that uses on-heap memory.
  */
 public final class OnHeapBitmapInvertedIndexCreator implements DictionaryBasedInvertedIndexCreator {
   private final File _invertedIndexFile;
-  private final MutableRoaringBitmap[] _bitmaps;
+  private final RoaringBitmapWriter<MutableRoaringBitmap>[] _bitmapWriters;
   private int _nextDocId;
 
+  @SuppressWarnings("unchecked")
   public OnHeapBitmapInvertedIndexCreator(File indexDir, String columnName, int cardinality) {
     _invertedIndexFile = new File(indexDir, columnName + V1Constants.Indexes.BITMAP_INVERTED_INDEX_FILE_EXTENSION);
-    _bitmaps = new MutableRoaringBitmap[cardinality];
+    _bitmapWriters = new RoaringBitmapWriter[cardinality];
     for (int i = 0; i < cardinality; i++) {
-      _bitmaps[i] = new MutableRoaringBitmap();
+      _bitmapWriters[i] = RoaringBitmapWriter.bufferWriter().get();
     }
   }
 
   @Override
   public void add(int dictId) {
-    _bitmaps[dictId].add(_nextDocId++);
+    _bitmapWriters[dictId].add(_nextDocId++);
   }
 
   @Override
   public void add(int[] dictIds, int length) {
     for (int i = 0; i < length; i++) {
-      _bitmaps[dictIds[i]].add(_nextDocId);
+      _bitmapWriters[dictIds[i]].add(_nextDocId);
     }
     _nextDocId++;
   }
 
   @Override
   public void seal()
       throws IOException {
-    try (DataOutputStream out = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)))) {
+    // calculate file size
+    int size = (_bitmapWriters.length + 1) * Integer.BYTES;
+    for (RoaringBitmapWriter<MutableRoaringBitmap> writer : _bitmapWriters) {
+      size += writer.get().serializedSizeInBytes();
+      // Check for int overflow
+      Preconditions.checkState(size > 0, "Inverted index file: %s exceeds 2GB limit", _invertedIndexFile);
+    }
+    ByteBuffer buffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      buffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, size).order(LITTLE_ENDIAN);

Review comment:
       Ok, I see you are swapping the bytes below before writing. So, this is writing BE bytes in LE file so that later during index load, read in BE format will work. In this case, why not just choose the BE format for writing as we do today. Does the usage of RoaringBitmapWriter enforce LE format?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r536458126



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +188,49 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      // map the offsets buffer
+      final int startOfBitmaps = (_cardinality + 1) * Integer.BYTES;
+      int bitmapOffset = startOfBitmaps;
+      offsetBuffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, bitmapOffset).order(LITTLE_ENDIAN);
+      offsetBuffer.putInt(reverseBytes(bitmapOffset));
+      MutableRoaringBitmap[] bitmaps = new MutableRoaringBitmap[_cardinality];

Review comment:
       This is a new allocation, but it's offset against the 16KB in stream buffers no longer allocated, and is a lot smaller than the bitmaps being materialised here anyway. This pattern where bitmaps are allocated just to obtain their serialized bytes could probably be supported by roaringbitmap without actually creating the intermediate bitmaps without much complication.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r536520136



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OnHeapBitmapInvertedIndexCreator.java
##########
@@ -19,68 +19,85 @@
 package org.apache.pinot.core.segment.creator.impl.inv;
 
 import com.google.common.base.Preconditions;
-import java.io.BufferedOutputStream;
-import java.io.DataOutputStream;
+
+
 import java.io.File;
-import java.io.FileOutputStream;
 import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.ByteBuffer;
+import java.nio.channels.FileChannel;
+
 import org.apache.commons.io.FileUtils;
 import org.apache.pinot.core.segment.creator.DictionaryBasedInvertedIndexCreator;
 import org.apache.pinot.core.segment.creator.impl.V1Constants;
+import org.apache.pinot.core.util.CleanerUtil;
+import org.roaringbitmap.RoaringBitmapWriter;
 import org.roaringbitmap.buffer.MutableRoaringBitmap;
 
+import static java.nio.ByteOrder.LITTLE_ENDIAN;
+
 
 /**
  * Implementation of {@link DictionaryBasedInvertedIndexCreator} that uses on-heap memory.
  */
 public final class OnHeapBitmapInvertedIndexCreator implements DictionaryBasedInvertedIndexCreator {
   private final File _invertedIndexFile;
-  private final MutableRoaringBitmap[] _bitmaps;
+  private final RoaringBitmapWriter<MutableRoaringBitmap>[] _bitmapWriters;
   private int _nextDocId;
 
+  @SuppressWarnings("unchecked")
   public OnHeapBitmapInvertedIndexCreator(File indexDir, String columnName, int cardinality) {
     _invertedIndexFile = new File(indexDir, columnName + V1Constants.Indexes.BITMAP_INVERTED_INDEX_FILE_EXTENSION);
-    _bitmaps = new MutableRoaringBitmap[cardinality];
+    _bitmapWriters = new RoaringBitmapWriter[cardinality];
     for (int i = 0; i < cardinality; i++) {
-      _bitmaps[i] = new MutableRoaringBitmap();
+      _bitmapWriters[i] = RoaringBitmapWriter.bufferWriter().get();
     }
   }
 
   @Override
   public void add(int dictId) {
-    _bitmaps[dictId].add(_nextDocId++);
+    _bitmapWriters[dictId].add(_nextDocId++);
   }
 
   @Override
   public void add(int[] dictIds, int length) {
     for (int i = 0; i < length; i++) {
-      _bitmaps[dictIds[i]].add(_nextDocId);
+      _bitmapWriters[dictIds[i]].add(_nextDocId);
     }
     _nextDocId++;
   }
 
   @Override
   public void seal()
       throws IOException {
-    try (DataOutputStream out = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)))) {
+    // calculate file size
+    int size = (_bitmapWriters.length + 1) * Integer.BYTES;
+    for (RoaringBitmapWriter<MutableRoaringBitmap> writer : _bitmapWriters) {
+      size += writer.get().serializedSizeInBytes();
+      // Check for int overflow
+      Preconditions.checkState(size > 0, "Inverted index file: %s exceeds 2GB limit", _invertedIndexFile);
+    }
+    ByteBuffer buffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      buffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, size).order(LITTLE_ENDIAN);

Review comment:
       Ok, I see you are swapping the bytes below before writing. So, this is writing BE bytes in LE file so that later during read in BE format will work. In this case, why not choose the BE format for writing as we do today. Does the usage of RoaringBitmapWriter enforce LE format?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r537124621



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OnHeapBitmapInvertedIndexCreator.java
##########
@@ -19,68 +19,96 @@
 package org.apache.pinot.core.segment.creator.impl.inv;
 
 import com.google.common.base.Preconditions;
-import java.io.BufferedOutputStream;
-import java.io.DataOutputStream;
+
+
 import java.io.File;
-import java.io.FileOutputStream;
 import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.ByteBuffer;
+import java.nio.ByteOrder;
+import java.nio.channels.FileChannel;
+
 import org.apache.commons.io.FileUtils;
 import org.apache.pinot.core.segment.creator.DictionaryBasedInvertedIndexCreator;
 import org.apache.pinot.core.segment.creator.impl.V1Constants;
-import org.roaringbitmap.buffer.MutableRoaringBitmap;
+import org.apache.pinot.core.util.CleanerUtil;
+import org.roaringbitmap.RoaringBitmap;
+import org.roaringbitmap.RoaringBitmapWriter;
 
 
 /**
  * Implementation of {@link DictionaryBasedInvertedIndexCreator} that uses on-heap memory.
  */
 public final class OnHeapBitmapInvertedIndexCreator implements DictionaryBasedInvertedIndexCreator {
   private final File _invertedIndexFile;
-  private final MutableRoaringBitmap[] _bitmaps;
+  private final RoaringBitmapWriter<RoaringBitmap>[] _bitmapWriters;
   private int _nextDocId;
 
+  @SuppressWarnings("unchecked")
   public OnHeapBitmapInvertedIndexCreator(File indexDir, String columnName, int cardinality) {
     _invertedIndexFile = new File(indexDir, columnName + V1Constants.Indexes.BITMAP_INVERTED_INDEX_FILE_EXTENSION);
-    _bitmaps = new MutableRoaringBitmap[cardinality];
+    _bitmapWriters = new RoaringBitmapWriter[cardinality];
     for (int i = 0; i < cardinality; i++) {
-      _bitmaps[i] = new MutableRoaringBitmap();
+      _bitmapWriters[i] = RoaringBitmapWriter.writer().get();
     }
   }
 
   @Override
   public void add(int dictId) {
-    _bitmaps[dictId].add(_nextDocId++);
+    _bitmapWriters[dictId].add(_nextDocId++);
   }
 
   @Override
   public void add(int[] dictIds, int length) {
     for (int i = 0; i < length; i++) {
-      _bitmaps[dictIds[i]].add(_nextDocId);
+      _bitmapWriters[dictIds[i]].add(_nextDocId);
     }
     _nextDocId++;
   }
 
   @Override
   public void seal()
       throws IOException {
-    try (DataOutputStream out = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)))) {
+    // calculate file size
+    int startOfBitmaps = (_bitmapWriters.length + 1) * Integer.BYTES;
+    int size = 0;
+    for (RoaringBitmapWriter<RoaringBitmap> writer : _bitmapWriters) {
+      size += writer.get().serializedSizeInBytes();

Review comment:
       Maybe we can use the same way as the off-heap creator to preallocate a huge buffer then truncate to same the calculation of the serialized size?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin edited a comment on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin edited a comment on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739241103


   This PR allocates about the same amount as before. TL;DR: as of the [last commit](https://github.com/apache/incubator-pinot/pull/6320/commits/60861bc25110b94011091332a70a5e955e38405b) this is faster on this benchmark, and allocates between 40% and 100% (i.e. the same) as on the main branch.
   
   before
   ```
   
   Benchmark                                                                           (assignment)  (cardinality)  (numDocs)  Mode  Cnt           Score           Error   Units
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN             10    1000000    ss    5           0.031 ±         0.009    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN             10    1000000    ss    5          11.935 ±         1.702  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN             10    1000000    ss    5     7164704.000 ±       355.929    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN             10    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN             10   10000000    ss    5           0.408 ±         0.019    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN             10   10000000    ss    5          62.589 ±         5.249  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN             10   10000000    ss    5    71375131.200 ±     56512.643    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN             10   10000000    ss    5          70.263 ±       370.500  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN             10   10000000    ss    5    81788928.000 ± 431258859.992    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN             10   10000000    ss    5           0.190 ±         1.632  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN             10   10000000    ss    5      217683.200 ±   1874318.001    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN             10   10000000    ss    5           2.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN             10   10000000    ss    5          16.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN             10  100000000    ss    5           4.193 ±         0.372    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN             10  100000000    ss    5         122.075 ±        12.717  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN             10  100000000    ss    5   712415345.600 ±     55691.481    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN             10  100000000    ss    5         120.923 ±        70.361  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN             10  100000000    ss    5   706320793.600 ± 422412916.188    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN             10  100000000    ss    5           1.682 ±         0.744  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN             10  100000000    ss    5     9832512.000 ±   4758932.013    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN             10  100000000    ss    5          18.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN             10  100000000    ss    5          35.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN           1000    1000000    ss    5           0.092 ±         0.014    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN           1000    1000000    ss    5          22.304 ±         0.194  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN           1000    1000000    ss    5    14236214.400 ±      1270.764    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN           1000    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN           1000   10000000    ss    5           1.010 ±         0.033    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN           1000   10000000    ss    5          76.099 ±         1.422  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN           1000   10000000    ss    5   139322380.800 ±     46579.496    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN           1000   10000000    ss    5          83.271 ±       179.254  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN           1000   10000000    ss    5   152672665.600 ± 328639423.512    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN           1000   10000000    ss    5           0.046 ±         0.399  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN           1000   10000000    ss    5       84628.800 ±    728679.490    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN           1000   10000000    ss    5           4.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN           1000   10000000    ss    5          29.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN           1000  100000000    ss    5          11.633 ±         0.451    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN           1000  100000000    ss    5         101.363 ±         2.794  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN           1000  100000000    ss    5  1385939961.600 ±     55294.403    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN           1000  100000000    ss    5         102.172 ±        30.699  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN           1000  100000000    ss    5  1396703232.000 ± 398076377.621    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN           1000  100000000    ss    5           0.011 ±         0.052  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN           1000  100000000    ss    5      145384.000 ±    710672.251    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN           1000  100000000    ss    5          37.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN           1000  100000000    ss    5          37.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN          10000    1000000    ss    5           0.150 ±         0.045    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN          10000    1000000    ss    5          46.513 ±         4.210  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN          10000    1000000    ss    5    33470916.800 ±     45678.008    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN          10000    1000000    ss    5          41.258 ±       355.241  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN          10000    1000000    ss    5    29779558.400 ± 256410978.784    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN          10000    1000000    ss    5           1.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN          10000    1000000    ss    5          10.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN          10000   10000000    ss    5           1.738 ±         0.386    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN          10000   10000000    ss    5         125.776 ±        12.578  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN          10000   10000000    ss    5   317429830.400 ±     64562.976    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN          10000   10000000    ss    5         110.973 ±       164.658  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN          10000   10000000    ss    5   279340646.400 ± 400867868.240    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN          10000   10000000    ss    5           0.143 ±         0.755  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN          10000   10000000    ss    5      354760.000 ±   1874217.875    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN          10000   10000000    ss    5           6.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN          10000   10000000    ss    5          57.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN          10000  100000000    ss    5          17.156 ±         0.148    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN          10000  100000000    ss    5         160.044 ±         1.436  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN          10000  100000000    ss    5  3116823352.000 ±     45545.903    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN          10000  100000000    ss    5         159.645 ±        26.123  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN          10000  100000000    ss    5  3108818124.800 ± 482114724.008    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN          10000  100000000    ss    5           0.026 ±         0.009  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN          10000  100000000    ss    5      497857.600 ±    172824.399    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN          10000  100000000    ss    5          68.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN          10000  100000000    ss    5          73.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM             10    1000000    ss    5           0.020 ±         0.002    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM             10    1000000    ss    5           2.138 ±         0.034  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM             10    1000000    ss    5     1184286.400 ±       378.164    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM             10    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM             10   10000000    ss    5           0.211 ±         0.001    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM             10   10000000    ss    5           9.535 ±         0.231  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM             10   10000000    ss    5     7565811.200 ±       502.228    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM             10   10000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM             10  100000000    ss    5           2.326 ±         0.034    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM             10  100000000    ss    5          20.862 ±         0.237  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM             10  100000000    ss    5    71689880.000 ±     56609.543    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM             10  100000000    ss    5          20.966 ±       110.550  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM             10  100000000    ss    5    72142028.800 ± 380384094.171    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space       SORTED_UNIFORM             10  100000000    ss    5           0.015 ±         0.127  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm  SORTED_UNIFORM             10  100000000    ss    5       50654.400 ±    436149.660    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM             10  100000000    ss    5           2.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM             10  100000000    ss    5          15.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM           1000    1000000    ss    5           0.074 ±         0.004    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM           1000    1000000    ss    5          12.163 ±         0.138  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM           1000    1000000    ss    5     7452222.400 ±      1019.460    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM           1000    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM           1000   10000000    ss    5           0.275 ±         0.013    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM           1000   10000000    ss    5          52.103 ±         1.439  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM           1000   10000000    ss    5    50582414.400 ±     45991.882    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM           1000   10000000    ss    5          42.028 ±       361.875  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM           1000   10000000    ss    5    41104179.200 ± 353919379.167    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM           1000   10000000    ss    5           1.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM           1000   10000000    ss    5           7.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM           1000  100000000    ss    5           2.099 ±         0.251    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM           1000  100000000    ss    5          35.099 ±         0.579  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM           1000  100000000    ss    5   114648668.800 ±     55423.632    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM           1000  100000000    ss    5          33.538 ±       117.892  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM           1000  100000000    ss    5   109471334.400 ± 384807165.034    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space       SORTED_UNIFORM           1000  100000000    ss    5           0.026 ±         0.223  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm  SORTED_UNIFORM           1000  100000000    ss    5       83841.600 ±    721901.461    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM           1000  100000000    ss    5           3.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM           1000  100000000    ss    5          22.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM          10000    1000000    ss    5           0.093 ±         0.011    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM          10000    1000000    ss    5          20.878 ±         1.225  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM          10000    1000000    ss    5    13544881.600 ±       191.637    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM          10000    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM          10000   10000000    ss    5           0.886 ±         0.244    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM          10000   10000000    ss    5          46.010 ±         1.042  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM          10000   10000000    ss    5    74304364.800 ±     46037.614    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM          10000   10000000    ss    5          24.963 ±       214.936  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM          10000   10000000    ss    5    40684748.800 ± 350307956.931    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM          10000   10000000    ss    5           1.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM          10000   10000000    ss    5           9.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM          10000  100000000    ss    5           3.826 ±         0.052    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM          10000  100000000    ss    5         100.180 ±         1.713  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM          10000  100000000    ss    5   505538409.600 ±     96799.515    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM          10000  100000000    ss    5          99.990 ±        79.926  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM          10000  100000000    ss    5   504574771.200 ± 402985207.107    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space       SORTED_UNIFORM          10000  100000000    ss    5           0.329 ±         2.728  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm  SORTED_UNIFORM          10000  100000000    ss    5     1669273.600 ±  13844467.735    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM          10000  100000000    ss    5          13.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM          10000  100000000    ss    5          28.000                      ms
   ```
   
   after
   ```
   Benchmark                                                                           (assignment)  (cardinality)  (numDocs)  Mode  Cnt           Score           Error   Units
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN             10    1000000    ss    5           0.029 ±         0.022    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN             10    1000000    ss    5          12.403 ±         1.117  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN             10    1000000    ss    5     7146006.400 ±     30985.905    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN             10    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN             10   10000000    ss    5           0.188 ±         0.026    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN             10   10000000    ss    5          87.108 ±         4.467  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN             10   10000000    ss    5    71108766.400 ±     78261.812    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN             10   10000000    ss    5          95.241 ±       502.203  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN             10   10000000    ss    5    78014054.400 ± 411385221.551    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN             10   10000000    ss    5           1.101 ±         9.482  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN             10   10000000    ss    5      899424.000 ±   7744311.890    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN             10   10000000    ss    5           2.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN             10   10000000    ss    5          25.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN             10  100000000    ss    5           1.947 ±         0.116    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN             10  100000000    ss    5         208.005 ±         7.371  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN             10  100000000    ss    5   709829240.000 ±     51497.263    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN             10  100000000    ss    5         201.399 ±        76.815  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN             10  100000000    ss    5   687026995.200 ± 247098882.679    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen                 ROUND_ROBIN             10  100000000    ss    5           2.224 ±        17.872  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen.norm            ROUND_ROBIN             10  100000000    ss    5     7549747.200 ±  60592304.093    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN             10  100000000    ss    5           4.788 ±        16.708  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN             10  100000000    ss    5    16357785.600 ±  57215688.521    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN             10  100000000    ss    5          17.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN             10  100000000    ss    5         256.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN           1000    1000000    ss    5           0.040 ±         0.023    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN           1000    1000000    ss    5          20.222 ±         1.683  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN           1000    1000000    ss    5    11859302.400 ±     30898.399    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN           1000    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN           1000   10000000    ss    5           0.293 ±         0.080    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN           1000   10000000    ss    5         123.802 ±        11.035  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN           1000   10000000    ss    5   113905795.200 ±     74048.230    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN           1000   10000000    ss    5         108.134 ±       386.562  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN           1000   10000000    ss    5   100663296.000 ± 360192977.778    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN           1000   10000000    ss    5           6.363 ±        34.666  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN           1000   10000000    ss    5     5872025.600 ±  32124407.719    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN           1000   10000000    ss    5           3.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN           1000   10000000    ss    5          35.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN           1000  100000000    ss    5           2.898 ±         0.114    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN           1000  100000000    ss    5         255.212 ±         6.003  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN           1000  100000000    ss    5  1129770475.200 ±    124788.799    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN           1000  100000000    ss    5         257.389 ±        62.089  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN           1000  100000000    ss    5  1139592396.800 ± 287647003.971    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen                 ROUND_ROBIN           1000  100000000    ss    5           6.228 ±        32.860  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen.norm            ROUND_ROBIN           1000  100000000    ss    5    27472691.200 ± 144918864.230    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN           1000  100000000    ss    5           7.259 ±        12.981  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN           1000  100000000    ss    5    32086425.600 ±  56930036.769    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN           1000  100000000    ss    5          24.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN           1000  100000000    ss    5        1023.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN          10000    1000000    ss    5           0.066 ±         0.037    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN          10000    1000000    ss    5          72.601 ±         4.420  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN          10000    1000000    ss    5    45097956.800 ±     48245.104    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN          10000    1000000    ss    5          74.815 ±       644.181  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN          10000    1000000    ss    5    47395635.200 ± 408090712.713    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN          10000    1000000    ss    5           1.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN          10000    1000000    ss    5          14.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN          10000   10000000    ss    5           0.557 ±         0.267    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN          10000   10000000    ss    5         186.074 ±        47.551  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN          10000   10000000    ss    5   223440820.800 ±     78325.490    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN          10000   10000000    ss    5         138.524 ±       337.002  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN          10000   10000000    ss    5   171127603.200 ± 416904247.274    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN          10000   10000000    ss    5           1.756 ±        15.119  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN          10000   10000000    ss    5     2097152.000 ±  18057111.182    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN          10000   10000000    ss    5           5.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN          10000   10000000    ss    5         298.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN          10000  100000000    ss    5           5.260 ±         0.325    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN          10000  100000000    ss    5         310.016 ±        14.954  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN          10000  100000000    ss    5  2143068185.600 ±     64550.566    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN          10000  100000000    ss    5         306.636 ±        42.750  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN          10000  100000000    ss    5  2120220672.000 ± 339646533.455    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN          10000  100000000    ss    5           8.017 ±         4.673  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN          10000  100000000    ss    5    55364812.800 ±  30537345.918    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN          10000  100000000    ss    5          39.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN          10000  100000000    ss    5        4087.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM             10    1000000    ss    5           0.017 ±         0.006    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM             10    1000000    ss    5           2.146 ±         0.059  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM             10    1000000    ss    5     1175939.200 ±     30372.447    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM             10    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM             10   10000000    ss    5           0.155 ±         0.001    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM             10   10000000    ss    5          10.358 ±         0.065  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM             10   10000000    ss    5     7563580.800 ±     30697.759    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM             10   10000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM             10  100000000    ss    5           1.548 ±         0.023    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM             10  100000000    ss    5          28.135 ±         0.251  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM             10  100000000    ss    5    71579300.800 ±     78187.548    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM             10  100000000    ss    5          28.278 ±       149.101  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM             10  100000000    ss    5    72142028.800 ± 380384094.171    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space       SORTED_UNIFORM             10  100000000    ss    5           0.028 ±         0.239  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm  SORTED_UNIFORM             10  100000000    ss    5       70888.000 ±    610367.059    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM             10  100000000    ss    5           2.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM             10  100000000    ss    5          17.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM           1000    1000000    ss    5           0.023 ±         0.032    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM           1000    1000000    ss    5          13.814 ±         1.193  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM           1000    1000000    ss    5     7669910.400 ±    300898.435    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM           1000    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM           1000   10000000    ss    5           0.181 ±         0.037    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM           1000   10000000    ss    5          66.605 ±         4.336  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM           1000   10000000    ss    5    50803308.800 ±     76359.327    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM           1000   10000000    ss    5          62.270 ±       536.163  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM           1000   10000000    ss    5    47395635.200 ± 408090712.713    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM           1000   10000000    ss    5           1.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM           1000   10000000    ss    5           8.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM           1000  100000000    ss    5           1.568 ±         0.020    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM           1000  100000000    ss    5          44.731 ±         0.340  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM           1000  100000000    ss    5   114796513.600 ±     73178.185    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM           1000  100000000    ss    5          43.084 ±       151.447  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM           1000  100000000    ss    5   110729625.600 ± 389230235.896    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space       SORTED_UNIFORM           1000  100000000    ss    5           0.032 ±         0.275  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm  SORTED_UNIFORM           1000  100000000    ss    5       82238.400 ±    708097.426    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM           1000  100000000    ss    5           3.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM           1000  100000000    ss    5          24.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM          10000    1000000    ss    5           0.034 ±         0.037    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM          10000    1000000    ss    5          27.006 ±         2.034  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM          10000    1000000    ss    5    15331587.200 ±     30511.819    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM          10000    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM          10000   10000000    ss    5           0.186 ±         0.016    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM          10000   10000000    ss    5          99.771 ±         3.199  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM          10000   10000000    ss    5    76239598.400 ±     76412.193    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM          10000   10000000    ss    5          53.525 ±       460.870  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM          10000   10000000    ss    5    41523609.600 ± 357530801.403    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space       SORTED_UNIFORM          10000   10000000    ss    5           0.062 ±         0.531  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm  SORTED_UNIFORM          10000   10000000    ss    5       47827.200 ±    411806.616    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM          10000   10000000    ss    5           1.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM          10000   10000000    ss    5           9.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM          10000  100000000    ss    5           1.736 ±         0.045    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM          10000  100000000    ss    5         184.429 ±         5.794  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM          10000  100000000    ss    5   507845612.800 ±     59214.207    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM          10000  100000000    ss    5         171.683 ±        16.667  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM          10000  100000000    ss    5   472698060.800 ±  32099022.960    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space       SORTED_UNIFORM          10000  100000000    ss    5           0.901 ±         4.227  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm  SORTED_UNIFORM          10000  100000000    ss    5     2492544.000 ±  11744141.868    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM          10000  100000000    ss    5          10.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM          10000  100000000    ss    5          49.000                      ms
   ```
   
   Allocation can be reduced and get a speed gain by guessing the approximate sizes of the bitmaps, and by reusing a single writer.
   
   latest commit
   ```
   Benchmark                                          (assignment)  (cardinality)  (numDocs)  Mode  Cnt  Score   Error  Units
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN             10    1000000    ss    5  0.021 ± 0.038   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN             10   10000000    ss    5  0.175 ± 0.015   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN             10  100000000    ss    5  1.780 ± 0.117   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN           1000    1000000    ss    5  0.024 ± 0.003   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN           1000   10000000    ss    5  0.243 ± 0.035   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN           1000  100000000    ss    5  2.711 ± 0.219   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN          10000    1000000    ss    5  0.052 ± 0.024   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN          10000   10000000    ss    5  0.539 ± 0.107   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN          10000  100000000    ss    5  4.500 ± 0.246   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM             10    1000000    ss    5  0.016 ± 0.003   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM             10   10000000    ss    5  0.148 ± 0.005   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM             10  100000000    ss    5  1.412 ± 0.026   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM           1000    1000000    ss    5  0.021 ± 0.026   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM           1000   10000000    ss    5  0.165 ± 0.019   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM           1000  100000000    ss    5  1.437 ± 0.021   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM          10000    1000000    ss    5  0.022 ± 0.005   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM          10000   10000000    ss    5  0.172 ± 0.022   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM          10000  100000000    ss    5  1.581 ± 0.095  s/op
   ```
   
   ```
   Benchmark                                                                           (assignment)  (cardinality)  (numDocs)  Mode  Cnt           Score           Error   Units
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN             10    1000000    ss    5           0.017 ±         0.002    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN             10    1000000    ss    5          12.633 ±         0.155  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN             10    1000000    ss    5     6986539.200 ±     30442.010    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN             10    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN             10   10000000    ss    5           0.176 ±         0.012    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN             10   10000000    ss    5          87.350 ±         1.202  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN             10   10000000    ss    5    69659446.400 ±     52302.495    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN             10   10000000    ss    5         110.528 ±       582.795  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN             10   10000000    ss    5    88499814.400 ± 466642710.250    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN             10   10000000    ss    5           2.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN             10   10000000    ss    5          12.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN             10  100000000    ss    5           1.783 ±         0.104    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN             10  100000000    ss    5         214.505 ±         7.603  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN             10  100000000    ss    5   695378540.800 ±    139270.627    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN             10  100000000    ss    5         214.143 ±       104.774  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN             10  100000000    ss    5   694576742.400 ± 352155335.840    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen                 ROUND_ROBIN             10  100000000    ss    5           1.321 ±        11.370  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen.norm            ROUND_ROBIN             10  100000000    ss    5     4256358.400 ±  36648529.462    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN             10  100000000    ss    5           3.111 ±         7.794  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN             10  100000000    ss    5    10067219.200 ±  25155045.841    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN             10  100000000    ss    5          23.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN             10  100000000    ss    5         275.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN           1000    1000000    ss    5           0.025 ±         0.022    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN           1000    1000000    ss    5          14.770 ±         1.072  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN           1000    1000000    ss    5     8325694.400 ±     30481.121    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN           1000    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN           1000   10000000    ss    5           0.244 ±         0.040    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN           1000   10000000    ss    5          92.415 ±         4.679  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN           1000   10000000    ss    5    80292582.400 ±     51274.216    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN           1000   10000000    ss    5          69.550 ±       366.944  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN           1000   10000000    ss    5    60817408.000 ± 320787006.897    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN           1000   10000000    ss    5           2.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN           1000   10000000    ss    5          22.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN           1000  100000000    ss    5           2.429 ±         0.248    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN           1000  100000000    ss    5         200.418 ±        13.277  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN           1000  100000000    ss    5   787876115.200 ±    256741.280    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN           1000  100000000    ss    5         193.777 ±       130.177  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN           1000  100000000    ss    5   762943897.600 ± 544097645.991    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen                 ROUND_ROBIN           1000  100000000    ss    5           1.467 ±        12.630  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen.norm            ROUND_ROBIN           1000  100000000    ss    5     5872025.600 ±  50559911.310    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN           1000  100000000    ss    5           4.343 ±         9.498  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN           1000  100000000    ss    5    17196646.400 ±  38474784.725    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN           1000  100000000    ss    5          23.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN           1000  100000000    ss    5         746.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN          10000    1000000    ss    5           0.054 ±         0.025    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN          10000    1000000    ss    5          29.477 ±         2.118  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN          10000    1000000    ss    5    17577993.600 ±     48406.649    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN          10000    1000000    ss    5          30.144 ±       259.553  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN          10000    1000000    ss    5    18035507.200 ± 155291156.165    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN          10000    1000000    ss    5           1.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN          10000    1000000    ss    5          12.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN          10000   10000000    ss    5           0.481 ±         0.050    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN          10000   10000000    ss    5         205.551 ±        10.013  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN          10000   10000000    ss    5   230199414.400 ±    124257.456    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN          10000   10000000    ss    5         200.550 ±       150.026  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN          10000   10000000    ss    5   224395264.000 ± 162413652.372    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen                 ROUND_ROBIN          10000   10000000    ss    5           1.853 ±        15.957  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen.norm            ROUND_ROBIN          10000   10000000    ss    5     2097152.000 ±  18057111.182    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN          10000   10000000    ss    5          11.056 ±        41.589  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN          10000   10000000    ss    5    12379680.000 ±  46553344.432    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN          10000   10000000    ss    5          11.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN          10000   10000000    ss    5         304.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                      ROUND_ROBIN          10000  100000000    ss    5           4.505 ±         0.168    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                       ROUND_ROBIN          10000  100000000    ss    5         220.361 ±         6.174  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm                  ROUND_ROBIN          10000  100000000    ss    5  1348977696.000 ±     48374.015    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space              ROUND_ROBIN          10000  100000000    ss    5         220.473 ±        89.136  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm         ROUND_ROBIN          10000  100000000    ss    5  1350146457.600 ± 565202002.425    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen                 ROUND_ROBIN          10000  100000000    ss    5           0.719 ±         6.193  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Old_Gen.norm            ROUND_ROBIN          10000  100000000    ss    5     4404019.200 ±  37919933.482    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space          ROUND_ROBIN          10000  100000000    ss    5           5.615 ±         5.115  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm     ROUND_ROBIN          10000  100000000    ss    5    34393292.800 ±  31587043.999    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                            ROUND_ROBIN          10000  100000000    ss    5          36.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                             ROUND_ROBIN          10000  100000000    ss    5        2728.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM             10    1000000    ss    5           0.022 ±         0.037    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM             10    1000000    ss    5           2.066 ±         0.233  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM             10    1000000    ss    5     1148363.200 ±     30594.344    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM             10    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM             10   10000000    ss    5           0.148 ±         0.003    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM             10   10000000    ss    5          10.223 ±         0.105  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM             10   10000000    ss    5     7390347.200 ±     30613.010    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM             10   10000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM             10  100000000    ss    5           1.410 ±         0.015    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM             10  100000000    ss    5          29.172 ±         0.183  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM             10  100000000    ss    5    70052542.400 ±    137755.129    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM             10  100000000    ss    5          29.999 ±       158.176  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM             10  100000000    ss    5    72142028.800 ± 380384094.171    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space       SORTED_UNIFORM             10  100000000    ss    5           0.034 ±         0.293  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm  SORTED_UNIFORM             10  100000000    ss    5       81944.000 ±    705562.553    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM             10  100000000    ss    5           2.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM             10  100000000    ss    5          16.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM           1000    1000000    ss    5           0.025 ±         0.040    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM           1000    1000000    ss    5          12.373 ±         1.158  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM           1000    1000000    ss    5     6932864.000 ±     30927.743    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM           1000    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM           1000   10000000    ss    5           0.164 ±         0.015    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM           1000   10000000    ss    5          66.973 ±         1.648  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM           1000   10000000    ss    5    49529336.000 ±     76261.103    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM           1000   10000000    ss    5          55.023 ±       473.763  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM           1000   10000000    ss    5    41104179.200 ± 353919379.167    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM           1000   10000000    ss    5           1.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM           1000   10000000    ss    5           8.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM           1000  100000000    ss    5           1.434 ±         0.018    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM           1000  100000000    ss    5          46.177 ±         0.422  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM           1000  100000000    ss    5   112162459.200 ±     73253.706    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM           1000  100000000    ss    5          44.479 ±       156.352  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM           1000  100000000    ss    5   108213043.200 ± 380384094.171    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space       SORTED_UNIFORM           1000  100000000    ss    5           0.033 ±         0.282  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm  SORTED_UNIFORM           1000  100000000    ss    5       80251.200 ±    686689.981    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM           1000  100000000    ss    5           3.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM           1000  100000000    ss    5          23.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM          10000    1000000    ss    5           0.023 ±         0.011    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM          10000    1000000    ss    5          20.115 ±         0.796  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM          10000    1000000    ss    5    11170392.000 ±     30879.694    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM          10000    1000000    ss    5             ≈ 0                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM          10000   10000000    ss    5           0.167 ±         0.016    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM          10000   10000000    ss    5          91.921 ±         7.404  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM          10000   10000000    ss    5    69227684.800 ±     52043.344    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM          10000   10000000    ss    5         114.528 ±       604.014  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM          10000   10000000    ss    5    87241523.200 ± 459999369.696    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM          10000   10000000    ss    5           2.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM          10000   10000000    ss    5          17.000                      ms
   BenchmarkOffheapBitmapInvertedIndexCreator.seal                                   SORTED_UNIFORM          10000  100000000    ss    5           1.603 ±         0.123    s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate                    SORTED_UNIFORM          10000  100000000    ss    5         190.115 ±         9.171  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.alloc.rate.norm               SORTED_UNIFORM          10000  100000000    ss    5   495109988.800 ±    101261.939    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space           SORTED_UNIFORM          10000  100000000    ss    5         197.630 ±       142.159  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Eden_Space.norm      SORTED_UNIFORM          10000  100000000    ss    5   514641100.800 ± 368630518.221    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space       SORTED_UNIFORM          10000  100000000    ss    5           0.842 ±         4.357  MB/sec
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.churn.G1_Survivor_Space.norm  SORTED_UNIFORM          10000  100000000    ss    5     2221792.000 ±  11637986.706    B/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.count                         SORTED_UNIFORM          10000  100000000    ss    5          11.000                  counts
   BenchmarkOffheapBitmapInvertedIndexCreator.seal:·gc.time                          SORTED_UNIFORM          10000  100000000    ss    5          49.000                      ms
   
   ```
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r537132715



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +188,51 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      // map the offsets buffer
+      final int startOfBitmaps = (_cardinality + 1) * Integer.BYTES;
+      int bitmapOffset = startOfBitmaps;
+      offsetBuffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, bitmapOffset).order(LITTLE_ENDIAN);
+      offsetBuffer.putInt(reverseBytes(bitmapOffset));
+      RoaringBitmap[] bitmaps = new RoaringBitmap[_cardinality];
+      RoaringBitmapWriter<RoaringBitmap> writer = RoaringBitmapWriter.writer()
+              .initialCapacity(((_nextDocId - 1) >>> 16) / _cardinality).get();

Review comment:
       I felt this heuristic was distracting so removed it. 
   
   In case this comes up in the future, expecting the range to be `[0, _nextDocId)` would allocate far too much memory for datasets sorted by the indexed dimension, but would be the right choice if the association between row id and dictionary id were uniformly random. Some metadata could probably be associated with each dictionary id (number of documents, min, max, density?) in memory to use these methods could likely be used more effectively, and not in a way that overfits to a benchmark.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739096511


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=h1) Report
   > Merging [#6320](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=desc) (03de308) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **increase** coverage by `7.71%`.
   > The diff coverage is `68.40%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6320/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #6320      +/-   ##
   ==========================================
   + Coverage   66.44%   74.16%   +7.71%     
   ==========================================
     Files        1075     1281     +206     
     Lines       54773    61823    +7050     
     Branches     8168     8986     +818     
   ==========================================
   + Hits        36396    45852    +9456     
   + Misses      15700    13007    -2693     
   - Partials     2677     2964     +287     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration | `45.79% <52.01%> (?)` | |
   | unittests | `65.37% <45.12%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `52.83% <0.00%> (-13.84%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `44.44% <0.00%> (-4.40%)` | :arrow_down: |
   | [...not/common/assignment/InstancePartitionsUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXNzaWdubWVudC9JbnN0YW5jZVBhcnRpdGlvbnNVdGlscy5qYXZh) | `78.57% <ø> (+5.40%)` | :arrow_up: |
   | [.../apache/pinot/common/exception/QueryException.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZXhjZXB0aW9uL1F1ZXJ5RXhjZXB0aW9uLmphdmE=) | `90.27% <ø> (+5.55%)` | :arrow_up: |
   | [...pinot/common/function/AggregationFunctionType.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vQWdncmVnYXRpb25GdW5jdGlvblR5cGUuamF2YQ==) | `98.27% <ø> (-1.73%)` | :arrow_down: |
   | [.../pinot/common/function/DateTimePatternHandler.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRGF0ZVRpbWVQYXR0ZXJuSGFuZGxlci5qYXZh) | `83.33% <ø> (ø)` | |
   | [...ot/common/function/FunctionDefinitionRegistry.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRnVuY3Rpb25EZWZpbml0aW9uUmVnaXN0cnkuamF2YQ==) | `88.88% <ø> (+44.44%)` | :arrow_up: |
   | [...org/apache/pinot/common/function/FunctionInfo.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRnVuY3Rpb25JbmZvLmphdmE=) | `100.00% <ø> (ø)` | |
   | ... and [1066 more](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=footer). Last update [41a7722...e3ec220](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739429849


   @Jackie-Jiang with the last commit there was another performance improvement and a reduction in allocation rate following your suggestions. I have also tried to adhere to the style guidance.
   
   before
   ```
   Benchmark                                          (assignment)  (cardinality)  (numDocs)  Mode  Cnt   Score   Error  Units
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN             10    1000000    ss    5   0.030 ± 0.003   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN             10   10000000    ss    5   0.374 ± 0.265   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN             10  100000000    ss    5   4.077 ± 0.191   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN           1000    1000000    ss    5   0.090 ± 0.012   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN           1000   10000000    ss    5   1.070 ± 0.304   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN           1000  100000000    ss    5  11.465 ± 0.308   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN          10000    1000000    ss    5   0.152 ± 0.040   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN          10000   10000000    ss    5   1.639 ± 0.067   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN          10000  100000000    ss    5  17.002 ± 0.217   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM             10    1000000    ss    5   0.021 ± 0.002   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM             10   10000000    ss    5   0.212 ± 0.004   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM             10  100000000    ss    5   2.283 ± 0.109   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM           1000    1000000    ss    5   0.075 ± 0.002   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM           1000   10000000    ss    5   0.296 ± 0.189   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM           1000  100000000    ss    5   2.094 ± 0.284   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM          10000    1000000    ss    5   0.093 ± 0.014   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM          10000   10000000    ss    5   0.891 ± 0.327   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM          10000  100000000    ss    5   3.736 ± 0.067   s/op
   ```
   
   after
   ```
   Benchmark                                          (assignment)  (cardinality)  (numDocs)  Mode  Cnt  Score   Error  Units
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN             10    1000000    ss    5  0.016 ± 0.004   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN             10   10000000    ss    5  0.179 ± 0.021   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN             10  100000000    ss    5  1.711 ± 0.014   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN           1000    1000000    ss    5  0.024 ± 0.002   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN           1000   10000000    ss    5  0.233 ± 0.034   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN           1000  100000000    ss    5  2.118 ± 0.007   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN          10000    1000000    ss    5  0.048 ± 0.025   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN          10000   10000000    ss    5  0.401 ± 0.090   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN          10000  100000000    ss    5  3.575 ± 0.007   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM             10    1000000    ss    5  0.014 ± 0.002   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM             10   10000000    ss    5  0.123 ± 0.004   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM             10  100000000    ss    5  1.444 ± 0.101   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM           1000    1000000    ss    5  0.018 ± 0.002   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM           1000   10000000    ss    5  0.147 ± 0.029   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM           1000  100000000    ss    5  1.513 ± 0.082   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM          10000    1000000    ss    5  0.022 ± 0.007   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM          10000   10000000    ss    5  0.175 ± 0.022   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM          10000  100000000    ss    5  1.350 ± 0.040   s/op
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r537828464



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +184,45 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      // map the offsets buffer
+      int startOfBitmaps = (_cardinality + 1) * Integer.BYTES;
+      offsetBuffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, startOfBitmaps)
+              .order(ByteOrder.BIG_ENDIAN);
+      offsetBuffer.putInt(startOfBitmaps);
+      bitmapBuffer = channel.map(FileChannel.MapMode.READ_WRITE, startOfBitmaps, Integer.MAX_VALUE - startOfBitmaps)
+              .order(ByteOrder.LITTLE_ENDIAN);
+      RoaringBitmapWriter<RoaringBitmap> writer = RoaringBitmapWriter.writer().get();

Review comment:
       I disabled run compression, which will save some cycles if this is only used for sparse bitmaps.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r537124621



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OnHeapBitmapInvertedIndexCreator.java
##########
@@ -19,68 +19,96 @@
 package org.apache.pinot.core.segment.creator.impl.inv;
 
 import com.google.common.base.Preconditions;
-import java.io.BufferedOutputStream;
-import java.io.DataOutputStream;
+
+
 import java.io.File;
-import java.io.FileOutputStream;
 import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.ByteBuffer;
+import java.nio.ByteOrder;
+import java.nio.channels.FileChannel;
+
 import org.apache.commons.io.FileUtils;
 import org.apache.pinot.core.segment.creator.DictionaryBasedInvertedIndexCreator;
 import org.apache.pinot.core.segment.creator.impl.V1Constants;
-import org.roaringbitmap.buffer.MutableRoaringBitmap;
+import org.apache.pinot.core.util.CleanerUtil;
+import org.roaringbitmap.RoaringBitmap;
+import org.roaringbitmap.RoaringBitmapWriter;
 
 
 /**
  * Implementation of {@link DictionaryBasedInvertedIndexCreator} that uses on-heap memory.
  */
 public final class OnHeapBitmapInvertedIndexCreator implements DictionaryBasedInvertedIndexCreator {
   private final File _invertedIndexFile;
-  private final MutableRoaringBitmap[] _bitmaps;
+  private final RoaringBitmapWriter<RoaringBitmap>[] _bitmapWriters;
   private int _nextDocId;
 
+  @SuppressWarnings("unchecked")
   public OnHeapBitmapInvertedIndexCreator(File indexDir, String columnName, int cardinality) {
     _invertedIndexFile = new File(indexDir, columnName + V1Constants.Indexes.BITMAP_INVERTED_INDEX_FILE_EXTENSION);
-    _bitmaps = new MutableRoaringBitmap[cardinality];
+    _bitmapWriters = new RoaringBitmapWriter[cardinality];
     for (int i = 0; i < cardinality; i++) {
-      _bitmaps[i] = new MutableRoaringBitmap();
+      _bitmapWriters[i] = RoaringBitmapWriter.writer().get();
     }
   }
 
   @Override
   public void add(int dictId) {
-    _bitmaps[dictId].add(_nextDocId++);
+    _bitmapWriters[dictId].add(_nextDocId++);
   }
 
   @Override
   public void add(int[] dictIds, int length) {
     for (int i = 0; i < length; i++) {
-      _bitmaps[dictIds[i]].add(_nextDocId);
+      _bitmapWriters[dictIds[i]].add(_nextDocId);
     }
     _nextDocId++;
   }
 
   @Override
   public void seal()
       throws IOException {
-    try (DataOutputStream out = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)))) {
+    // calculate file size
+    int startOfBitmaps = (_bitmapWriters.length + 1) * Integer.BYTES;
+    int size = 0;
+    for (RoaringBitmapWriter<RoaringBitmap> writer : _bitmapWriters) {
+      size += writer.get().serializedSizeInBytes();

Review comment:
       Maybe we can use the same way as the off-heap creator to preallocate a huge buffer then truncate to save the calculation of the serialized size (we calculate that twice for each bitmap)?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] kishoreg commented on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
kishoreg commented on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739781501


   > @Jackie-Jiang since the branch is churning somewhat do you mind if I squash the commits to what we have so far and force push?
   
   it's ok to do that if it helps you. Note that, we automatically perform squash and merge during the final merge. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739096511


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=h1) Report
   > Merging [#6320](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=desc) (e3ec220) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **decrease** coverage by `1.08%`.
   > The diff coverage is `45.12%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6320/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #6320      +/-   ##
   ==========================================
   - Coverage   66.44%   65.36%   -1.09%     
   ==========================================
     Files        1075     1281     +206     
     Lines       54773    61823    +7050     
     Branches     8168     8986     +818     
   ==========================================
   + Hits        36396    40412    +4016     
   - Misses      15700    18521    +2821     
   - Partials     2677     2890     +213     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `65.36% <45.12%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...e/pinot/broker/api/resources/PinotBrokerDebug.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYXBpL3Jlc291cmNlcy9QaW5vdEJyb2tlckRlYnVnLmphdmE=) | `0.00% <0.00%> (-79.32%)` | :arrow_down: |
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `71.42% <ø> (-28.58%)` | :arrow_down: |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `33.96% <0.00%> (-32.71%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ava/org/apache/pinot/client/AbstractResultSet.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Fic3RyYWN0UmVzdWx0U2V0LmphdmE=) | `66.66% <0.00%> (+9.52%)` | :arrow_up: |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `35.55% <0.00%> (-13.29%)` | :arrow_down: |
   | [...inot/client/JsonAsyncHttpPinotClientTransport.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0pzb25Bc3luY0h0dHBQaW5vdENsaWVudFRyYW5zcG9ydC5qYXZh) | `10.90% <0.00%> (-51.10%)` | :arrow_down: |
   | [...not/common/assignment/InstancePartitionsUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXNzaWdubWVudC9JbnN0YW5jZVBhcnRpdGlvbnNVdGlscy5qYXZh) | `73.80% <ø> (+0.63%)` | :arrow_up: |
   | [.../apache/pinot/common/exception/QueryException.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZXhjZXB0aW9uL1F1ZXJ5RXhjZXB0aW9uLmphdmE=) | `90.27% <ø> (+5.55%)` | :arrow_up: |
   | [...pinot/common/function/AggregationFunctionType.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vQWdncmVnYXRpb25GdW5jdGlvblR5cGUuamF2YQ==) | `98.27% <ø> (-1.73%)` | :arrow_down: |
   | ... and [1128 more](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=footer). Last update [41a7722...e3ec220](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r537789457



##########
File path: pinot-perf/src/main/java/org/apache/pinot/perf/BenchmarkOffheapBitmapInvertedIndexCreator.java
##########
@@ -0,0 +1,105 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.pinot.perf;
+
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.core.segment.creator.impl.inv.OffHeapBitmapInvertedIndexCreator;
+import org.apache.pinot.spi.data.DimensionFieldSpec;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.openjdk.jmh.annotations.*;
+import org.openjdk.jmh.runner.Runner;
+import org.openjdk.jmh.runner.options.ChainedOptionsBuilder;
+import org.openjdk.jmh.runner.options.OptionsBuilder;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+
+@State(Scope.Benchmark)
+public class BenchmarkOffheapBitmapInvertedIndexCreator {

Review comment:
       Please reformat this file to fix the indentation (should be 2 spaces) for this file

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +184,45 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      // map the offsets buffer
+      int startOfBitmaps = (_cardinality + 1) * Integer.BYTES;
+      offsetBuffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, startOfBitmaps)
+              .order(ByteOrder.BIG_ENDIAN);
+      offsetBuffer.putInt(startOfBitmaps);
+      bitmapBuffer = channel.map(FileChannel.MapMode.READ_WRITE, startOfBitmaps, Integer.MAX_VALUE - startOfBitmaps)
+              .order(ByteOrder.LITTLE_ENDIAN);
+      RoaringBitmapWriter<RoaringBitmap> writer = RoaringBitmapWriter.writer().get();

Review comment:
       I see your point. We won't create bitmap inverted index if the values are sorted (we have another RLE format), so in most cases, the docIds will spread over all the arrays. But for high cardinality case where each bitmap only contain very few values, pre-allocation might end up allocating too many arrays. So agree with letting it grow automatically.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r537123298



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +188,51 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      // map the offsets buffer
+      final int startOfBitmaps = (_cardinality + 1) * Integer.BYTES;
+      int bitmapOffset = startOfBitmaps;
+      offsetBuffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, bitmapOffset).order(LITTLE_ENDIAN);
+      offsetBuffer.putInt(reverseBytes(bitmapOffset));
+      RoaringBitmap[] bitmaps = new RoaringBitmap[_cardinality];
+      RoaringBitmapWriter<RoaringBitmap> writer = RoaringBitmapWriter.writer()
+              .initialCapacity(((_nextDocId - 1) >>> 16) / _cardinality).get();

Review comment:
       From my understanding, this method preallocate the arrays based on the size of the prefix keys. Since we are going to reuse this writer for all the bitmaps, we will always end up using all the prefix keys from `(0, _nextDocId)`, and we should use `.expectedRange(0, _nextDocId)` to preallocate all of them. Maybe I missed something here?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r536916030



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OnHeapBitmapInvertedIndexCreator.java
##########
@@ -19,68 +19,85 @@
 package org.apache.pinot.core.segment.creator.impl.inv;
 
 import com.google.common.base.Preconditions;
-import java.io.BufferedOutputStream;
-import java.io.DataOutputStream;
+
+
 import java.io.File;
-import java.io.FileOutputStream;
 import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.ByteBuffer;
+import java.nio.channels.FileChannel;
+
 import org.apache.commons.io.FileUtils;
 import org.apache.pinot.core.segment.creator.DictionaryBasedInvertedIndexCreator;
 import org.apache.pinot.core.segment.creator.impl.V1Constants;
+import org.apache.pinot.core.util.CleanerUtil;
+import org.roaringbitmap.RoaringBitmapWriter;
 import org.roaringbitmap.buffer.MutableRoaringBitmap;
 
+import static java.nio.ByteOrder.LITTLE_ENDIAN;
+
 
 /**
  * Implementation of {@link DictionaryBasedInvertedIndexCreator} that uses on-heap memory.
  */
 public final class OnHeapBitmapInvertedIndexCreator implements DictionaryBasedInvertedIndexCreator {
   private final File _invertedIndexFile;
-  private final MutableRoaringBitmap[] _bitmaps;
+  private final RoaringBitmapWriter<MutableRoaringBitmap>[] _bitmapWriters;
   private int _nextDocId;
 
+  @SuppressWarnings("unchecked")
   public OnHeapBitmapInvertedIndexCreator(File indexDir, String columnName, int cardinality) {
     _invertedIndexFile = new File(indexDir, columnName + V1Constants.Indexes.BITMAP_INVERTED_INDEX_FILE_EXTENSION);
-    _bitmaps = new MutableRoaringBitmap[cardinality];
+    _bitmapWriters = new RoaringBitmapWriter[cardinality];
     for (int i = 0; i < cardinality; i++) {
-      _bitmaps[i] = new MutableRoaringBitmap();
+      _bitmapWriters[i] = RoaringBitmapWriter.bufferWriter().get();
     }
   }
 
   @Override
   public void add(int dictId) {
-    _bitmaps[dictId].add(_nextDocId++);
+    _bitmapWriters[dictId].add(_nextDocId++);
   }
 
   @Override
   public void add(int[] dictIds, int length) {
     for (int i = 0; i < length; i++) {
-      _bitmaps[dictIds[i]].add(_nextDocId);
+      _bitmapWriters[dictIds[i]].add(_nextDocId);
     }
     _nextDocId++;
   }
 
   @Override
   public void seal()
       throws IOException {
-    try (DataOutputStream out = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)))) {
+    // calculate file size
+    int size = (_bitmapWriters.length + 1) * Integer.BYTES;
+    for (RoaringBitmapWriter<MutableRoaringBitmap> writer : _bitmapWriters) {
+      size += writer.get().serializedSizeInBytes();
+      // Check for int overflow
+      Preconditions.checkState(size > 0, "Inverted index file: %s exceeds 2GB limit", _invertedIndexFile);
+    }
+    ByteBuffer buffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      buffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, size).order(LITTLE_ENDIAN);

Review comment:
       I was writing big endian bytes _to a part of the buffer_ whereas the bulk of the file is serialised by the roaring bitmap library, which is little endian by specification. So I was doing a little extra work on the offsets, to favour the bitmaps. Does that make sense?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739096511


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=h1) Report
   > Merging [#6320](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=desc) (0a7af36) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **increase** coverage by `7.67%`.
   > The diff coverage is `68.40%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6320/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #6320      +/-   ##
   ==========================================
   + Coverage   66.44%   74.12%   +7.67%     
   ==========================================
     Files        1075     1281     +206     
     Lines       54773    61813    +7040     
     Branches     8168     8986     +818     
   ==========================================
   + Hits        36396    45818    +9422     
   + Misses      15700    13033    -2667     
   - Partials     2677     2962     +285     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration | `45.65% <52.01%> (?)` | |
   | unittests | `65.34% <45.12%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `52.83% <0.00%> (-13.84%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `44.44% <0.00%> (-4.40%)` | :arrow_down: |
   | [...not/common/assignment/InstancePartitionsUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXNzaWdubWVudC9JbnN0YW5jZVBhcnRpdGlvbnNVdGlscy5qYXZh) | `78.57% <ø> (+5.40%)` | :arrow_up: |
   | [.../apache/pinot/common/exception/QueryException.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZXhjZXB0aW9uL1F1ZXJ5RXhjZXB0aW9uLmphdmE=) | `90.27% <ø> (+5.55%)` | :arrow_up: |
   | [...pinot/common/function/AggregationFunctionType.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vQWdncmVnYXRpb25GdW5jdGlvblR5cGUuamF2YQ==) | `98.27% <ø> (-1.73%)` | :arrow_down: |
   | [.../pinot/common/function/DateTimePatternHandler.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRGF0ZVRpbWVQYXR0ZXJuSGFuZGxlci5qYXZh) | `83.33% <ø> (ø)` | |
   | [...ot/common/function/FunctionDefinitionRegistry.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRnVuY3Rpb25EZWZpbml0aW9uUmVnaXN0cnkuamF2YQ==) | `88.88% <ø> (+44.44%)` | :arrow_up: |
   | [...org/apache/pinot/common/function/FunctionInfo.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRnVuY3Rpb25JbmZvLmphdmE=) | `100.00% <ø> (ø)` | |
   | ... and [1066 more](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=footer). Last update [41a7722...0a7af36](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r536912358



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +188,49 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {

Review comment:
       I had a hard time believing it when this came up, but did enough profiling to figure out what was going on. The difference is _up to_ 20x, and it depends on the composition of the bitmap: `DataOutput` penalises dense bitmaps more than others. `DataOutput` has to do things like manually extract bytes from `int`, `long` etc. and this adds up to a lot compared to a simple move instruction. Because of endianness, you get away without reversing the bytes, and with `ByteBuffer` various bounds checks get eliminated. You can see the difference even with a pre-sized `ByteArrayOutputStream`, so it's not a question of unbuffered IO. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] Jackie-Jiang commented on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739398210


   Find this blog from @richardstartin explaining the optimization very well: https://richardstartin.github.io/posts/roaringbitmaps-construction-performance


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] Jackie-Jiang merged pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang merged pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r536910850



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +188,51 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      // map the offsets buffer
+      final int startOfBitmaps = (_cardinality + 1) * Integer.BYTES;
+      int bitmapOffset = startOfBitmaps;
+      offsetBuffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, bitmapOffset).order(LITTLE_ENDIAN);
+      offsetBuffer.putInt(reverseBytes(bitmapOffset));
+      RoaringBitmap[] bitmaps = new RoaringBitmap[_cardinality];

Review comment:
       > Do RoaringBitmap and MutableRoaringBitmap serialize to the same bytes?
   
   Yes, they do.
   
   > For off-heap creator, we don't want to keep all bitmaps on heap. We should try to create and serialize the bitmaps one by one. Not sure about the cost if we map a buffer per bitmap
   
   I can see the incentive not to allocate so much on heap, but the alternative is mapping many times (and it's a relatively costly operation) or mapping too many bytes and truncating the channel. I have done the latter in the past to get round this, but there may be downsides I'm unaware of. I've updated to use this strategy and allocation pressure is indeed reduced.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r537874091



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +184,45 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      // map the offsets buffer
+      int startOfBitmaps = (_cardinality + 1) * Integer.BYTES;
+      offsetBuffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, startOfBitmaps)
+              .order(ByteOrder.BIG_ENDIAN);
+      offsetBuffer.putInt(startOfBitmaps);
+      bitmapBuffer = channel.map(FileChannel.MapMode.READ_WRITE, startOfBitmaps, Integer.MAX_VALUE - startOfBitmaps)
+              .order(ByteOrder.LITTLE_ENDIAN);
+      RoaringBitmapWriter<RoaringBitmap> writer = RoaringBitmapWriter.writer().get();

Review comment:
       No run length encoding was being done before




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r537132814



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OnHeapBitmapInvertedIndexCreator.java
##########
@@ -19,68 +19,96 @@
 package org.apache.pinot.core.segment.creator.impl.inv;
 
 import com.google.common.base.Preconditions;
-import java.io.BufferedOutputStream;
-import java.io.DataOutputStream;
+
+
 import java.io.File;
-import java.io.FileOutputStream;
 import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.ByteBuffer;
+import java.nio.ByteOrder;
+import java.nio.channels.FileChannel;
+
 import org.apache.commons.io.FileUtils;
 import org.apache.pinot.core.segment.creator.DictionaryBasedInvertedIndexCreator;
 import org.apache.pinot.core.segment.creator.impl.V1Constants;
-import org.roaringbitmap.buffer.MutableRoaringBitmap;
+import org.apache.pinot.core.util.CleanerUtil;
+import org.roaringbitmap.RoaringBitmap;
+import org.roaringbitmap.RoaringBitmapWriter;
 
 
 /**
  * Implementation of {@link DictionaryBasedInvertedIndexCreator} that uses on-heap memory.
  */
 public final class OnHeapBitmapInvertedIndexCreator implements DictionaryBasedInvertedIndexCreator {
   private final File _invertedIndexFile;
-  private final MutableRoaringBitmap[] _bitmaps;
+  private final RoaringBitmapWriter<RoaringBitmap>[] _bitmapWriters;
   private int _nextDocId;
 
+  @SuppressWarnings("unchecked")
   public OnHeapBitmapInvertedIndexCreator(File indexDir, String columnName, int cardinality) {
     _invertedIndexFile = new File(indexDir, columnName + V1Constants.Indexes.BITMAP_INVERTED_INDEX_FILE_EXTENSION);
-    _bitmaps = new MutableRoaringBitmap[cardinality];
+    _bitmapWriters = new RoaringBitmapWriter[cardinality];
     for (int i = 0; i < cardinality; i++) {
-      _bitmaps[i] = new MutableRoaringBitmap();
+      _bitmapWriters[i] = RoaringBitmapWriter.writer().get();
     }
   }
 
   @Override
   public void add(int dictId) {
-    _bitmaps[dictId].add(_nextDocId++);
+    _bitmapWriters[dictId].add(_nextDocId++);
   }
 
   @Override
   public void add(int[] dictIds, int length) {
     for (int i = 0; i < length; i++) {
-      _bitmaps[dictIds[i]].add(_nextDocId);
+      _bitmapWriters[dictIds[i]].add(_nextDocId);
     }
     _nextDocId++;
   }
 
   @Override
   public void seal()
       throws IOException {
-    try (DataOutputStream out = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)))) {
+    // calculate file size
+    int startOfBitmaps = (_bitmapWriters.length + 1) * Integer.BYTES;
+    int size = 0;
+    for (RoaringBitmapWriter<RoaringBitmap> writer : _bitmapWriters) {
+      size += writer.get().serializedSizeInBytes();

Review comment:
       Yes, good suggestion, I applied this change. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739096511


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=h1) Report
   > Merging [#6320](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=desc) (72e9d8d) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **decrease** coverage by `20.81%`.
   > The diff coverage is `52.01%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6320/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master    #6320       +/-   ##
   ===========================================
   - Coverage   66.44%   45.63%   -20.82%     
   ===========================================
     Files        1075     1283      +208     
     Lines       54773    61937     +7164     
     Branches     8168     8996      +828     
   ===========================================
   - Hits        36396    28265     -8131     
   - Misses      15700    31364    +15664     
   + Partials     2677     2308      -369     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration | `45.63% <52.01%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `52.83% <0.00%> (-13.84%)` | :arrow_down: |
   | [...org/apache/pinot/broker/queryquota/HitCounter.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcXVlcnlxdW90YS9IaXRDb3VudGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...che/pinot/broker/queryquota/MaxHitRateTracker.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcXVlcnlxdW90YS9NYXhIaXRSYXRlVHJhY2tlci5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...ache/pinot/broker/queryquota/QueryQuotaEntity.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcXVlcnlxdW90YS9RdWVyeVF1b3RhRW50aXR5LmphdmE=) | `0.00% <0.00%> (-50.00%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ceselector/StrictReplicaGroupInstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL1N0cmljdFJlcGxpY2FHcm91cEluc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `22.22% <0.00%> (-26.62%)` | :arrow_down: |
   | [...not/common/assignment/InstancePartitionsUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXNzaWdubWVudC9JbnN0YW5jZVBhcnRpdGlvbnNVdGlscy5qYXZh) | `64.28% <ø> (-8.89%)` | :arrow_down: |
   | [.../apache/pinot/common/exception/QueryException.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZXhjZXB0aW9uL1F1ZXJ5RXhjZXB0aW9uLmphdmE=) | `90.27% <ø> (+5.55%)` | :arrow_up: |
   | ... and [1278 more](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=footer). Last update [41a7722...0d90376](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739096511


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=h1) Report
   > Merging [#6320](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=desc) (92d9953) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **decrease** coverage by `1.09%`.
   > The diff coverage is `45.12%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6320/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #6320      +/-   ##
   ==========================================
   - Coverage   66.44%   65.35%   -1.10%     
   ==========================================
     Files        1075     1281     +206     
     Lines       54773    61815    +7042     
     Branches     8168     8983     +815     
   ==========================================
   + Hits        36396    40397    +4001     
   - Misses      15700    18527    +2827     
   - Partials     2677     2891     +214     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `65.35% <45.12%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...e/pinot/broker/api/resources/PinotBrokerDebug.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYXBpL3Jlc291cmNlcy9QaW5vdEJyb2tlckRlYnVnLmphdmE=) | `0.00% <0.00%> (-79.32%)` | :arrow_down: |
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `71.42% <ø> (-28.58%)` | :arrow_down: |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `33.96% <0.00%> (-32.71%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ava/org/apache/pinot/client/AbstractResultSet.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Fic3RyYWN0UmVzdWx0U2V0LmphdmE=) | `66.66% <0.00%> (+9.52%)` | :arrow_up: |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `35.55% <0.00%> (-13.29%)` | :arrow_down: |
   | [...inot/client/JsonAsyncHttpPinotClientTransport.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0pzb25Bc3luY0h0dHBQaW5vdENsaWVudFRyYW5zcG9ydC5qYXZh) | `10.90% <0.00%> (-51.10%)` | :arrow_down: |
   | [...not/common/assignment/InstancePartitionsUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXNzaWdubWVudC9JbnN0YW5jZVBhcnRpdGlvbnNVdGlscy5qYXZh) | `73.80% <ø> (+0.63%)` | :arrow_up: |
   | [.../apache/pinot/common/exception/QueryException.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZXhjZXB0aW9uL1F1ZXJ5RXhjZXB0aW9uLmphdmE=) | `90.27% <ø> (+5.55%)` | :arrow_up: |
   | [...pinot/common/function/AggregationFunctionType.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vQWdncmVnYXRpb25GdW5jdGlvblR5cGUuamF2YQ==) | `98.27% <ø> (-1.73%)` | :arrow_down: |
   | ... and [1128 more](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=footer). Last update [41a7722...92d9953](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739076353


   Regarding the test failure, this is because the bitmap is being run length encoded incrementally. It becomes deterministic when setting
   
   ```java
   _bitmapWriters[i] = RoaringBitmapWriter.bufferWriter().runCompress(false).get();
   ```
   
   To make the test deterministic, RLE could be disabled, but this is bad for file size. Or the off-heap index creator could run-optimise its bitmaps, but that's bad for speed compared to doing it incrementally. Alternatively the test could be modified to check that the bitmaps are equal after memory mapping. 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739154073


   I added a benchmark which times the single shot time to seal an off heap index creator, with a range of cardinalities, numbers of docs, and different assignment strategies of document ids to dictionary ids. One is the best case for roaring (sorted), the other is the worst case (round robin). The benchmark takes quite a long time to run so I only ran one fork and reduced the number of iterations, but this PR demonstrates a significant improvement:
   
   before
   ```
   Benchmark                                          (assignment)  (cardinality)  (numDocs)  Mode  Cnt   Score   Error  Units
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN             10    1000000    ss    5   0.030 ± 0.003   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN             10   10000000    ss    5   0.374 ± 0.265   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN             10  100000000    ss    5   4.077 ± 0.191   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN           1000    1000000    ss    5   0.090 ± 0.012   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN           1000   10000000    ss    5   1.070 ± 0.304   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN           1000  100000000    ss    5  11.465 ± 0.308   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN          10000    1000000    ss    5   0.152 ± 0.040   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN          10000   10000000    ss    5   1.639 ± 0.067   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN          10000  100000000    ss    5  17.002 ± 0.217   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM             10    1000000    ss    5   0.021 ± 0.002   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM             10   10000000    ss    5   0.212 ± 0.004   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM             10  100000000    ss    5   2.283 ± 0.109   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM           1000    1000000    ss    5   0.075 ± 0.002   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM           1000   10000000    ss    5   0.296 ± 0.189   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM           1000  100000000    ss    5   2.094 ± 0.284   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM          10000    1000000    ss    5   0.093 ± 0.014   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM          10000   10000000    ss    5   0.891 ± 0.327   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM          10000  100000000    ss    5   3.736 ± 0.067   s/op
   ```
   
   after
   ```
   Benchmark                                          (assignment)  (cardinality)  (numDocs)  Mode  Cnt  Score   Error  Units
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN             10    1000000    ss    5  0.021 ± 0.011   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN             10   10000000    ss    5  0.195 ± 0.026   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN             10  100000000    ss    5  1.966 ± 0.133   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN           1000    1000000    ss    5  0.032 ± 0.009   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN           1000   10000000    ss    5  0.287 ± 0.108   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN           1000  100000000    ss    5  2.943 ± 0.176   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN          10000    1000000    ss    5  0.057 ± 0.036   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN          10000   10000000    ss    5  0.491 ± 0.131   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal     ROUND_ROBIN          10000  100000000    ss    5  5.502 ± 0.225   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM             10    1000000    ss    5  0.018 ± 0.005   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM             10   10000000    ss    5  0.159 ± 0.012   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM             10  100000000    ss    5  1.554 ± 0.028   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM           1000    1000000    ss    5  0.021 ± 0.006   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM           1000   10000000    ss    5  0.181 ± 0.028   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM           1000  100000000    ss    5  1.580 ± 0.028   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM          10000    1000000    ss    5  0.031 ± 0.003   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM          10000   10000000    ss    5  0.193 ± 0.020   s/op
   BenchmarkOffheapBitmapInvertedIndexCreator.seal  SORTED_UNIFORM          10000  100000000    ss    5  1.752 ± 0.089   s/op
   ```
   
   Please review the benchmark and let me know if you think the improvements look interesting - I am not going to be able to run cluster scale performance tests on this, but have prior experience of these APIs reducing resource cost in the system I wrote them for.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r536887337



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -19,18 +19,25 @@
 package org.apache.pinot.core.segment.creator.impl.inv;
 
 import com.google.common.base.Preconditions;
-import java.io.BufferedOutputStream;
+
 import java.io.Closeable;
-import java.io.DataOutputStream;
 import java.io.File;
-import java.io.FileOutputStream;
 import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.ByteBuffer;
+import java.nio.channels.FileChannel;
+
 import org.apache.commons.io.FileUtils;
 import org.apache.pinot.core.segment.creator.DictionaryBasedInvertedIndexCreator;
 import org.apache.pinot.core.segment.creator.impl.V1Constants;
 import org.apache.pinot.core.segment.memory.PinotDataBuffer;
+import org.apache.pinot.core.util.CleanerUtil;
 import org.apache.pinot.spi.data.FieldSpec;
-import org.roaringbitmap.buffer.MutableRoaringBitmap;
+import org.roaringbitmap.RoaringBitmap;
+import org.roaringbitmap.RoaringBitmapWriter;
+
+import static java.lang.Integer.reverseBytes;

Review comment:
       Don't use static import in production class

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +188,51 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      // map the offsets buffer
+      final int startOfBitmaps = (_cardinality + 1) * Integer.BYTES;
+      int bitmapOffset = startOfBitmaps;
+      offsetBuffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, bitmapOffset).order(LITTLE_ENDIAN);
+      offsetBuffer.putInt(reverseBytes(bitmapOffset));
+      RoaringBitmap[] bitmaps = new RoaringBitmap[_cardinality];

Review comment:
       For off-heap creator, we don't want to keep all bitmaps on heap. We should try to create and serialize the bitmaps one by one. Not sure about the cost if we map a buffer per bitmap

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +188,51 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      // map the offsets buffer
+      final int startOfBitmaps = (_cardinality + 1) * Integer.BYTES;
+      int bitmapOffset = startOfBitmaps;
+      offsetBuffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, bitmapOffset).order(LITTLE_ENDIAN);

Review comment:
       The offset buffer does not have to be LE, as all the values are written as BE

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +188,51 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      // map the offsets buffer
+      final int startOfBitmaps = (_cardinality + 1) * Integer.BYTES;

Review comment:
       (nit) we don't usually use final for local variable

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +188,51 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      // map the offsets buffer
+      final int startOfBitmaps = (_cardinality + 1) * Integer.BYTES;
+      int bitmapOffset = startOfBitmaps;
+      offsetBuffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, bitmapOffset).order(LITTLE_ENDIAN);
+      offsetBuffer.putInt(reverseBytes(bitmapOffset));
+      RoaringBitmap[] bitmaps = new RoaringBitmap[_cardinality];

Review comment:
       Do `RoaringBitmap` and `MutableRoaringBitmap` serialize to the same bytes?

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +188,51 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      // map the offsets buffer
+      final int startOfBitmaps = (_cardinality + 1) * Integer.BYTES;
+      int bitmapOffset = startOfBitmaps;
+      offsetBuffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, bitmapOffset).order(LITTLE_ENDIAN);
+      offsetBuffer.putInt(reverseBytes(bitmapOffset));
+      RoaringBitmap[] bitmaps = new RoaringBitmap[_cardinality];
+      RoaringBitmapWriter<RoaringBitmap> writer = RoaringBitmapWriter.writer()
+              .initialCapacity(((_nextDocId - 1) >>> 16) / _cardinality).get();

Review comment:
       Why do you need to divide it with `_cardinality`? Though this should be `.expectedRange(0, _nextDocId)`?

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +188,51 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      // map the offsets buffer
+      final int startOfBitmaps = (_cardinality + 1) * Integer.BYTES;
+      int bitmapOffset = startOfBitmaps;
+      offsetBuffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, bitmapOffset).order(LITTLE_ENDIAN);
+      offsetBuffer.putInt(reverseBytes(bitmapOffset));
+      RoaringBitmap[] bitmaps = new RoaringBitmap[_cardinality];
+      RoaringBitmapWriter<RoaringBitmap> writer = RoaringBitmapWriter.writer()
+              .initialCapacity(((_nextDocId - 1) >>> 16) / _cardinality).get();
       int startIndex = 0;
       for (int dictId = 0; dictId < _cardinality; dictId++) {
-        MutableRoaringBitmap bitmap = new MutableRoaringBitmap();
         int endIndex = getInt(_invertedIndexLengthBuffer, dictId);
         for (int i = startIndex; i < endIndex; i++) {
-          bitmap.add(getInt(_invertedIndexValueBuffer, i));
+          writer.add(getInt(_invertedIndexValueBuffer, i));
         }
-        startIndex = endIndex;
-
-        // Write offset and bitmap into file
-        bitmapOffset += bitmap.serializedSizeInBytes();
+        bitmaps[dictId] = writer.get();
+        writer.reset();
+        int serializedSize = bitmaps[dictId].serializedSizeInBytes();
+        bitmapOffset += serializedSize;
         // Check for int overflow
         Preconditions.checkState(bitmapOffset > 0, "Inverted index file: %s exceeds 2GB limit", _invertedIndexFile);
-        offsetDataStream.writeInt(bitmapOffset);
-        bitmap.serialize(bitmapDataStream);
+        // write offset into file
+        offsetBuffer.putInt(reverseBytes(bitmapOffset));
+        startIndex = endIndex;
+      }
+      // we know how long the file should be now, so can map it
+      bitmapBuffer = channel.map(FileChannel.MapMode.READ_WRITE, startOfBitmaps, bitmapOffset - startOfBitmaps);
+      for (RoaringBitmap bitmap : bitmaps) {
+        bitmap.serialize(bitmapBuffer);
       }
     } catch (Exception e) {
       FileUtils.deleteQuietly(_invertedIndexFile);
       throw e;
+    } finally {
+      if (CleanerUtil.UNMAP_SUPPORTED) {
+        CleanerUtil.BufferCleaner cleaner = CleanerUtil.getCleaner();
+        if (null != offsetBuffer) {

Review comment:
       (nit) same for other places
   ```suggestion
           if (offsetBuffer != null) {
   ```

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OnHeapBitmapInvertedIndexCreator.java
##########
@@ -19,68 +19,85 @@
 package org.apache.pinot.core.segment.creator.impl.inv;
 
 import com.google.common.base.Preconditions;
-import java.io.BufferedOutputStream;
-import java.io.DataOutputStream;
+
+
 import java.io.File;
-import java.io.FileOutputStream;
 import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.ByteBuffer;
+import java.nio.channels.FileChannel;
+
 import org.apache.commons.io.FileUtils;
 import org.apache.pinot.core.segment.creator.DictionaryBasedInvertedIndexCreator;
 import org.apache.pinot.core.segment.creator.impl.V1Constants;
+import org.apache.pinot.core.util.CleanerUtil;
+import org.roaringbitmap.RoaringBitmapWriter;
 import org.roaringbitmap.buffer.MutableRoaringBitmap;
 
+import static java.nio.ByteOrder.LITTLE_ENDIAN;
+
 
 /**
  * Implementation of {@link DictionaryBasedInvertedIndexCreator} that uses on-heap memory.
  */
 public final class OnHeapBitmapInvertedIndexCreator implements DictionaryBasedInvertedIndexCreator {
   private final File _invertedIndexFile;
-  private final MutableRoaringBitmap[] _bitmaps;
+  private final RoaringBitmapWriter<MutableRoaringBitmap>[] _bitmapWriters;
   private int _nextDocId;
 
+  @SuppressWarnings("unchecked")
   public OnHeapBitmapInvertedIndexCreator(File indexDir, String columnName, int cardinality) {
     _invertedIndexFile = new File(indexDir, columnName + V1Constants.Indexes.BITMAP_INVERTED_INDEX_FILE_EXTENSION);
-    _bitmaps = new MutableRoaringBitmap[cardinality];
+    _bitmapWriters = new RoaringBitmapWriter[cardinality];
     for (int i = 0; i < cardinality; i++) {
-      _bitmaps[i] = new MutableRoaringBitmap();
+      _bitmapWriters[i] = RoaringBitmapWriter.bufferWriter().get();
     }
   }
 
   @Override
   public void add(int dictId) {
-    _bitmaps[dictId].add(_nextDocId++);
+    _bitmapWriters[dictId].add(_nextDocId++);
   }
 
   @Override
   public void add(int[] dictIds, int length) {
     for (int i = 0; i < length; i++) {
-      _bitmaps[dictIds[i]].add(_nextDocId);
+      _bitmapWriters[dictIds[i]].add(_nextDocId);
     }
     _nextDocId++;
   }
 
   @Override
   public void seal()
       throws IOException {
-    try (DataOutputStream out = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)))) {
+    // calculate file size
+    int size = (_bitmapWriters.length + 1) * Integer.BYTES;
+    for (RoaringBitmapWriter<MutableRoaringBitmap> writer : _bitmapWriters) {
+      size += writer.get().serializedSizeInBytes();
+      // Check for int overflow
+      Preconditions.checkState(size > 0, "Inverted index file: %s exceeds 2GB limit", _invertedIndexFile);
+    }
+    ByteBuffer buffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      buffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, size).order(LITTLE_ENDIAN);
       // Write bitmap offsets
-      int bitmapOffset = (_bitmaps.length + 1) * Integer.BYTES;
-      out.writeInt(bitmapOffset);
-      for (MutableRoaringBitmap bitmap : _bitmaps) {
-        bitmapOffset += bitmap.serializedSizeInBytes();
-        // Check for int overflow
-        Preconditions.checkState(bitmapOffset > 0, "Inverted index file: %s exceeds 2GB limit", _invertedIndexFile);
-        out.writeInt(bitmapOffset);
+      int bitmapOffset = (_bitmapWriters.length + 1) * Integer.BYTES;
+      buffer.putInt(Integer.reverseBytes(bitmapOffset));
+      for (RoaringBitmapWriter<MutableRoaringBitmap> writer : _bitmapWriters) {
+        bitmapOffset += writer.getUnderlying().serializedSizeInBytes();

Review comment:
       Do you need to `flush()` before `getUnderlying()`? Or just use `get()` to retrieve the bitmap?

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OnHeapBitmapInvertedIndexCreator.java
##########
@@ -19,68 +19,85 @@
 package org.apache.pinot.core.segment.creator.impl.inv;
 
 import com.google.common.base.Preconditions;
-import java.io.BufferedOutputStream;
-import java.io.DataOutputStream;
+
+
 import java.io.File;
-import java.io.FileOutputStream;
 import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.ByteBuffer;
+import java.nio.channels.FileChannel;
+
 import org.apache.commons.io.FileUtils;
 import org.apache.pinot.core.segment.creator.DictionaryBasedInvertedIndexCreator;
 import org.apache.pinot.core.segment.creator.impl.V1Constants;
+import org.apache.pinot.core.util.CleanerUtil;
+import org.roaringbitmap.RoaringBitmapWriter;
 import org.roaringbitmap.buffer.MutableRoaringBitmap;
 
+import static java.nio.ByteOrder.LITTLE_ENDIAN;
+
 
 /**
  * Implementation of {@link DictionaryBasedInvertedIndexCreator} that uses on-heap memory.
  */
 public final class OnHeapBitmapInvertedIndexCreator implements DictionaryBasedInvertedIndexCreator {
   private final File _invertedIndexFile;
-  private final MutableRoaringBitmap[] _bitmaps;
+  private final RoaringBitmapWriter<MutableRoaringBitmap>[] _bitmapWriters;
   private int _nextDocId;
 
+  @SuppressWarnings("unchecked")
   public OnHeapBitmapInvertedIndexCreator(File indexDir, String columnName, int cardinality) {
     _invertedIndexFile = new File(indexDir, columnName + V1Constants.Indexes.BITMAP_INVERTED_INDEX_FILE_EXTENSION);
-    _bitmaps = new MutableRoaringBitmap[cardinality];
+    _bitmapWriters = new RoaringBitmapWriter[cardinality];
     for (int i = 0; i < cardinality; i++) {
-      _bitmaps[i] = new MutableRoaringBitmap();
+      _bitmapWriters[i] = RoaringBitmapWriter.bufferWriter().get();
     }
   }
 
   @Override
   public void add(int dictId) {
-    _bitmaps[dictId].add(_nextDocId++);
+    _bitmapWriters[dictId].add(_nextDocId++);
   }
 
   @Override
   public void add(int[] dictIds, int length) {
     for (int i = 0; i < length; i++) {
-      _bitmaps[dictIds[i]].add(_nextDocId);
+      _bitmapWriters[dictIds[i]].add(_nextDocId);
     }
     _nextDocId++;
   }
 
   @Override
   public void seal()
       throws IOException {
-    try (DataOutputStream out = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)))) {
+    // calculate file size
+    int size = (_bitmapWriters.length + 1) * Integer.BYTES;
+    for (RoaringBitmapWriter<MutableRoaringBitmap> writer : _bitmapWriters) {
+      size += writer.get().serializedSizeInBytes();
+      // Check for int overflow
+      Preconditions.checkState(size > 0, "Inverted index file: %s exceeds 2GB limit", _invertedIndexFile);
+    }
+    ByteBuffer buffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      buffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, size).order(LITTLE_ENDIAN);

Review comment:
       Maybe keeping 2 buffers similar to the off-heap one?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r537132715



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +188,51 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      // map the offsets buffer
+      final int startOfBitmaps = (_cardinality + 1) * Integer.BYTES;
+      int bitmapOffset = startOfBitmaps;
+      offsetBuffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, bitmapOffset).order(LITTLE_ENDIAN);
+      offsetBuffer.putInt(reverseBytes(bitmapOffset));
+      RoaringBitmap[] bitmaps = new RoaringBitmap[_cardinality];
+      RoaringBitmapWriter<RoaringBitmap> writer = RoaringBitmapWriter.writer()
+              .initialCapacity(((_nextDocId - 1) >>> 16) / _cardinality).get();

Review comment:
       I felt this heuristic was distracting so removed it. 
   
   In case this comes up in the future, expecting the range to be `[0, _nextDocId)` would allocate far too much memory for datasets sorted by the indexed dimension, but would be the right choice if the association between row id and dictionary id were uniformly random. Some in-memory metadata could probably be associated with each dictionary id (number of documents, min, max, density?) and would mean these methods could likely be used more effectively, and not in a way that overfits to a benchmark.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r536517138



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OnHeapBitmapInvertedIndexCreator.java
##########
@@ -19,68 +19,85 @@
 package org.apache.pinot.core.segment.creator.impl.inv;
 
 import com.google.common.base.Preconditions;
-import java.io.BufferedOutputStream;
-import java.io.DataOutputStream;
+
+
 import java.io.File;
-import java.io.FileOutputStream;
 import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.ByteBuffer;
+import java.nio.channels.FileChannel;
+
 import org.apache.commons.io.FileUtils;
 import org.apache.pinot.core.segment.creator.DictionaryBasedInvertedIndexCreator;
 import org.apache.pinot.core.segment.creator.impl.V1Constants;
+import org.apache.pinot.core.util.CleanerUtil;
+import org.roaringbitmap.RoaringBitmapWriter;
 import org.roaringbitmap.buffer.MutableRoaringBitmap;
 
+import static java.nio.ByteOrder.LITTLE_ENDIAN;
+
 
 /**
  * Implementation of {@link DictionaryBasedInvertedIndexCreator} that uses on-heap memory.
  */
 public final class OnHeapBitmapInvertedIndexCreator implements DictionaryBasedInvertedIndexCreator {
   private final File _invertedIndexFile;
-  private final MutableRoaringBitmap[] _bitmaps;
+  private final RoaringBitmapWriter<MutableRoaringBitmap>[] _bitmapWriters;
   private int _nextDocId;
 
+  @SuppressWarnings("unchecked")
   public OnHeapBitmapInvertedIndexCreator(File indexDir, String columnName, int cardinality) {
     _invertedIndexFile = new File(indexDir, columnName + V1Constants.Indexes.BITMAP_INVERTED_INDEX_FILE_EXTENSION);
-    _bitmaps = new MutableRoaringBitmap[cardinality];
+    _bitmapWriters = new RoaringBitmapWriter[cardinality];
     for (int i = 0; i < cardinality; i++) {
-      _bitmaps[i] = new MutableRoaringBitmap();
+      _bitmapWriters[i] = RoaringBitmapWriter.bufferWriter().get();
     }
   }
 
   @Override
   public void add(int dictId) {
-    _bitmaps[dictId].add(_nextDocId++);
+    _bitmapWriters[dictId].add(_nextDocId++);
   }
 
   @Override
   public void add(int[] dictIds, int length) {
     for (int i = 0; i < length; i++) {
-      _bitmaps[dictIds[i]].add(_nextDocId);
+      _bitmapWriters[dictIds[i]].add(_nextDocId);
     }
     _nextDocId++;
   }
 
   @Override
   public void seal()
       throws IOException {
-    try (DataOutputStream out = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)))) {
+    // calculate file size
+    int size = (_bitmapWriters.length + 1) * Integer.BYTES;
+    for (RoaringBitmapWriter<MutableRoaringBitmap> writer : _bitmapWriters) {
+      size += writer.get().serializedSizeInBytes();
+      // Check for int overflow
+      Preconditions.checkState(size > 0, "Inverted index file: %s exceeds 2GB limit", _invertedIndexFile);
+    }
+    ByteBuffer buffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      buffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, size).order(LITTLE_ENDIAN);

Review comment:
       I am assuming the choice of LITTLE_ENDIAN comes from the fact that index generation (typically happening on x86 machines) will not require byte swap when writing the index?
   
   When we load the index, we specifically use BIG_ENDIAN byte order because throughout the index generation code we use BIG_ENDIAN. This works with overhead of byte swap for both read and write. 
   
   I am confused how is this change expected to work today for the reader? The reader will load the file specifying the order as BIG_ENDIAN. This means every read operation will swap the bytes on x86. However, with this change index file is generated in LE format which doesn't require any swapping. So swapping bytes during inverted index read will result in incorrect data imo. I am surprised that tests are passing. 
   
   See the code in index buffer loader. This code doesn't know index file was written in LE format. 
   
   ```
   // Backward-compatible: index file is always big-endian
       PinotDataBuffer buffer;
       if (readMode == ReadMode.heap) {
         buffer = PinotDataBuffer.loadFile(indexFile, fromFilePos, size, ByteOrder.BIG_ENDIAN, context);
       } else {
         buffer = PinotDataBuffer.mapFile(indexFile, true, fromFilePos, size, ByteOrder.BIG_ENDIAN, context);
       }
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] kishoreg commented on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
kishoreg commented on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-740314450


   > I'm working on a feature which requires extracting the common part of the on-heap and off-heap inverted index creator. Can I add a commit to this PR?
   
   Let's not mix with this PR. We can do that after we push this.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739096511


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=h1) Report
   > Merging [#6320](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=desc) (725e218) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **decrease** coverage by `1.05%`.
   > The diff coverage is `45.12%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6320/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #6320      +/-   ##
   ==========================================
   - Coverage   66.44%   65.39%   -1.06%     
   ==========================================
     Files        1075     1283     +208     
     Lines       54773    61978    +7205     
     Branches     8168     8998     +830     
   ==========================================
   + Hits        36396    40528    +4132     
   - Misses      15700    18563    +2863     
   - Partials     2677     2887     +210     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `65.39% <45.12%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...e/pinot/broker/api/resources/PinotBrokerDebug.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYXBpL3Jlc291cmNlcy9QaW5vdEJyb2tlckRlYnVnLmphdmE=) | `0.00% <0.00%> (-79.32%)` | :arrow_down: |
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `71.42% <ø> (-28.58%)` | :arrow_down: |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `33.96% <0.00%> (-32.71%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ava/org/apache/pinot/client/AbstractResultSet.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Fic3RyYWN0UmVzdWx0U2V0LmphdmE=) | `66.66% <0.00%> (+9.52%)` | :arrow_up: |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `35.55% <0.00%> (-13.29%)` | :arrow_down: |
   | [...inot/client/JsonAsyncHttpPinotClientTransport.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0pzb25Bc3luY0h0dHBQaW5vdENsaWVudFRyYW5zcG9ydC5qYXZh) | `10.90% <0.00%> (-51.10%)` | :arrow_down: |
   | [...not/common/assignment/InstancePartitionsUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXNzaWdubWVudC9JbnN0YW5jZVBhcnRpdGlvbnNVdGlscy5qYXZh) | `73.80% <ø> (+0.63%)` | :arrow_up: |
   | [.../apache/pinot/common/exception/QueryException.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZXhjZXB0aW9uL1F1ZXJ5RXhjZXB0aW9uLmphdmE=) | `90.27% <ø> (+5.55%)` | :arrow_up: |
   | [...pinot/common/function/AggregationFunctionType.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vQWdncmVnYXRpb25GdW5jdGlvblR5cGUuamF2YQ==) | `100.00% <ø> (ø)` | |
   | ... and [1131 more](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=footer). Last update [41a7722...725e218](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739096511


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=h1) Report
   > Merging [#6320](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=desc) (725e218) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **increase** coverage by `7.67%`.
   > The diff coverage is `68.40%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6320/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #6320      +/-   ##
   ==========================================
   + Coverage   66.44%   74.11%   +7.67%     
   ==========================================
     Files        1075     1283     +208     
     Lines       54773    61978    +7205     
     Branches     8168     8998     +830     
   ==========================================
   + Hits        36396    45938    +9542     
   + Misses      15700    13075    -2625     
   - Partials     2677     2965     +288     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration | `45.57% <52.01%> (?)` | |
   | unittests | `65.39% <45.12%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `52.83% <0.00%> (-13.84%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `44.44% <0.00%> (-4.40%)` | :arrow_down: |
   | [...not/common/assignment/InstancePartitionsUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXNzaWdubWVudC9JbnN0YW5jZVBhcnRpdGlvbnNVdGlscy5qYXZh) | `78.57% <ø> (+5.40%)` | :arrow_up: |
   | [.../apache/pinot/common/exception/QueryException.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZXhjZXB0aW9uL1F1ZXJ5RXhjZXB0aW9uLmphdmE=) | `90.27% <ø> (+5.55%)` | :arrow_up: |
   | [...pinot/common/function/AggregationFunctionType.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vQWdncmVnYXRpb25GdW5jdGlvblR5cGUuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [.../pinot/common/function/DateTimePatternHandler.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRGF0ZVRpbWVQYXR0ZXJuSGFuZGxlci5qYXZh) | `83.33% <ø> (ø)` | |
   | [...ot/common/function/FunctionDefinitionRegistry.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRnVuY3Rpb25EZWZpbml0aW9uUmVnaXN0cnkuamF2YQ==) | `88.88% <ø> (+44.44%)` | :arrow_up: |
   | [...org/apache/pinot/common/function/FunctionInfo.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vRnVuY3Rpb25JbmZvLmphdmE=) | `100.00% <ø> (ø)` | |
   | ... and [1069 more](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=footer). Last update [41a7722...725e218](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r536911397



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OnHeapBitmapInvertedIndexCreator.java
##########
@@ -19,68 +19,85 @@
 package org.apache.pinot.core.segment.creator.impl.inv;
 
 import com.google.common.base.Preconditions;
-import java.io.BufferedOutputStream;
-import java.io.DataOutputStream;
+
+
 import java.io.File;
-import java.io.FileOutputStream;
 import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.ByteBuffer;
+import java.nio.channels.FileChannel;
+
 import org.apache.commons.io.FileUtils;
 import org.apache.pinot.core.segment.creator.DictionaryBasedInvertedIndexCreator;
 import org.apache.pinot.core.segment.creator.impl.V1Constants;
+import org.apache.pinot.core.util.CleanerUtil;
+import org.roaringbitmap.RoaringBitmapWriter;
 import org.roaringbitmap.buffer.MutableRoaringBitmap;
 
+import static java.nio.ByteOrder.LITTLE_ENDIAN;
+
 
 /**
  * Implementation of {@link DictionaryBasedInvertedIndexCreator} that uses on-heap memory.
  */
 public final class OnHeapBitmapInvertedIndexCreator implements DictionaryBasedInvertedIndexCreator {
   private final File _invertedIndexFile;
-  private final MutableRoaringBitmap[] _bitmaps;
+  private final RoaringBitmapWriter<MutableRoaringBitmap>[] _bitmapWriters;
   private int _nextDocId;
 
+  @SuppressWarnings("unchecked")
   public OnHeapBitmapInvertedIndexCreator(File indexDir, String columnName, int cardinality) {
     _invertedIndexFile = new File(indexDir, columnName + V1Constants.Indexes.BITMAP_INVERTED_INDEX_FILE_EXTENSION);
-    _bitmaps = new MutableRoaringBitmap[cardinality];
+    _bitmapWriters = new RoaringBitmapWriter[cardinality];
     for (int i = 0; i < cardinality; i++) {
-      _bitmaps[i] = new MutableRoaringBitmap();
+      _bitmapWriters[i] = RoaringBitmapWriter.bufferWriter().get();
     }
   }
 
   @Override
   public void add(int dictId) {
-    _bitmaps[dictId].add(_nextDocId++);
+    _bitmapWriters[dictId].add(_nextDocId++);
   }
 
   @Override
   public void add(int[] dictIds, int length) {
     for (int i = 0; i < length; i++) {
-      _bitmaps[dictIds[i]].add(_nextDocId);
+      _bitmapWriters[dictIds[i]].add(_nextDocId);
     }
     _nextDocId++;
   }
 
   @Override
   public void seal()
       throws IOException {
-    try (DataOutputStream out = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)))) {
+    // calculate file size
+    int size = (_bitmapWriters.length + 1) * Integer.BYTES;
+    for (RoaringBitmapWriter<MutableRoaringBitmap> writer : _bitmapWriters) {
+      size += writer.get().serializedSizeInBytes();
+      // Check for int overflow
+      Preconditions.checkState(size > 0, "Inverted index file: %s exceeds 2GB limit", _invertedIndexFile);
+    }
+    ByteBuffer buffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      buffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, size).order(LITTLE_ENDIAN);
       // Write bitmap offsets
-      int bitmapOffset = (_bitmaps.length + 1) * Integer.BYTES;
-      out.writeInt(bitmapOffset);
-      for (MutableRoaringBitmap bitmap : _bitmaps) {
-        bitmapOffset += bitmap.serializedSizeInBytes();
-        // Check for int overflow
-        Preconditions.checkState(bitmapOffset > 0, "Inverted index file: %s exceeds 2GB limit", _invertedIndexFile);
-        out.writeInt(bitmapOffset);
+      int bitmapOffset = (_bitmapWriters.length + 1) * Integer.BYTES;
+      buffer.putInt(Integer.reverseBytes(bitmapOffset));
+      for (RoaringBitmapWriter<MutableRoaringBitmap> writer : _bitmapWriters) {
+        bitmapOffset += writer.getUnderlying().serializedSizeInBytes();

Review comment:
       This API is a little bit overcomplicated. `getUnderlying()` gets the bitmap, without flushing. `flush()` flushes. `get()` flushes and gets the bitmap. I shouldn't have used `getUnderlying()` here because it's confusing, perhaps it should be deprecated

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OnHeapBitmapInvertedIndexCreator.java
##########
@@ -19,68 +19,85 @@
 package org.apache.pinot.core.segment.creator.impl.inv;
 
 import com.google.common.base.Preconditions;
-import java.io.BufferedOutputStream;
-import java.io.DataOutputStream;
+
+
 import java.io.File;
-import java.io.FileOutputStream;
 import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.ByteBuffer;
+import java.nio.channels.FileChannel;
+
 import org.apache.commons.io.FileUtils;
 import org.apache.pinot.core.segment.creator.DictionaryBasedInvertedIndexCreator;
 import org.apache.pinot.core.segment.creator.impl.V1Constants;
+import org.apache.pinot.core.util.CleanerUtil;
+import org.roaringbitmap.RoaringBitmapWriter;
 import org.roaringbitmap.buffer.MutableRoaringBitmap;
 
+import static java.nio.ByteOrder.LITTLE_ENDIAN;
+
 
 /**
  * Implementation of {@link DictionaryBasedInvertedIndexCreator} that uses on-heap memory.
  */
 public final class OnHeapBitmapInvertedIndexCreator implements DictionaryBasedInvertedIndexCreator {
   private final File _invertedIndexFile;
-  private final MutableRoaringBitmap[] _bitmaps;
+  private final RoaringBitmapWriter<MutableRoaringBitmap>[] _bitmapWriters;
   private int _nextDocId;
 
+  @SuppressWarnings("unchecked")
   public OnHeapBitmapInvertedIndexCreator(File indexDir, String columnName, int cardinality) {
     _invertedIndexFile = new File(indexDir, columnName + V1Constants.Indexes.BITMAP_INVERTED_INDEX_FILE_EXTENSION);
-    _bitmaps = new MutableRoaringBitmap[cardinality];
+    _bitmapWriters = new RoaringBitmapWriter[cardinality];
     for (int i = 0; i < cardinality; i++) {
-      _bitmaps[i] = new MutableRoaringBitmap();
+      _bitmapWriters[i] = RoaringBitmapWriter.bufferWriter().get();
     }
   }
 
   @Override
   public void add(int dictId) {
-    _bitmaps[dictId].add(_nextDocId++);
+    _bitmapWriters[dictId].add(_nextDocId++);
   }
 
   @Override
   public void add(int[] dictIds, int length) {
     for (int i = 0; i < length; i++) {
-      _bitmaps[dictIds[i]].add(_nextDocId);
+      _bitmapWriters[dictIds[i]].add(_nextDocId);
     }
     _nextDocId++;
   }
 
   @Override
   public void seal()
       throws IOException {
-    try (DataOutputStream out = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)))) {
+    // calculate file size
+    int size = (_bitmapWriters.length + 1) * Integer.BYTES;
+    for (RoaringBitmapWriter<MutableRoaringBitmap> writer : _bitmapWriters) {
+      size += writer.get().serializedSizeInBytes();
+      // Check for int overflow
+      Preconditions.checkState(size > 0, "Inverted index file: %s exceeds 2GB limit", _invertedIndexFile);
+    }
+    ByteBuffer buffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      buffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, size).order(LITTLE_ENDIAN);

Review comment:
       Yes that's a good idea.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] codecov-io edited a comment on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739096511


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=h1) Report
   > Merging [#6320](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=desc) (0d90376) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **decrease** coverage by `1.12%`.
   > The diff coverage is `45.12%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6320/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #6320      +/-   ##
   ==========================================
   - Coverage   66.44%   65.31%   -1.13%     
   ==========================================
     Files        1075     1283     +208     
     Lines       54773    61937    +7164     
     Branches     8168     8996     +828     
   ==========================================
   + Hits        36396    40457    +4061     
   - Misses      15700    18589    +2889     
   - Partials     2677     2891     +214     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | unittests | `65.31% <45.12%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...e/pinot/broker/api/resources/PinotBrokerDebug.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYXBpL3Jlc291cmNlcy9QaW5vdEJyb2tlckRlYnVnLmphdmE=) | `0.00% <0.00%> (-79.32%)` | :arrow_down: |
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `71.42% <ø> (-28.58%)` | :arrow_down: |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `33.96% <0.00%> (-32.71%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ava/org/apache/pinot/client/AbstractResultSet.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Fic3RyYWN0UmVzdWx0U2V0LmphdmE=) | `66.66% <0.00%> (+9.52%)` | :arrow_up: |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `35.55% <0.00%> (-13.29%)` | :arrow_down: |
   | [...inot/client/JsonAsyncHttpPinotClientTransport.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0pzb25Bc3luY0h0dHBQaW5vdENsaWVudFRyYW5zcG9ydC5qYXZh) | `10.90% <0.00%> (-51.10%)` | :arrow_down: |
   | [...not/common/assignment/InstancePartitionsUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXNzaWdubWVudC9JbnN0YW5jZVBhcnRpdGlvbnNVdGlscy5qYXZh) | `73.80% <ø> (+0.63%)` | :arrow_up: |
   | [.../apache/pinot/common/exception/QueryException.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZXhjZXB0aW9uL1F1ZXJ5RXhjZXB0aW9uLmphdmE=) | `90.27% <ø> (+5.55%)` | :arrow_up: |
   | [...pinot/common/function/AggregationFunctionType.java](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vQWdncmVnYXRpb25GdW5jdGlvblR5cGUuamF2YQ==) | `98.27% <ø> (-1.73%)` | :arrow_down: |
   | ... and [1130 more](https://codecov.io/gh/apache/incubator-pinot/pull/6320/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=footer). Last update [41a7722...0d90376](https://codecov.io/gh/apache/incubator-pinot/pull/6320?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] kishoreg commented on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
kishoreg commented on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-740315985


   Here is the excel sheet with an embedded chart to visualize your perf changes.
   https://docs.google.com/spreadsheets/d/1EhK69Y5UZB-NFrgWBKCT-XXZeJ3oA8WoTDN_B2t37lA/edit?usp=sharing


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r536559080



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OnHeapBitmapInvertedIndexCreator.java
##########
@@ -19,68 +19,85 @@
 package org.apache.pinot.core.segment.creator.impl.inv;
 
 import com.google.common.base.Preconditions;
-import java.io.BufferedOutputStream;
-import java.io.DataOutputStream;
+
+
 import java.io.File;
-import java.io.FileOutputStream;
 import java.io.IOException;
+import java.io.RandomAccessFile;
+import java.nio.ByteBuffer;
+import java.nio.channels.FileChannel;
+
 import org.apache.commons.io.FileUtils;
 import org.apache.pinot.core.segment.creator.DictionaryBasedInvertedIndexCreator;
 import org.apache.pinot.core.segment.creator.impl.V1Constants;
+import org.apache.pinot.core.util.CleanerUtil;
+import org.roaringbitmap.RoaringBitmapWriter;
 import org.roaringbitmap.buffer.MutableRoaringBitmap;
 
+import static java.nio.ByteOrder.LITTLE_ENDIAN;
+
 
 /**
  * Implementation of {@link DictionaryBasedInvertedIndexCreator} that uses on-heap memory.
  */
 public final class OnHeapBitmapInvertedIndexCreator implements DictionaryBasedInvertedIndexCreator {
   private final File _invertedIndexFile;
-  private final MutableRoaringBitmap[] _bitmaps;
+  private final RoaringBitmapWriter<MutableRoaringBitmap>[] _bitmapWriters;
   private int _nextDocId;
 
+  @SuppressWarnings("unchecked")
   public OnHeapBitmapInvertedIndexCreator(File indexDir, String columnName, int cardinality) {
     _invertedIndexFile = new File(indexDir, columnName + V1Constants.Indexes.BITMAP_INVERTED_INDEX_FILE_EXTENSION);
-    _bitmaps = new MutableRoaringBitmap[cardinality];
+    _bitmapWriters = new RoaringBitmapWriter[cardinality];
     for (int i = 0; i < cardinality; i++) {
-      _bitmaps[i] = new MutableRoaringBitmap();
+      _bitmapWriters[i] = RoaringBitmapWriter.bufferWriter().get();
     }
   }
 
   @Override
   public void add(int dictId) {
-    _bitmaps[dictId].add(_nextDocId++);
+    _bitmapWriters[dictId].add(_nextDocId++);
   }
 
   @Override
   public void add(int[] dictIds, int length) {
     for (int i = 0; i < length; i++) {
-      _bitmaps[dictIds[i]].add(_nextDocId);
+      _bitmapWriters[dictIds[i]].add(_nextDocId);
     }
     _nextDocId++;
   }
 
   @Override
   public void seal()
       throws IOException {
-    try (DataOutputStream out = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)))) {
+    // calculate file size
+    int size = (_bitmapWriters.length + 1) * Integer.BYTES;
+    for (RoaringBitmapWriter<MutableRoaringBitmap> writer : _bitmapWriters) {
+      size += writer.get().serializedSizeInBytes();
+      // Check for int overflow
+      Preconditions.checkState(size > 0, "Inverted index file: %s exceeds 2GB limit", _invertedIndexFile);
+    }
+    ByteBuffer buffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {
+      buffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, size).order(LITTLE_ENDIAN);

Review comment:
       I chose little endian because roaring bitmaps are always serialised in little endian byte order anyway, and they make up the bulk of the file. If you serialise to a `DataOutput` it will reverse every word, and if you serialise to a big endian `ByteBuffer`, it will just slice the buffer and change the order. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739086582


   The solution I came up with to make sure the same bytes are produced by both on-heap and off-heap creators was to apply the same changes to `OffHeapBitmapInvertedIndexCreator`, which will now also be run-optimised incrementally.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] richardstartin commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r537828747



##########
File path: pinot-perf/src/main/java/org/apache/pinot/perf/BenchmarkOffheapBitmapInvertedIndexCreator.java
##########
@@ -0,0 +1,105 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.pinot.perf;
+
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.core.segment.creator.impl.inv.OffHeapBitmapInvertedIndexCreator;
+import org.apache.pinot.spi.data.DimensionFieldSpec;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.openjdk.jmh.annotations.*;
+import org.openjdk.jmh.runner.Runner;
+import org.openjdk.jmh.runner.options.ChainedOptionsBuilder;
+import org.openjdk.jmh.runner.options.OptionsBuilder;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+
+@State(Scope.Benchmark)
+public class BenchmarkOffheapBitmapInvertedIndexCreator {

Review comment:
       Done




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull request #6320: More efficient use of RoaringBitmap in OnHeapBitmapInvertedIndexCreator and OffHeapBitmapInvertedIndexCreator

Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #6320:
URL: https://github.com/apache/incubator-pinot/pull/6320#discussion_r536521274



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/inv/OffHeapBitmapInvertedIndexCreator.java
##########
@@ -181,33 +188,49 @@ public void seal()
     }
 
     // Create bitmaps from inverted index buffers and serialize them to file
-    try (DataOutputStream offsetDataStream = new DataOutputStream(
-        new BufferedOutputStream(new FileOutputStream(_invertedIndexFile)));
-        FileOutputStream bitmapFileStream = new FileOutputStream(_invertedIndexFile);
-        DataOutputStream bitmapDataStream = new DataOutputStream(new BufferedOutputStream(bitmapFileStream))) {
-      int bitmapOffset = (_cardinality + 1) * Integer.BYTES;
-      offsetDataStream.writeInt(bitmapOffset);
-      bitmapFileStream.getChannel().position(bitmapOffset);
-
+    ByteBuffer offsetBuffer = null;
+    ByteBuffer bitmapBuffer = null;
+    try (FileChannel channel = new RandomAccessFile(_invertedIndexFile, "rw").getChannel()) {

Review comment:
       Where exactly is the potential benefit coming from?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org