[GitHub] [incubator-druid] sanastas commented on issue #5698: Oak: New Concurrent Key-Value Map

[GitBox <gi...@apache.org>]

sanastas commented on issue #5698: Oak: New Concurrent Key-Value Map 
URL: https://github.com/apache/incubator-druid/issues/5698#issuecomment-493201879
 
 
   Hi Everyone!
   
   We are back! I have just created a new PR #7676 in order to continue discussion about the new type of Incremental Index based on Oak concurrent navigable map.
   
   For those yet not familiar with Oak - it is an off-heap (direct-allocation) concurrent navigable (ordered) KV-map. More details about Oak can be found in the previous comments in this issue and in PR #7676 introduction.
   
   For those who are familiar with Oak, this is what have been done so far (since this project went on a pause):
   1. The Druid's integration was reconsidered and renewed to impose less changes in Druid. (The old refactoring schema doesn't imply any longer.) The Oak integration code is rebased on the up-to-date Druid's code.
   2. Oak is now E2E connected and Druid cluster based on OakIncrementalIndex can be created. Details how to activate in PR #7676.
   
   The Oak isolated benchmarks can be found here (in order to see the potential in using Oak vs ConcurrentSkipListMap): [OAK PERFORMANCE.pdf](https://github.com/apache/incubator-druid/files/3188690/OAK.PERFORMANCE.pdf)
   We will soon publish OakIncrementalIndex JMH benchmarks available in Druid. However, those benchmarks are single threaded and we might need some modifications in order to show Oak benefits well.
   
   Please take a look on PR #7676 and raise your questions here or there. Clearly we have more work to do, this is just a start! :) We will join the Druid developer sync meeting next Tuesday, May 21, to discuss the Oak possibility further.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org