You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@druid.apache.org by Anastasia Braginsky <an...@verizonmedia.com.INVALID> on 2019/05/16 20:04:22 UTC

Oak ConcurrentNavigableMap for Druid

Hi Guys,
I would like to draw your attention to reactivation of the integration of Oak (ConcurrentNavigableMap) into Druid's incremental index.
OakMap is an *off-heap* (direct-allocation) concurrent navigable (ordered) KV-map. OakMap is part of the Oak open source project (https://github.com/yahoo/Oak/). 


Using OakMap opens new opportunities for real-time indexing in Druid, including: 
   
   -    
Faster data ingestion rates. 

   -    
Building fewer, larger segments, thereby reducing the query latencies and  merging/compaction overhead. 

   -    
Real-time queries of segments currently being built. 

   -    
Much faster, zero-overhead reverse scans. 


For more details please take a look on Issue#5698 "Oak: New Concurrent KV-Map" and on PR#7676 "Add OakIncrementalIndex to Druid"
 We will join the Druid developer sync meeting next Tuesday, May 21, to discuss the Oak possibility further. 
Thanks,Anastasia