You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by "Yixue (Andrew) Zhu" <yx...@gmail.com> on 2012/12/05 03:06:54 UTC
Review Request: SkipList Ledger Store

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8350/
-----------------------------------------------------------

Review request for bookkeeper, Ivan Kelly, Sijie Guo, and Aniruddha Laud.


Description
-------

Uses skip list to sort entries before adding them to entry log file, to improve ledger read performance. Memory arena is used to allocate skip list entries, to avoid GC impact.
A single-threaded scheduler is used to flush skip list to buffered entry log file channel, once configured data size limit is reached. Sync thread is notified as well to flush file buffers.
Compaction uses Skip list, to  merge entries together as well as remove duplicate entries.
This change also fix an existing issue of old entry logs being removed w/o forcing new entry logs flushed, which is more obvious with SkipList store.


This addresses bug BOOKKEEPER-432.
    https://issues.apache.org/jira/browse/BOOKKEEPER-432


Diffs
-----

  bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java 929be51 
  bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/CacheCallback.java PRE-CREATION 
  bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryKey.java PRE-CREATION 
  bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryKeyValue.java PRE-CREATION 
  bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryMemTable.java PRE-CREATION 
  bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java cecb74a 
  bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/InterleavedLedgerStorage.java c3f5149 
  bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCacheImpl.java c8d2b21 
  bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerStorage.java 14f1338 
  bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/SkipListArena.java PRE-CREATION 
  bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/SkipListFlusher.java PRE-CREATION 
  bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/SkipListLedgerStorage.java PRE-CREATION 
  bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingAddOp.java acef474 
  bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ServerConfiguration.java d7e5b2c 
  bookkeeper-server/src/main/java/org/apache/bookkeeper/proto/BookieServer.java 5d7082a 
  bookkeeper-server/src/main/resources/findbugsExclude.xml 8404e3f 
  bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CompactionTest.java 720f07b 
  bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/EntryLogTest.java 2d9fa78 
  bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/LedgerCacheTest.java a24b1e2 
  bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/TestEntryMemTable.java PRE-CREATION 
  bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/TestSkipListArena.java PRE-CREATION 
  bookkeeper-server/src/test/java/org/apache/bookkeeper/test/LedgerDeleteTest.java 984c2f8 
  bookkeeper-server/src/test/java/org/apache/bookkeeper/test/ReadOnlyBookieTest.java a375f09 

Diff: https://reviews.apache.org/r/8350/diff/


Testing
-------

Unit tests (new + existing)


Thanks,

Yixue (Andrew) Zhu