[jira] [Commented] (SAMZA-543) Disable WAL in RocksDB KV store

["Navina Ramesh (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMZA-543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359553#comment-14359553 ] 

Navina Ramesh commented on SAMZA-543:
-------------------------------------

Having read the discussion above, I have a couple of questions:
* should disableDataSync be set to true ? I couldn't figure out the default value used in the JNI bindings.
* Do we want to run any performance tests before we commit this change? 

We can examine more improvements on RocksDB, such as changing the compaction style, batch write etc. My suggestion is to do this one change at a time and measure the performance improvement. Please comment on this as well!

> Disable WAL in RocksDB KV store
> -------------------------------
>
>                 Key: SAMZA-543
>                 URL: https://issues.apache.org/jira/browse/SAMZA-543
>             Project: Samza
>          Issue Type: Bug
>          Components: kv
>    Affects Versions: 0.9.0
>            Reporter: Chris Riccomini
>            Assignee: Navina Ramesh
>             Fix For: 0.9.0
>
>         Attachments: SAMZA-543-0.patch
>
>
> RocksDB uses a write-ahead log by default. This is unnecessary in Samza, since we have full durability from a state store's changelog topic. We should [disable the WAL|https://github.com/facebook/rocksdb/wiki/Basic-Operations#asynchronous-writes] in the RocksDB KV store.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)