[jira] [Comment Edited] (KAFKA-9148) Consider forking RocksDB for Streams

["Sophie Blee-Goldman (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-9148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16968119#comment-16968119 ] 

Sophie Blee-Goldman edited comment on KAFKA-9148 at 11/6/19 6:37 AM:
---------------------------------------------------------------------

{quote}seems like this is the kind of problem that the RocksDB team _would_ care about
{quote}
I agree, which is why I found it odd that it's been two months since the regression issue was first brought to their notice, and the most recent comment on the issue is actually just the issue's author still trying to get attention on it from the rocksdb folks with three weeks of silence since :) 

(To be fair that doesn't mean they definitely aren't investigating it currently, but I'll just say that response level is representative of other major issues in the past – and any Java-specific performance problems seem to almost always get dropped on the floor)

I should clarify that I'm not necessarily advocating strongly for forking it _right now_ – but if we find ourselves coming back to this list and adding more things, or in a place where the lack of some feature or Java API is holding Streams back, then we should consider it in a serious way.


was (Author: ableegoldman):
> seems like this is the kind of problem that the RocksDB team _would_ care about
I agree, which is why I found it odd that it's been two months since the regression issue was first brought to their notice, and the most recent comment on the issue is actually just the issue's author still trying to get attention on it from the rocksdb folks with three weeks of silence since :) 

(To be fair that doesn't mean they definitely aren't investigating it currently, but I'll just say that response level is representative of other major issues in the past – and any Java-specific performance problems seem to almost always get dropped on the floor)

I should clarify that I'm not necessarily advocating strongly for forking it _right now_ – but if we find ourselves coming back to this list and adding more things, or in a place where the lack of some feature or Java API is holding Streams back, then we should consider it in a serious way.

> Consider forking RocksDB for Streams 
> -------------------------------------
>
>                 Key: KAFKA-9148
>                 URL: https://issues.apache.org/jira/browse/KAFKA-9148
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Sophie Blee-Goldman
>            Priority: Major
>
> We recently upgraded our RocksDB dependency to 5.18 for its memory-management abilities (namely the WriteBufferManager, see KAFKA-8215). Unfortunately, someone from Flink recently discovered a ~8% [performance regression|https://github.com/facebook/rocksdb/issues/5774] that exists in all versions 5.18+ (up through the current newest version, 6.2.2). Flink was able to react to this by downgrading to 5.17 and [picking the WriteBufferManage|https://github.com/dataArtisans/frocksdb/pull/4]r to their fork (fRocksDB).
> Due to this and other reasons enumerated below, we should consider also forking our own RocksDB for Streams.
> Pros:
>  * We can avoid passing sudden breaking changes on to our users, such removal of methods with no deprecation period (see discussion on KAFKA-8897)
>  * We can pick whichever version has the best performance for our needs, and pick over any new features, metrics, etc that we need to use rather than being forced to upgrade (and breaking user code, introducing regression, etc)
>  * The Java API seems to be a very low priority to the rocksdb folks.
>  ** They leave out critical functionality, features, and configuration options that have been in the c++ API for a very long time
>  ** Those that do make it over often have random gaps in the API such as setters but no getters (see [rocksdb PR #5186|https://github.com/facebook/rocksdb/pull/5186])
>  ** Others are poorly designed and require too many trips across the JNI, making otherwise incredibly useful features prohibitively expensive.
>  *** [Custom comparator|#issuecomment-83145980]]: a custom comparator could significantly improve the performance of session windows. This is trivial to do but given the high performance cost of crossing the jni, it is currently only practical to use a c++ comparator
>  *** [Prefix Seek|https://github.com/facebook/rocksdb/issues/6004]: not currently used by Streams but a commonly requested feature, and may also allow improved range queries
>  ** Even when an external contributor develops a solution for poorly performing Java functionality and helpfully tries to contribute their patch back to rocksdb, it gets ignored by the rocksdb people ([rocksdb PR #2283|https://github.com/facebook/rocksdb/pull/2283])
> Cons:
>  * more work
> Given that we rarely upgrade the Rocks dependency, use only some fraction of its features, and would need or want to make only minimal changes ourselves, it seems like we could actually get away with very little extra work by forking rocksdb. Note that as of this writing the frocksdb repo has only needed to open 5 PRs on top of the actual rocksdb (two of them trivial).
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)