[jira] [Updated] (SAMZA-984) Upgrade to RocksDB to 4.8.0

["Prateek Maheshwari (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Prateek Maheshwari updated SAMZA-984:
-------------------------------------
    Fix Version/s: 0.13.0

> Upgrade to RocksDB to 4.8.0
> ---------------------------
>
>                 Key: SAMZA-984
>                 URL: https://issues.apache.org/jira/browse/SAMZA-984
>             Project: Samza
>          Issue Type: Improvement
>            Reporter: Chris Pettitt
>            Assignee: Prateek Maheshwari
>             Fix For: 0.13.0
>
>
> We're currently on a ~9 month old build of RocksDB.
> I was experimenting with setting limits on the amount of logging done from RocksDB with the keys:
> - rocksdb.max.log.file.size.bytes
> - rocksdb.keep.log.file.num
> With these keys enabled I consistently get the java process to SIGABRT within about a minute with the following backtrace:
> {code}
> Thread 3 Crashed:
> 0   libsystem_kernel.dylib        	0x00007fff849cf002 __pthread_kill + 10
> 1   libsystem_pthread.dylib       	0x00007fff96c365c5 pthread_kill + 90
> 2   libsystem_c.dylib             	0x00007fff862446e7 abort + 129
> 3   libc++abi.dylib               	0x00007fff911dff81 abort_message + 257
> 4   libc++abi.dylib               	0x00007fff91205a2f default_terminate_handler() + 243
> 5   libobjc.A.dylib               	0x00007fff957d36c3 _objc_terminate() + 124
> 6   libc++abi.dylib               	0x00007fff9120319e std::__terminate(void (*)()) + 8
> 7   libc++abi.dylib               	0x00007fff91202c12 __cxa_throw + 121
> 8   libc++.1.dylib                	0x00007fff94e79781 std::__1::__vector_base_common<true>::__throw_out_of_range() const + 71
> 9   librocksdbjni213472405106490933..jnilib	0x000000012c51503f rocksdb::DBImpl::PurgeObsoleteFiles(rocksdb::JobContext const&) + 3535
> 10  librocksdbjni213472405106490933..jnilib	0x000000012c516403 rocksdb::DBImpl::DeleteObsoleteFiles() + 531
> {code}
> Upgrading to a more recent version of RocksDB (i.e. 4.8.0) resolves this issue. The upgrade worked with a drop in replacement.
> To properly do upgrade we'd need:
> - Integration testing (do we have something for this?)
> - Perf testing



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)