[jira] [Updated] (SAMZA-2456) Upgrade RocksDB to 6.5.3

["Bharath Kumarasubramanian (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bharath Kumarasubramanian updated SAMZA-2456:
---------------------------------------------
    Fix Version/s: 1.5

> Upgrade RocksDB to 6.5.3
> ------------------------
>
>                 Key: SAMZA-2456
>                 URL: https://issues.apache.org/jira/browse/SAMZA-2456
>             Project: Samza
>          Issue Type: Improvement
>    Affects Versions: 1.3
>            Reporter: Eric Pan
>            Assignee: Eric Pan
>            Priority: Major
>             Fix For: 1.5
>
>         Attachments: RocksDB performance.xlsx
>
>          Time Spent: 50m
>  Remaining Estimate: 0h
>
> We are currently on RocksDB 5.7.3 version which is over 2 and 1/2 years old build. As such we should upgrade to the latest RocksDB version to include any fixes or new features. Also there is a segment fault when trying to run RocksDB 5.7.3 with Tensorflow described below:
> RocksDB version in Samza is 5.7.3<[https://jarvis.corp.linkedin.com/codesearch/result/?name=dependency-versions.gradle&path=samza-li%2Fsamza%2Fgradle&reponame=samza%2Fsamza-li]>. However, this would cause a segment fault if the dependency order is not properly provisioned. For example, if RockDB library is loaded before Tensorflow library, segment fault is triggered. In contrast, if Tensorflow library is loaded first, then there is no run-time error. Please see the following experiments for more details.
> ===========experiments==============
> public class RocksDBTest{
> public static void main(String[] args) throws InterruptedException
> { //System.load("/home/yangzhan/libs/librocksdbjni7259590262328036607.so"); // RocksDB v=5.0.1 => Failed // System.load("/home/yangzhan/libs/librocksdbjni575105474288895339.so"); // RocksDB v=5.7.3 => Failed System.load("/home/yangzhan/libs/librocksdbjni6484631531885492614.so"); // RockdsDB v=5.15.10 => Passed System.load("/home/yangzhan/libs/libtensorflow_framework.so"); }
> }
> =========================
> This issue was reported in a Tensorflow open-source ticket<[https://github.com/tensorflow/tensorflow/issues/22307]> created by @Shanthoosh Venkataraman<[svenkataraman@linkedin.com|mailto:svenkataraman@linkedin.com]>.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)