You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Eric Pan (Jira)" <ji...@apache.org> on 2020/02/12 00:36:00 UTC
[jira] [Updated] (SAMZA-2456) Upgrade RocksDB to 6.5.3
[ https://issues.apache.org/jira/browse/SAMZA-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Pan updated SAMZA-2456:
----------------------------
Description:
We are currently on RocksDB 5.7.3 version which is over 2 and 1/2 years old build. As such we should upgrade to the latest RocksDB version to include any fixes or new features. Also there is a segment fault when trying to run RocksDB 5.7.3 with Tensorflow described below:
RocksDB version in Samza is 5.7.3<[https://jarvis.corp.linkedin.com/codesearch/result/?name=dependency-versions.gradle&path=samza-li%2Fsamza%2Fgradle&reponame=samza%2Fsamza-li]>. However, this would cause a segment fault if the dependency order is not properly provisioned. For example, if RockDB library is loaded before Tensorflow library, segment fault is triggered. In contrast, if Tensorflow library is loaded first, then there is no run-time error. Please see the following experiments for more details.
===========experiments==============
public class RocksDBTest{
public static void main(String[] args) throws InterruptedException
{ //System.load("/home/yangzhan/libs/librocksdbjni7259590262328036607.so"); // RocksDB v=5.0.1 => Failed // System.load("/home/yangzhan/libs/librocksdbjni575105474288895339.so"); // RocksDB v=5.7.3 => Failed System.load("/home/yangzhan/libs/librocksdbjni6484631531885492614.so"); // RockdsDB v=5.15.10 => Passed System.load("/home/yangzhan/libs/libtensorflow_framework.so"); }
}
=========================
This issue was reported in a Tensorflow open-source ticket<[https://github.com/tensorflow/tensorflow/issues/22307]> created by @Shanthoosh Venkataraman<[svenkataraman@linkedin.com|mailto:svenkataraman@linkedin.com]>.
was:We are currently on RocksDB 5.7.3 version which is over 2 and 1/2 years old build. As such we should upgrade to the latest RocksDB version to include any fixes or new features.
> Upgrade RocksDB to 6.5.3
> ------------------------
>
> Key: SAMZA-2456
> URL: https://issues.apache.org/jira/browse/SAMZA-2456
> Project: Samza
> Issue Type: Improvement
> Affects Versions: 1.3
> Reporter: Eric Pan
> Priority: Major
>
> We are currently on RocksDB 5.7.3 version which is over 2 and 1/2 years old build. As such we should upgrade to the latest RocksDB version to include any fixes or new features. Also there is a segment fault when trying to run RocksDB 5.7.3 with Tensorflow described below:
> RocksDB version in Samza is 5.7.3<[https://jarvis.corp.linkedin.com/codesearch/result/?name=dependency-versions.gradle&path=samza-li%2Fsamza%2Fgradle&reponame=samza%2Fsamza-li]>. However, this would cause a segment fault if the dependency order is not properly provisioned. For example, if RockDB library is loaded before Tensorflow library, segment fault is triggered. In contrast, if Tensorflow library is loaded first, then there is no run-time error. Please see the following experiments for more details.
> ===========experiments==============
> public class RocksDBTest{
> public static void main(String[] args) throws InterruptedException
> { //System.load("/home/yangzhan/libs/librocksdbjni7259590262328036607.so"); // RocksDB v=5.0.1 => Failed // System.load("/home/yangzhan/libs/librocksdbjni575105474288895339.so"); // RocksDB v=5.7.3 => Failed System.load("/home/yangzhan/libs/librocksdbjni6484631531885492614.so"); // RockdsDB v=5.15.10 => Passed System.load("/home/yangzhan/libs/libtensorflow_framework.so"); }
> }
> =========================
> This issue was reported in a Tensorflow open-source ticket<[https://github.com/tensorflow/tensorflow/issues/22307]> created by @Shanthoosh Venkataraman<[svenkataraman@linkedin.com|mailto:svenkataraman@linkedin.com]>.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)