You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kafka.apache.org by gu...@apache.org on 2016/06/23 21:36:42 UTC

kafka git commit: MINOR: Pass absolute directory path to RocksDB.open

Repository: kafka
Updated Branches:
  refs/heads/trunk 8ec4e4b7a -> 41a54358b


MINOR: Pass absolute directory path to RocksDB.open

The method `RocksDB.open` assumes an absolute file path. If a relative path is configured, it leads to an exception like the following:
```
org.apache.kafka.streams.errors.ProcessorStateException: Error opening store CustomerIdToUserIdLookup at location ./tmp/rocksdb/CustomerIdToUserIdLookup
	at org.rocksdb.RocksDB.open(Native Method)
	at org.rocksdb.RocksDB.open(RocksDB.java:183)
	at org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:214)
	at org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:165)
	at org.apache.kafka.streams.state.internals.RocksDBStore.init(RocksDBStore.java:170)
	at org.apache.kafka.streams.state.internals.MeteredKeyValueStore.init(MeteredKeyValueStore.java:85)
	at org.apache.kafka.test.KStreamTestDriver.<init>(KStreamTestDriver.java:64)
	at org.apache.kafka.test.KStreamTestDriver.<init>(KStreamTestDriver.java:50)
	at com.simple.estuary.transform.streaming.CartesianTransactionEnrichmentJobTest.testBuilder(CartesianTransactionEnrichmentJobTest.java:41)
```

Is there any risk to always fetching the absolute path as proposed here?

Let me know if you think this requires a JIRA issue or a unit test. I started working on a unit test, but don't know of a great solution for writing out a file to a relative directory.

This contribution is my original work and I license the work to the project under the project's open source license.

Author: Jeff Klukas <je...@klukas.net>

Reviewers: Guozhang Wang <wa...@gmail.com>

Closes #1481 from jklukas/rocksdb-abspath


Project: http://git-wip-us.apache.org/repos/asf/kafka/repo
Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/41a54358
Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/41a54358
Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/41a54358

Branch: refs/heads/trunk
Commit: 41a54358bececc42ba680267281ba01b3f33f77f
Parents: 8ec4e4b
Author: Jeff Klukas <je...@klukas.net>
Authored: Thu Jun 23 14:36:38 2016 -0700
Committer: Guozhang Wang <wa...@gmail.com>
Committed: Thu Jun 23 14:36:38 2016 -0700

----------------------------------------------------------------------
 .../org/apache/kafka/streams/state/internals/RocksDBStore.java     | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kafka/blob/41a54358/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java
----------------------------------------------------------------------
diff --git a/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java b/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java
index d9670b3..4109ebf 100644
--- a/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java
+++ b/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java
@@ -212,7 +212,7 @@ public class RocksDBStore<K, V> implements KeyValueStore<K, V> {
         try {
             if (ttl == TTL_NOT_USED) {
                 dir.getParentFile().mkdirs();
-                return RocksDB.open(options, dir.toString());
+                return RocksDB.open(options, dir.getAbsolutePath());
             } else {
                 throw new UnsupportedOperationException("Change log is not supported for store " + this.name + " since it is TTL based.");
                 // TODO: support TTL with change log?