You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-commits@jackrabbit.apache.org by ch...@apache.org on 2014/04/02 14:25:44 UTC
svn commit: r1583994 - in /jackrabbit/oak/trunk/oak-core/src:
main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/OakFileDataStore.java
test/java/org/apache/jackrabbit/oak/plugins/blob/datastore/OakFileDataStoreTest.java
Author: chetanm
Date: Wed Apr 2 12:25:44 2014
New Revision: 1583994
URL: http://svn.apache.org/r1583994
Log:
OAK-1666 - FileDataStore inUse map causes contention in concurrent env
Temporary workaround which uses a NoOpMap for inUseMap untill fix for JCR-3764 is ready
Modified:
jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/OakFileDataStore.java
jackrabbit/oak/trunk/oak-core/src/test/java/org/apache/jackrabbit/oak/plugins/blob/datastore/OakFileDataStoreTest.java
Modified: jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/OakFileDataStore.java
URL: http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/OakFileDataStore.java?rev=1583994&r1=1583993&r2=1583994&view=diff
==============================================================================
--- jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/OakFileDataStore.java (original)
+++ jackrabbit/oak/trunk/oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/OakFileDataStore.java Wed Apr 2 12:25:44 2014
@@ -20,7 +20,11 @@
package org.apache.jackrabbit.oak.plugins.blob.datastore;
import java.io.File;
+import java.lang.ref.WeakReference;
+import java.util.AbstractMap;
+import java.util.Collections;
import java.util.Iterator;
+import java.util.Set;
import com.google.common.base.Charsets;
import com.google.common.base.Function;
@@ -38,6 +42,13 @@ import org.apache.jackrabbit.core.data.F
public class OakFileDataStore extends FileDataStore {
private byte[] referenceKey;
+ public OakFileDataStore() {
+ //TODO FIXME Temporary workaround for OAK-1666. Override the default
+ //synchronized map with a Noop. This should be removed when fix
+ //for JCR-3764 is part of release.
+ inUse = new NoOpMap<DataIdentifier, WeakReference<DataIdentifier>>();
+ }
+
@Override
public Iterator<DataIdentifier> getAllIdentifiers() {
return Files.fileTreeTraverser().postOrderTraversal(new File(getPath()))
@@ -89,4 +100,21 @@ public class OakFileDataStore extends Fi
public void setReferenceKey(byte[] referenceKey) {
this.referenceKey = referenceKey;
}
+
+ /**
+ * Noop map which eats up all the put call
+ */
+ static class NoOpMap<K,V> extends AbstractMap<K,V> {
+
+ @Override
+ public V put(K key, V value) {
+ //Eat the put call
+ return null;
+ }
+
+ @Override
+ public Set<Entry<K, V>> entrySet() {
+ return Collections.emptySet();
+ }
+ }
}
Modified: jackrabbit/oak/trunk/oak-core/src/test/java/org/apache/jackrabbit/oak/plugins/blob/datastore/OakFileDataStoreTest.java
URL: http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-core/src/test/java/org/apache/jackrabbit/oak/plugins/blob/datastore/OakFileDataStoreTest.java?rev=1583994&r1=1583993&r2=1583994&view=diff
==============================================================================
--- jackrabbit/oak/trunk/oak-core/src/test/java/org/apache/jackrabbit/oak/plugins/blob/datastore/OakFileDataStoreTest.java (original)
+++ jackrabbit/oak/trunk/oak-core/src/test/java/org/apache/jackrabbit/oak/plugins/blob/datastore/OakFileDataStoreTest.java Wed Apr 2 12:25:44 2014
@@ -21,6 +21,7 @@ package org.apache.jackrabbit.oak.plugin
import java.io.File;
import java.util.Iterator;
+import java.util.Map;
import java.util.Set;
import javax.annotation.Nullable;
@@ -34,6 +35,7 @@ import org.apache.jackrabbit.core.data.F
import org.junit.Test;
import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
public class OakFileDataStoreTest {
@@ -60,4 +62,12 @@ public class OakFileDataStoreTest {
assertEquals(expectedNames, fileNames);
FileUtils.cleanDirectory(testDir);
}
+
+ @Test
+ public void testNoOpMap() throws Exception{
+ Map<String, String> noop = new OakFileDataStore.NoOpMap<String, String>();
+ noop.put("a","b");
+ noop.remove("foo");
+ assertTrue(noop.isEmpty());
+ }
}
Re: svn commit: r1583994 - in /jackrabbit/oak/trunk/oak-core/src:
main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/OakFileDataStore.java
test/java/org/apache/jackrabbit/oak/plugins/blob/datastore/OakFileDataStoreTest.java
Posted by Chetan Mehrotra <ch...@gmail.com>.
On Wed, Apr 2, 2014 at 6:30 PM, Jukka Zitting <ju...@gmail.com> wrote:
> The inUse map is in FileDataStore for a reason.
Ack. From what I have understood from Blob GC logic in Oak is that it
relies on blob last modified value to distinguish between active used
blobs. So for performing GC only those blob would be considered whose
lastModified value is say 1 day. Only these blobs would be candidate
for deletion. This ensures that any blob created in transient space
are not considered for GC.
So current logic does make an assumption that 1 day is sufficient time
and hence not foolproof. However the current impl of inUse would
probably only work for a single node system and would fail for shared
DataStore scenario as its an in memory state and its hard to determine
inUse state for whole cluster. For supporting such case we would have
to rely on lastModified time interval to distinguish between active
used blobs
regards
Chetan
Chetan Mehrotra
Re: svn commit: r1583994 - in /jackrabbit/oak/trunk/oak-core/src:
main/java/org/apache/jackrabbit/oak/plugins/blob/datastore/OakFileDataStore.java
test/java/org/apache/jackrabbit/oak/plugins/blob/datastore/OakFileDataStoreTest.java
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Wed, Apr 2, 2014 at 8:25 AM, <ch...@apache.org> wrote:
> + //TODO FIXME Temporary workaround for OAK-1666. Override the default
> + //synchronized map with a Noop. This should be removed when fix
> + //for JCR-3764 is part of release.
> + inUse = new NoOpMap<DataIdentifier, WeakReference<DataIdentifier>>();
This breaks the following client:
Binary binary = session.getValueFactory().createBinary(...);
// wait over a garbage collection cycle
session.getRootNode().setProperty("foo", binary);
session.save();
Note that the wait in between could be anything, in the worst case
just bad timing or more likely some other long-running statements like
waiting for user input or creating other large binaries.
The inUse map is in FileDataStore for a reason. If it's causing
performance issues, the right solution is *not* to just disable it but
rather to figure out how the same functionality could be achieved more
efficiently.
BR,
Jukka Zitting