You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-commits@jackrabbit.apache.org by ad...@apache.org on 2023/07/27 13:43:36 UTC
[jackrabbit-oak] branch trunk updated (c18d0c5145 -> 1643f1ba56)
This is an automated email from the ASF dual-hosted git repository.
adulceanu pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/jackrabbit-oak.git
from c18d0c5145 OAK-10369: oak-segment-azure/oak-blob-cloud-azure leak Guava dependency - adjust import range (#1040)
add 6545b7912c OAK-9922 Parallel Compaction
add 2abd1e6895 Make SegmentBufferWriterPool implementation a configurable option
add 38b6c78172 Address review comments
add 00967073c3 Address further comments
new ed8494b5e3 OAK-9922 - Parallel Compaction Solved conflicts due to guava changes
new c7aa4f0f7b OAK-9922 - Parallel Compaction Merge branch 'lweitzendorf-breadth-first-scan' into trunk
new 1643f1ba56 Merge branch 'trunk' of github.com:apache/jackrabbit-oak into trunk
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../resources/img/railroad/div-d.png => .DS_Store | Bin 8289 -> 6148 bytes
.../plugins/index/counter/ApproximateCounter.java | 9 +-
.../plugins/index/counter/NodeCounterEditor.java | 1 +
.../oak/plugins/index/counter/jmx/NodeCounter.java | 2 +-
.../index/property/PropertyIndexInfoProvider.java | 2 +-
.../strategy/ContentMirrorStoreStrategy.java | 2 +-
.../strategy/UniqueEntryStoreStrategy.java | 2 +-
.../strategy/ContentMirrorStoreStrategyTest.java | 2 +-
.../site/markdown/nodestore/segment/overview.md | 27 +-
.../apache/jackrabbit/oak/run/CompactCommand.java | 54 +--
.../oak/segment/aws/tool/AwsCompact.java | 34 +-
.../oak/segment/aws/tool/AwsToolUtils.java | 19 +-
.../oak/segment/azure/tool/AzureCompact.java | 21 +-
.../oak/segment/azure/tool/ToolUtils.java | 26 +-
.../oak/segment/CheckpointCompactor.java | 32 +-
.../jackrabbit/oak/segment/ClassicCompactor.java | 21 +-
.../jackrabbit/oak/segment/CompactorUtils.java | 21 +-
.../oak/segment/DefaultSegmentWriterBuilder.java | 90 +++--
.../jackrabbit/oak/segment/ParallelCompactor.java | 370 +++++++++++++++++++
.../apache/jackrabbit/oak/segment/RecordCache.java | 119 +++----
.../oak/segment/SegmentBufferWriterPool.java | 390 ++++++++++++++-------
.../oak/segment/compaction/SegmentGCOptions.java | 47 ++-
.../segment/file/AbstractCompactionStrategy.java | 31 +-
.../jackrabbit/oak/segment/file/FileStore.java | 7 +-
.../oak/segment/file/FileStoreBuilder.java | 14 +-
.../oak/segment/file/GCNodeWriteMonitor.java | 66 ++--
.../jackrabbit/oak/segment/file/PriorityCache.java | 275 +++++++++------
.../jackrabbit/oak/segment/memory/MemoryStore.java | 14 +-
.../jackrabbit/oak/segment/tool/Compact.java | 22 +-
...java => AbstractCompactorExternalBlobTest.java} | 29 +-
...mpactorTest.java => AbstractCompactorTest.java} | 27 +-
.../CheckpointCompactorExternalBlobTest.java | 135 +------
.../oak/segment/CheckpointCompactorTest.java | 111 +-----
.../oak/segment/CompactionAndCleanupIT.java | 13 +-
...actorTestUtils.java => CompactorTestUtils.java} | 20 +-
.../jackrabbit/oak/segment/NodeRecordTest.java | 4 +-
.../segment/ParallelCompactorExternalBlobTest.java | 63 ++++
.../oak/segment/ParallelCompactorTest.java | 63 ++++
.../oak/segment/RecordCacheStatsTest.java | 20 +-
.../jackrabbit/oak/segment/RecordCacheTest.java | 54 ++-
.../oak/segment/SegmentBufferWriterPoolTest.java | 67 ++--
.../segment/file/ConcurrentPriorityCacheTest.java | 236 +++++++++++++
.../oak/segment/file/PriorityCacheTest.java | 27 +-
.../div-d.png => oak-shaded-guava/.DS_Store | Bin 8289 -> 6148 bytes
.../oak/plugins/index}/ApproximateCounter.java | 2 +-
.../oak/plugins/index}/ApproximateCounterTest.java | 9 +-
46 files changed, 1752 insertions(+), 848 deletions(-)
copy oak-doc/src/site/resources/img/railroad/div-d.png => .DS_Store (64%)
copy oak-upgrade/src/main/java/org/apache/jackrabbit/oak/segment/CheckpointAccessor.java => oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/CompactorUtils.java (68%)
create mode 100644 oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/ParallelCompactor.java
copy oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/{CheckpointCompactorExternalBlobTest.java => AbstractCompactorExternalBlobTest.java} (83%)
copy oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/{CheckpointCompactorTest.java => AbstractCompactorTest.java} (83%)
rename oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/{CheckpointCompactorTestUtils.java => CompactorTestUtils.java} (88%)
create mode 100644 oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/ParallelCompactorExternalBlobTest.java
create mode 100644 oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/ParallelCompactorTest.java
create mode 100644 oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/file/ConcurrentPriorityCacheTest.java
copy oak-doc/src/site/resources/img/railroad/div-d.png => oak-shaded-guava/.DS_Store (64%)
copy {oak-core/src/main/java/org/apache/jackrabbit/oak/plugins/index/counter => oak-store-spi/src/main/java/org/apache/jackrabbit/oak/plugins/index}/ApproximateCounter.java (99%)
rename {oak-core/src/test/java/org/apache/jackrabbit/oak/plugins/index/counter => oak-store-spi/src/test/java/org/apache/jackrabbit/oak/plugins/index}/ApproximateCounterTest.java (96%)
[jackrabbit-oak] 01/03: OAK-9922 - Parallel Compaction Solved conflicts due to guava changes
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adulceanu pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/jackrabbit-oak.git
commit ed8494b5e3a90f0021f3cd6e3fc7112ec9ec6653
Merge: 951d69e57a 00967073c3
Author: dulceanu <an...@gmail.com>
AuthorDate: Thu Jul 27 16:40:00 2023 +0300
OAK-9922 - Parallel Compaction
Solved conflicts due to guava changes
.DS_Store | Bin 0 -> 6148 bytes
.../plugins/index/counter/ApproximateCounter.java | 9 +-
.../plugins/index/counter/NodeCounterEditor.java | 1 +
.../oak/plugins/index/counter/jmx/NodeCounter.java | 2 +-
.../index/property/PropertyIndexInfoProvider.java | 2 +-
.../strategy/ContentMirrorStoreStrategy.java | 2 +-
.../strategy/UniqueEntryStoreStrategy.java | 2 +-
.../strategy/ContentMirrorStoreStrategyTest.java | 2 +-
.../site/markdown/nodestore/segment/overview.md | 27 +-
.../apache/jackrabbit/oak/run/CompactCommand.java | 54 +--
.../oak/segment/aws/tool/AwsCompact.java | 34 +-
.../oak/segment/aws/tool/AwsToolUtils.java | 19 +-
.../oak/segment/azure/tool/AzureCompact.java | 21 +-
.../oak/segment/azure/tool/ToolUtils.java | 26 +-
.../oak/segment/CheckpointCompactor.java | 32 +-
.../jackrabbit/oak/segment/ClassicCompactor.java | 21 +-
.../jackrabbit/oak/segment/CompactorUtils.java | 35 ++
.../oak/segment/DefaultSegmentWriterBuilder.java | 90 +++--
.../jackrabbit/oak/segment/ParallelCompactor.java | 370 +++++++++++++++++++
.../apache/jackrabbit/oak/segment/RecordCache.java | 119 +++----
.../oak/segment/SegmentBufferWriterPool.java | 390 ++++++++++++++-------
.../oak/segment/compaction/SegmentGCOptions.java | 47 ++-
.../segment/file/AbstractCompactionStrategy.java | 31 +-
.../jackrabbit/oak/segment/file/FileStore.java | 7 +-
.../oak/segment/file/FileStoreBuilder.java | 14 +-
.../oak/segment/file/GCNodeWriteMonitor.java | 66 ++--
.../jackrabbit/oak/segment/file/PriorityCache.java | 275 +++++++++------
.../jackrabbit/oak/segment/memory/MemoryStore.java | 14 +-
.../jackrabbit/oak/segment/tool/Compact.java | 22 +-
...java => AbstractCompactorExternalBlobTest.java} | 29 +-
...mpactorTest.java => AbstractCompactorTest.java} | 27 +-
.../CheckpointCompactorExternalBlobTest.java | 135 +------
.../oak/segment/CheckpointCompactorTest.java | 111 +-----
.../oak/segment/CompactionAndCleanupIT.java | 13 +-
...actorTestUtils.java => CompactorTestUtils.java} | 20 +-
.../jackrabbit/oak/segment/NodeRecordTest.java | 4 +-
.../segment/ParallelCompactorExternalBlobTest.java | 63 ++++
.../oak/segment/ParallelCompactorTest.java | 63 ++++
.../oak/segment/RecordCacheStatsTest.java | 20 +-
.../jackrabbit/oak/segment/RecordCacheTest.java | 54 ++-
.../oak/segment/SegmentBufferWriterPoolTest.java | 67 ++--
.../segment/file/ConcurrentPriorityCacheTest.java | 236 +++++++++++++
.../oak/segment/file/PriorityCacheTest.java | 27 +-
oak-shaded-guava/.DS_Store | Bin 0 -> 6148 bytes
.../oak/plugins/index}/ApproximateCounter.java | 2 +-
.../oak/plugins/index}/ApproximateCounterTest.java | 9 +-
46 files changed, 1774 insertions(+), 840 deletions(-)
diff --cc .DS_Store
index 0000000000,0000000000..c9f58500aa
new file mode 100644
Binary files differ
diff --cc oak-run/src/main/java/org/apache/jackrabbit/oak/run/CompactCommand.java
index 212c7a0435,315ac68240..157c863b26
--- a/oak-run/src/main/java/org/apache/jackrabbit/oak/run/CompactCommand.java
+++ b/oak-run/src/main/java/org/apache/jackrabbit/oak/run/CompactCommand.java
@@@ -18,17 -18,15 +18,16 @@@
package org.apache.jackrabbit.oak.run;
import java.io.File;
-import com.google.common.base.StandardSystemProperty;
+
- import org.apache.jackrabbit.guava.common.base.StandardSystemProperty;
import joptsimple.OptionParser;
import joptsimple.OptionSet;
import joptsimple.OptionSpec;
import org.apache.jackrabbit.oak.run.commons.Command;
+ import org.apache.jackrabbit.oak.segment.aws.tool.AwsCompact;
import org.apache.jackrabbit.oak.segment.azure.tool.AzureCompact;
- import org.apache.jackrabbit.oak.segment.azure.tool.AzureCompact.Builder;
import org.apache.jackrabbit.oak.segment.compaction.SegmentGCOptions.CompactorType;
- import org.apache.jackrabbit.oak.segment.aws.tool.AwsCompact;
import org.apache.jackrabbit.oak.segment.tool.Compact;
++import org.apache.jackrabbit.guava.common.base.StandardSystemProperty;
class CompactCommand implements Command {
diff --cc oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/DefaultSegmentWriterBuilder.java
index 94ae636a4b,c461084d29..8e222b9693
--- a/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/DefaultSegmentWriterBuilder.java
+++ b/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/DefaultSegmentWriterBuilder.java
@@@ -19,10 -19,9 +19,10 @@@
package org.apache.jackrabbit.oak.segment;
- import static org.apache.jackrabbit.guava.common.base.Preconditions.checkNotNull;
-import static com.google.common.base.Preconditions.checkNotNull;
+ import static org.apache.jackrabbit.oak.segment.SegmentBufferWriterPool.PoolType;
+
++import static java.util.Objects.requireNonNull;
+
- import org.apache.jackrabbit.guava.common.base.Supplier;
- import org.apache.jackrabbit.guava.common.base.Suppliers;
import org.apache.jackrabbit.oak.segment.WriterCacheManager.Empty;
import org.apache.jackrabbit.oak.segment.file.FileStore;
import org.apache.jackrabbit.oak.segment.file.ReadOnlyFileStore;
@@@ -62,7 -64,7 +65,7 @@@ public final class DefaultSegmentWriter
private WriterCacheManager cacheManager = new WriterCacheManager.Default();
private DefaultSegmentWriterBuilder(@NotNull String name) {
-- this.name = checkNotNull(name);
++ this.name = requireNonNull(name);
}
/**
@@@ -87,7 -89,7 +90,7 @@@
*/
@NotNull
public DefaultSegmentWriterBuilder withGeneration(@NotNull Supplier<GCGeneration> generation) {
-- this.generation = checkNotNull(generation);
++ this.generation = requireNonNull(generation);
return this;
}
@@@ -97,7 -99,7 +100,7 @@@
*/
@NotNull
public DefaultSegmentWriterBuilder withGeneration(@NotNull GCGeneration generation) {
- this.generation = Suppliers.ofInstance(checkNotNull(generation));
- this.generation = () -> checkNotNull(generation);
++ this.generation = () -> requireNonNull(generation);
return this;
}
@@@ -126,7 -133,7 +134,7 @@@
*/
@NotNull
public DefaultSegmentWriterBuilder with(WriterCacheManager cacheManager) {
-- this.cacheManager = checkNotNull(cacheManager);
++ this.cacheManager = requireNonNull(cacheManager);
return this;
}
@@@ -146,7 -153,7 +154,7 @@@
@NotNull
public DefaultSegmentWriter build(@NotNull FileStore store) {
return new DefaultSegmentWriter(
-- checkNotNull(store),
++ requireNonNull(store),
store.getReader(),
store.getSegmentIdProvider(),
store.getBlobStore(),
@@@ -164,7 -171,7 +172,7 @@@
@NotNull
public DefaultSegmentWriter build(@NotNull ReadOnlyFileStore store) {
return new DefaultSegmentWriter(
-- checkNotNull(store),
++ requireNonNull(store),
store.getReader(),
store.getSegmentIdProvider(),
store.getBlobStore(),
@@@ -199,7 -206,7 +207,7 @@@
@NotNull
public DefaultSegmentWriter build(@NotNull MemoryStore store) {
return new DefaultSegmentWriter(
-- checkNotNull(store),
++ requireNonNull(store),
store.getReader(),
store.getSegmentIdProvider(),
store.getBlobStore(),
diff --cc oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/RecordCache.java
index 3b8817ad89,ec00bf3635..cd9d22f7d1
--- a/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/RecordCache.java
+++ b/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/RecordCache.java
@@@ -19,16 -19,15 +19,20 @@@
package org.apache.jackrabbit.oak.segment;
- import static org.apache.jackrabbit.guava.common.base.Preconditions.checkNotNull;
-import com.google.common.base.Supplier;
-import com.google.common.cache.CacheBuilder;
-import com.google.common.cache.CacheStats;
-import com.google.common.cache.RemovalListener;
-import com.google.common.cache.Weigher;
++import static java.util.Objects.requireNonNull;
+
+import java.util.LinkedHashMap;
+import java.util.Map;
+
- import org.apache.jackrabbit.guava.common.cache.CacheStats;
- import org.apache.jackrabbit.guava.common.cache.Weigher;
import org.jetbrains.annotations.NotNull;
-import java.util.Objects;
+import org.apache.jackrabbit.guava.common.base.Supplier;
++import org.apache.jackrabbit.guava.common.cache.CacheBuilder;
++import org.apache.jackrabbit.guava.common.cache.CacheStats;
++import org.apache.jackrabbit.guava.common.cache.RemovalListener;
++import org.apache.jackrabbit.guava.common.cache.Weigher;
++
+ import java.util.concurrent.atomic.LongAdder;
/**
* Partial mapping of keys of type {@code K} to values of type {@link RecordId}. This is
@@@ -91,7 -83,7 +88,7 @@@ public abstract class RecordCache<K> im
if (size <= 0) {
return Empty.emptyFactory();
} else {
- return Default.defaultFactory(size, checkNotNull(weigher));
- return Default.defaultFactory(size, Objects.requireNonNull(weigher));
++ return Default.defaultFactory(size, requireNonNull(weigher));
}
}
@@@ -141,22 -136,25 +141,25 @@@
}
private static class Default<K> extends RecordCache<K> {
-
@NotNull
- private final Map<K, RecordId> records;
-
- private final com.google.common.cache.Cache<K, RecordId> cache;
++ private final org.apache.jackrabbit.guava.common.cache.Cache<K, RecordId> cache;
@NotNull
private final Weigher<K, RecordId> weigher;
+ @NotNull
+ private final LongAdder weight = new LongAdder();
+ @NotNull
+ private final LongAdder loadCount = new LongAdder();
- private long weight = 0;
+ @Override
+ public @NotNull CacheStats getStats() {
+ CacheStats internalStats = cache.stats();
+ // any addition to the cache counts as load by our definition
+ return new CacheStats(internalStats.hitCount(), internalStats.missCount(),
+ loadCount.sum(), 0, 0, internalStats.evictionCount());
+ }
- static final <K> Supplier<RecordCache<K>> defaultFactory(final int size, @NotNull final Weigher<K, RecordId> weigher) {
- return new Supplier<RecordCache<K>>() {
- @Override
- public RecordCache<K> get() {
- return new Default<>(size, checkNotNull(weigher));
- }
- };
+ static <K> Supplier<RecordCache<K>> defaultFactory(final int size, @NotNull final Weigher<K, RecordId> weigher) {
- return () -> new Default<>(size, Objects.requireNonNull(weigher));
++ return () -> new Default<>(size, requireNonNull(weigher));
}
Default(final int size, @NotNull final Weigher<K, RecordId> weigher) {
diff --cc oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/SegmentBufferWriterPool.java
index 59c4fadaaf,59236469f5..6d9bc5fa8f
--- a/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/SegmentBufferWriterPool.java
+++ b/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/SegmentBufferWriterPool.java
@@@ -19,24 -19,27 +19,29 @@@
package org.apache.jackrabbit.oak.segment;
-import com.google.common.util.concurrent.Monitor;
-import org.apache.jackrabbit.oak.segment.file.tar.GCGeneration;
-import org.jetbrains.annotations.NotNull;
+import static org.apache.jackrabbit.guava.common.base.Preconditions.checkNotNull;
+import static org.apache.jackrabbit.guava.common.base.Preconditions.checkState;
+import static org.apache.jackrabbit.guava.common.collect.Lists.newArrayList;
++import static org.apache.jackrabbit.guava.common.collect.Maps.newConcurrentMap;
+import static org.apache.jackrabbit.guava.common.collect.Maps.newHashMap;
+import static org.apache.jackrabbit.guava.common.collect.Sets.newHashSet;
+import static java.lang.Thread.currentThread;
++import static java.util.Objects.requireNonNull;
import java.io.IOException;
import java.util.AbstractMap.SimpleImmutableEntry;
import java.util.List;
import java.util.Map;
import java.util.Set;
+ import java.util.concurrent.ConcurrentMap;
+ import java.util.concurrent.locks.ReadWriteLock;
+ import java.util.concurrent.locks.ReentrantReadWriteLock;
+ import java.util.function.Supplier;
- import org.apache.jackrabbit.guava.common.base.Supplier;
-import static com.google.common.base.Preconditions.checkState;
-import static com.google.common.collect.Lists.newArrayList;
-import static com.google.common.collect.Maps.newConcurrentMap;
-import static com.google.common.collect.Maps.newHashMap;
-import static com.google.common.collect.Sets.newHashSet;
-import static java.lang.Thread.currentThread;
-import static java.util.Objects.requireNonNull;
+import org.apache.jackrabbit.guava.common.util.concurrent.Monitor;
+import org.apache.jackrabbit.guava.common.util.concurrent.Monitor.Guard;
+import org.apache.jackrabbit.oak.segment.file.tar.GCGeneration;
+import org.jetbrains.annotations.NotNull;
/**
* This {@link WriteOperationHandler} uses a pool of {@link SegmentBufferWriter}s,
diff --cc oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/AbstractCompactionStrategy.java
index fc13fa9819,27a783fd23..cdde3c7918
--- a/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/AbstractCompactionStrategy.java
+++ b/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/AbstractCompactionStrategy.java
@@@ -27,14 -27,15 +27,15 @@@ import static org.apache.jackrabbit.oak
import static org.apache.jackrabbit.oak.segment.file.TarRevisions.EXPEDITE_OPTION;
import static org.apache.jackrabbit.oak.segment.file.TarRevisions.timeout;
-import com.google.common.base.Function;
+import org.apache.jackrabbit.guava.common.base.Function;
- import org.apache.jackrabbit.oak.segment.CheckpointCompactor;
- import org.apache.jackrabbit.oak.segment.ClassicCompactor;
- import org.apache.jackrabbit.oak.segment.Compactor;
import org.apache.jackrabbit.oak.segment.RecordId;
import org.apache.jackrabbit.oak.segment.SegmentNodeState;
+ import org.apache.jackrabbit.oak.segment.Compactor;
import org.apache.jackrabbit.oak.segment.SegmentWriter;
+ import org.apache.jackrabbit.oak.segment.ClassicCompactor;
+ import org.apache.jackrabbit.oak.segment.CheckpointCompactor;
+ import org.apache.jackrabbit.oak.segment.ParallelCompactor;
import org.apache.jackrabbit.oak.segment.compaction.SegmentGCOptions.CompactorType;
import org.apache.jackrabbit.oak.segment.compaction.SegmentGCOptions.GCType;
import org.apache.jackrabbit.oak.segment.file.cancel.Cancellation;
diff --cc oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/FileStore.java
index fa9fd05b63,8ec8e76b7e..18375e28e7
--- a/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/FileStore.java
+++ b/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/FileStore.java
@@@ -35,13 -35,16 +35,15 @@@ import java.util.concurrent.ExecutionEx
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.function.Consumer;
-import com.google.common.base.Supplier;
-import com.google.common.io.Closer;
-import com.google.common.util.concurrent.UncheckedExecutionException;
-
+import org.apache.jackrabbit.guava.common.base.Supplier;
+import org.apache.jackrabbit.guava.common.io.Closer;
+import org.apache.jackrabbit.guava.common.util.concurrent.UncheckedExecutionException;
import org.apache.jackrabbit.oak.commons.Buffer;
- import org.apache.jackrabbit.oak.segment.RecordId;
import org.apache.jackrabbit.oak.segment.Segment;
+ import org.apache.jackrabbit.oak.segment.RecordId;
import org.apache.jackrabbit.oak.segment.SegmentId;
+ import org.apache.jackrabbit.oak.segment.SegmentWriter;
+ import org.apache.jackrabbit.oak.segment.SegmentBufferWriterPool;
import org.apache.jackrabbit.oak.segment.SegmentNodeState;
import org.apache.jackrabbit.oak.segment.SegmentNotFoundException;
import org.apache.jackrabbit.oak.segment.SegmentNotFoundExceptionListener;
diff --cc oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/PriorityCache.java
index 00038e446b,6ff2c025bd..bea1e9e096
--- a/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/PriorityCache.java
+++ b/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/file/PriorityCache.java
@@@ -19,23 -19,27 +19,29 @@@
package org.apache.jackrabbit.oak.segment.file;
-import com.google.common.base.Predicate;
-import com.google.common.base.Supplier;
-import com.google.common.cache.CacheStats;
-import com.google.common.cache.Weigher;
-import org.apache.jackrabbit.oak.segment.CacheWeights;
-import org.jetbrains.annotations.NotNull;
-import org.jetbrains.annotations.Nullable;
+
-import java.util.concurrent.atomic.AtomicInteger;
-import java.util.concurrent.atomic.AtomicLong;
-import java.util.concurrent.atomic.LongAdder;
-import java.util.concurrent.locks.ReentrantLock;
+import static org.apache.jackrabbit.guava.common.base.Preconditions.checkArgument;
+import static org.apache.jackrabbit.guava.common.base.Preconditions.checkNotNull;
+
-import static com.google.common.base.Preconditions.checkArgument;
-import static com.google.common.base.Preconditions.checkNotNull;
import static java.lang.Integer.bitCount;
import static java.lang.Integer.numberOfTrailingZeros;
import static java.lang.Long.numberOfLeadingZeros;
import static java.lang.Math.max;
import static java.util.Arrays.fill;
++import java.util.concurrent.atomic.AtomicInteger;
++import java.util.concurrent.atomic.AtomicLong;
++import java.util.concurrent.atomic.LongAdder;
++import java.util.concurrent.locks.ReentrantLock;
++
++import org.apache.jackrabbit.guava.common.base.Predicate;
++import org.apache.jackrabbit.guava.common.base.Supplier;
+import org.apache.jackrabbit.guava.common.cache.CacheStats;
+import org.apache.jackrabbit.guava.common.cache.Weigher;
+import org.apache.jackrabbit.oak.segment.CacheWeights;
+import org.jetbrains.annotations.NotNull;
+import org.jetbrains.annotations.Nullable;
+
- import org.apache.jackrabbit.guava.common.base.Predicate;
- import org.apache.jackrabbit.guava.common.base.Supplier;
-
/**
* {@code PriorityCache} implements a partial mapping from keys of type {@code K} to values
* of type {@code V}. Mappings are associates with a cost, which states how expensive it is
@@@ -133,7 -132,7 +134,7 @@@ public class PriorityCache<K, V>
* @return the next power of two starting from {@code size}.
*/
public static long nextPowerOfTwo(int size) {
-- return 1L << (64L - numberOfLeadingZeros((long)max(1, size) - 1L));
++ return 1L << (64L - numberOfLeadingZeros(max(1, size) - 1L));
}
/**
diff --cc oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/memory/MemoryStore.java
index 50b0ee99f7,2474b8376c..ddda4676a0
--- a/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/memory/MemoryStore.java
+++ b/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/memory/MemoryStore.java
@@@ -24,22 -24,22 +24,22 @@@ import java.io.IOException
import java.util.Set;
import java.util.concurrent.ConcurrentMap;
-import com.google.common.collect.Maps;
+import org.apache.jackrabbit.guava.common.collect.Maps;
import org.apache.jackrabbit.oak.commons.Buffer;
- import org.apache.jackrabbit.oak.segment.CachingSegmentReader;
- import org.apache.jackrabbit.oak.segment.Revisions;
import org.apache.jackrabbit.oak.segment.Segment;
import org.apache.jackrabbit.oak.segment.SegmentId;
+ import org.apache.jackrabbit.oak.segment.Revisions;
+ import org.apache.jackrabbit.oak.segment.SegmentStore;
+ import org.apache.jackrabbit.oak.segment.SegmentReader;
+ import org.apache.jackrabbit.oak.segment.SegmentWriter;
+ import org.apache.jackrabbit.oak.segment.SegmentTracker;
import org.apache.jackrabbit.oak.segment.SegmentIdFactory;
import org.apache.jackrabbit.oak.segment.SegmentIdProvider;
+ import org.apache.jackrabbit.oak.segment.CachingSegmentReader;
import org.apache.jackrabbit.oak.segment.SegmentNotFoundException;
- import org.apache.jackrabbit.oak.segment.SegmentReader;
- import org.apache.jackrabbit.oak.segment.SegmentStore;
- import org.apache.jackrabbit.oak.segment.SegmentTracker;
- import org.apache.jackrabbit.oak.segment.SegmentWriter;
- import org.apache.jackrabbit.oak.spi.blob.BlobStore;
import org.apache.jackrabbit.oak.stats.NoopStats;
+ import org.apache.jackrabbit.oak.spi.blob.BlobStore;
import org.jetbrains.annotations.NotNull;
import org.jetbrains.annotations.Nullable;
diff --cc oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/RecordCacheStatsTest.java
index 98db981b14,ee04ddd5ad..e177200f9b
--- a/oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/RecordCacheStatsTest.java
+++ b/oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/RecordCacheStatsTest.java
@@@ -18,18 -18,15 +18,17 @@@
package org.apache.jackrabbit.oak.segment;
+import static org.apache.jackrabbit.oak.segment.RecordCache.newRecordCache;
+import static org.junit.Assert.assertEquals;
+
- import java.io.IOException;
- import java.util.Random;
-
+import org.apache.jackrabbit.guava.common.cache.CacheStats;
import org.apache.jackrabbit.oak.segment.memory.MemoryStore;
import org.junit.Before;
import org.junit.Test;
- import org.apache.jackrabbit.guava.common.base.Supplier;
+ import java.io.IOException;
+ import java.util.Random;
-
-import static org.apache.jackrabbit.oak.segment.RecordCache.newRecordCache;
-import static org.junit.Assert.assertEquals;
++import java.util.function.Supplier;
public class RecordCacheStatsTest {
private static final String NAME = "cache stats";
diff --cc oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/file/PriorityCacheTest.java
index 3d2be39e04,87058a3ac9..380b595305
--- a/oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/file/PriorityCacheTest.java
+++ b/oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/file/PriorityCacheTest.java
@@@ -19,21 -19,19 +19,20 @@@
package org.apache.jackrabbit.oak.segment.file;
-import com.google.common.cache.Weigher;
-import org.apache.jackrabbit.oak.segment.CacheWeights;
-import org.junit.Test;
-
-import java.util.Random;
-
import static java.lang.Integer.valueOf;
import static org.junit.Assert.assertEquals;
- import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertNull;
+ import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;
import static org.junit.Assume.assumeTrue;
- import java.util.Random;
-
+import org.apache.jackrabbit.guava.common.cache.Weigher;
+import org.apache.jackrabbit.oak.segment.CacheWeights;
+import org.junit.Test;
+
- import org.apache.jackrabbit.guava.common.base.Predicate;
++import java.util.Random;
++import java.util.function.Predicate;
+
public class PriorityCacheTest {
@Test(expected = IllegalArgumentException.class)
diff --cc oak-shaded-guava/.DS_Store
index 0000000000,0000000000..91a7914c43
new file mode 100644
Binary files differ
[jackrabbit-oak] 02/03: OAK-9922 - Parallel Compaction Merge branch 'lweitzendorf-breadth-first-scan' into trunk
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adulceanu pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/jackrabbit-oak.git
commit c7aa4f0f7b19567544a633a1e2df6e10fb1b42f9
Merge: 951d69e57a ed8494b5e3
Author: dulceanu <an...@gmail.com>
AuthorDate: Thu Jul 27 16:40:37 2023 +0300
OAK-9922 - Parallel Compaction
Merge branch 'lweitzendorf-breadth-first-scan' into trunk
.DS_Store | Bin 0 -> 6148 bytes
.../plugins/index/counter/ApproximateCounter.java | 9 +-
.../plugins/index/counter/NodeCounterEditor.java | 1 +
.../oak/plugins/index/counter/jmx/NodeCounter.java | 2 +-
.../index/property/PropertyIndexInfoProvider.java | 2 +-
.../strategy/ContentMirrorStoreStrategy.java | 2 +-
.../strategy/UniqueEntryStoreStrategy.java | 2 +-
.../strategy/ContentMirrorStoreStrategyTest.java | 2 +-
.../site/markdown/nodestore/segment/overview.md | 27 +-
.../apache/jackrabbit/oak/run/CompactCommand.java | 54 +--
.../oak/segment/aws/tool/AwsCompact.java | 34 +-
.../oak/segment/aws/tool/AwsToolUtils.java | 19 +-
.../oak/segment/azure/tool/AzureCompact.java | 21 +-
.../oak/segment/azure/tool/ToolUtils.java | 26 +-
.../oak/segment/CheckpointCompactor.java | 32 +-
.../jackrabbit/oak/segment/ClassicCompactor.java | 21 +-
.../jackrabbit/oak/segment/CompactorUtils.java | 35 ++
.../oak/segment/DefaultSegmentWriterBuilder.java | 90 +++--
.../jackrabbit/oak/segment/ParallelCompactor.java | 370 +++++++++++++++++++
.../apache/jackrabbit/oak/segment/RecordCache.java | 119 +++----
.../oak/segment/SegmentBufferWriterPool.java | 390 ++++++++++++++-------
.../oak/segment/compaction/SegmentGCOptions.java | 47 ++-
.../segment/file/AbstractCompactionStrategy.java | 31 +-
.../jackrabbit/oak/segment/file/FileStore.java | 7 +-
.../oak/segment/file/FileStoreBuilder.java | 14 +-
.../oak/segment/file/GCNodeWriteMonitor.java | 66 ++--
.../jackrabbit/oak/segment/file/PriorityCache.java | 275 +++++++++------
.../jackrabbit/oak/segment/memory/MemoryStore.java | 14 +-
.../jackrabbit/oak/segment/tool/Compact.java | 22 +-
...java => AbstractCompactorExternalBlobTest.java} | 29 +-
...mpactorTest.java => AbstractCompactorTest.java} | 27 +-
.../CheckpointCompactorExternalBlobTest.java | 135 +------
.../oak/segment/CheckpointCompactorTest.java | 111 +-----
.../oak/segment/CompactionAndCleanupIT.java | 13 +-
...actorTestUtils.java => CompactorTestUtils.java} | 20 +-
.../jackrabbit/oak/segment/NodeRecordTest.java | 4 +-
.../segment/ParallelCompactorExternalBlobTest.java | 63 ++++
.../oak/segment/ParallelCompactorTest.java | 63 ++++
.../oak/segment/RecordCacheStatsTest.java | 20 +-
.../jackrabbit/oak/segment/RecordCacheTest.java | 54 ++-
.../oak/segment/SegmentBufferWriterPoolTest.java | 67 ++--
.../segment/file/ConcurrentPriorityCacheTest.java | 236 +++++++++++++
.../oak/segment/file/PriorityCacheTest.java | 27 +-
oak-shaded-guava/.DS_Store | Bin 0 -> 6148 bytes
.../oak/plugins/index}/ApproximateCounter.java | 2 +-
.../oak/plugins/index}/ApproximateCounterTest.java | 9 +-
46 files changed, 1774 insertions(+), 840 deletions(-)
[jackrabbit-oak] 03/03: Merge branch 'trunk' of github.com:apache/jackrabbit-oak into trunk
Posted by ad...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
adulceanu pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/jackrabbit-oak.git
commit 1643f1ba56284033042a2f965b599528854f04c7
Merge: c7aa4f0f7b c18d0c5145
Author: dulceanu <an...@gmail.com>
AuthorDate: Thu Jul 27 16:42:47 2023 +0300
Merge branch 'trunk' of github.com:apache/jackrabbit-oak into trunk
.../jackrabbit/oak/api/jmx/CacheStatsMBean.java | 16 +++---
oak-auth-external/pom.xml | 7 ---
oak-authorization-cug/pom.xml | 7 ---
oak-authorization-principalbased/pom.xml | 7 ---
.../oak/benchmark/ReplicaCrashResilienceTest.java | 4 +-
oak-blob-cloud-azure/pom.xml | 2 +-
.../apache/jackrabbit/oak/commons/LongUtils.java | 4 +-
.../apache/jackrabbit/oak/commons/Profiler.java | 2 -
oak-it-osgi/pom.xml | 5 --
oak-it-osgi/test-bundles.xml | 1 -
oak-jcr/pom.xml | 7 ---
oak-lucene/pom.xml | 7 ---
oak-parent/pom.xml | 57 +---------------------
.../oak/run/osgi/JaasConfigSpiTest.groovy | 2 +-
.../oak/run/osgi/LuceneSupportTest.groovy | 2 +-
.../oak/run/osgi/SegmentNodeStoreConfigTest.groovy | 4 +-
.../oak/run/osgi/TokenAuthenticationTest.groovy | 2 +-
.../oak/console/commands/CdCommand.groovy | 4 +-
.../oak/console/commands/LsCommand.groovy | 2 +-
.../oak/console/commands/LuceneCommand.groovy | 4 +-
.../console/commands/PrintDocumentCommand.groovy | 2 +-
oak-search-mt/pom.xml | 1 -
oak-search/pom.xml | 5 --
oak-segment-azure/pom.xml | 2 +-
oak-segment-tar/pom.xml | 8 ---
oak-store-composite/pom.xml | 6 +++
oak-store-document/pom.xml | 8 ---
.../oak/plugins/document/DocumentNodeStore.java | 4 +-
.../plugins/document/mongo/MongoDocumentStore.java | 4 +-
oak-store-spi/pom.xml | 10 ----
30 files changed, 36 insertions(+), 160 deletions(-)