You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by bt...@apache.org on 2020/07/23 07:53:28 UTC
[james-project] 17/18: JAMES-3313 Drop support for Hybrid BlobStore
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 20812f2e335242783da4dce13ced7b907e80d9d1
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Fri Jul 17 17:24:01 2020 +0700
JAMES-3313 Drop support for Hybrid BlobStore
---
CHANGELOG.md | 10 +-
.../destination/conf/blob.properties | 11 +-
.../destination/conf/blob.properties | 11 +-
pom.xml | 5 -
server/blob/blob-union/pom.xml | 75 ---
.../apache/james/blob/union/HybridBlobStore.java | 239 ---------
.../james/blob/union/HybridBlobStoreTest.java | 536 ---------------------
server/blob/pom.xml | 1 -
.../guice/cassandra-rabbitmq-guice/pom.xml | 4 -
.../modules/blobstore/BlobStoreChoosingModule.java | 23 -
.../modules/blobstore/BlobStoreConfiguration.java | 7 +-
.../modules/blobstore/BlobStoreModulesChooser.java | 49 +-
.../blobstore/BlobStoreConfigurationTest.java | 33 +-
.../blobstore/BlobStoreModulesChooserTest.java | 64 ---
src/site/xdoc/server/config-blobstore.xml | 9 -
15 files changed, 14 insertions(+), 1063 deletions(-)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index f5c8690..31f9e86 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -71,11 +71,6 @@ of tasks being currently executed.
- JAMES-3305 Avoid crashes upon deserialization issues when consuming RabbitMQ messages, leverage dead-letter feature
- JAMES-3212 JMAP Handle subcrible/unsubcrible child's folder when update mailbox
-### Deprecated
-- HybridBlobStore. This will be removed after 3.6.0 release. Introduced to fasten small blob access, its usage could be
-compared to a cache, but with a sub-optimal implementation (no eviction, default replication factor, no circuit breaking).
-Use BlobStore cache instead.
-
### Removed
- Classes marked as deprecated whose removal was planned after 3.4.0 release (See JAMES-2703). This includes:
- SieveDefaultRepository. Please use SieveFileRepository instead.
@@ -90,7 +85,10 @@ This parameter could cause body content alteration leading to DKIM invalid DKIM
Thanks to Sergey B. for the report.
More details about the property is at [java mail doc](https://javaee.github.io/javamail/docs/api/com/sun/mail/smtp/package-summary.html)
- JAMES-3122 LogEnabled API in Spring product had been removed for Log4J2 adoption for Java 9+ runtime compatibility.
-
+ - HybridBlobStore. This will be removed after 3.6.0 release. Introduced to fasten small blob access, its usage could be
+ compared to a cache, but with a sub-optimal implementation (no eviction, default replication factor, no circuit breaking).
+ Use BlobStore cache instead.
+
### Third party softwares
- The distributed James server product (relying on Guice, Cassandra, ElasticSearch, RabbitMQ and optionally Swift) now needs at least RabbitMQ 3.8.1.
- Tika prior 1.24 is subject to multiple CVEs. We recommend the upgrade.
diff --git a/dockerfiles/run/guice/cassandra-rabbitmq-ldap/destination/conf/blob.properties b/dockerfiles/run/guice/cassandra-rabbitmq-ldap/destination/conf/blob.properties
index 4fa1dcd..40aee03 100644
--- a/dockerfiles/run/guice/cassandra-rabbitmq-ldap/destination/conf/blob.properties
+++ b/dockerfiles/run/guice/cassandra-rabbitmq-ldap/destination/conf/blob.properties
@@ -2,18 +2,9 @@
# Read https://james.apache.org/server/config-blobstore.html for further details
# Choose your BlobStore implementation
-# Mandatory, allowed values are: cassandra, objectstorage, hybrid (deprecated)
-# hybrid is using both objectstorage for unfrequently read or big blobs & cassandra for small, often read blobs
-# Deprecated. Use CachedBlobStore instead. Introduced to fasten small blob access, its usage could be compared
-# to a cache, but with a sub-optimal implementation (no eviction, default replication factor, no circuit breaking).
+# Mandatory, allowed values are: cassandra, objectstorage
implementation=objectstorage
-# ========================================= Hybrid BlobStore ======================================
-# hybrid is using both objectstorage for unfrequently read or big blobs & cassandra for small, often read blobs
-# Size threshold for considering a blob as 'big', causing it to be saved in the low cost blobStore
-# Optional, defaults to 32768 bytes (32KB), must be positive
-hybrid.size.threshold=32768
-
# ========================================= Cassandra BlobStore Cache ======================================
# A cassandra cache can be enabled to reduce latency when reading small blobs frequently
# A dedicated keyspace with a replication factor of one is then used
diff --git a/dockerfiles/run/guice/cassandra-rabbitmq/destination/conf/blob.properties b/dockerfiles/run/guice/cassandra-rabbitmq/destination/conf/blob.properties
index da0945f..8c98913 100644
--- a/dockerfiles/run/guice/cassandra-rabbitmq/destination/conf/blob.properties
+++ b/dockerfiles/run/guice/cassandra-rabbitmq/destination/conf/blob.properties
@@ -2,10 +2,7 @@
# Read https://james.apache.org/server/config-blobstore.html for further details
# Choose your BlobStore implementation
-# Mandatory, allowed values are: cassandra, objectstorage, hybrid (deprecated)
-# hybrid is using both objectstorage for unfrequently read or big blobs & cassandra for small, often read blobs
-# Deprecated. Use CachedBlobStore instead. Introduced to fasten small blob access, its usage could be compared
-# to a cache, but with a sub-optimal implementation (no eviction, default replication factor, no circuit breaking).
+# Mandatory, allowed values are: cassandra, objectstorage
implementation=objectstorage
# ========================================= Cassandra BlobStore Cache ======================================
@@ -30,12 +27,6 @@ cache.enable=false
# Units: bytes, Kib, MiB, GiB, TiB
# cache.sizeThresholdInBytes=8 KiB
-# ========================================= Hybrid BlobStore ======================================
-# hybrid is using both objectstorage for unfrequently read or big blobs & cassandra for small, often read blobs
-# Size threshold for considering a blob as 'big', causing it to be saved in the low cost blobStore
-# Optional, defaults to 32768 bytes (32KB), must be positive
-hybrid.size.threshold=32768
-
# ============================================== ObjectStorage ============================================
# ========================================= ObjectStorage Codec ======================================
diff --git a/pom.xml b/pom.xml
index 8881160..fbdf065 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1196,11 +1196,6 @@
<version>${project.version}</version>
<type>test-jar</type>
</dependency>
- <dependency>
- <groupId>${james.groupId}</groupId>
- <artifactId>blob-union</artifactId>
- <version>${project.version}</version>
- </dependency>
<dependency>
<groupId>${james.groupId}</groupId>
<artifactId>event-sourcing-core</artifactId>
diff --git a/server/blob/blob-union/pom.xml b/server/blob/blob-union/pom.xml
deleted file mode 100644
index bcc1d4b..0000000
--- a/server/blob/blob-union/pom.xml
+++ /dev/null
@@ -1,75 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!--
- Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
--->
-<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
- <modelVersion>4.0.0</modelVersion>
-
- <parent>
- <artifactId>james-server-blob</artifactId>
- <groupId>org.apache.james</groupId>
- <version>3.6.0-SNAPSHOT</version>
- <relativePath>../pom.xml</relativePath>
- </parent>
-
- <artifactId>blob-union</artifactId>
- <packaging>jar</packaging>
-
- <name>Apache James :: Server :: Blob :: Union Blob Storage</name>
- <description>
- An implementation of BlobStore which relies on a current and a legacy BlobStore by order for reading and writing
- blobs with fallback mechanism.
- </description>
-
- <dependencies>
- <dependency>
- <groupId>${james.groupId}</groupId>
- <artifactId>blob-api</artifactId>
- </dependency>
- <dependency>
- <groupId>${james.groupId}</groupId>
- <artifactId>blob-api</artifactId>
- <type>test-jar</type>
- <scope>test</scope>
- </dependency>
- <dependency>
- <groupId>${james.groupId}</groupId>
- <artifactId>blob-memory</artifactId>
- <scope>test</scope>
- </dependency>
- <dependency>
- <groupId>${james.groupId}</groupId>
- <artifactId>james-server-testing</artifactId>
- <scope>test</scope>
- </dependency>
- <dependency>
- <groupId>${james.groupId}</groupId>
- <artifactId>james-server-util</artifactId>
- </dependency>
- <dependency>
- <groupId>${james.groupId}</groupId>
- <artifactId>testing-base</artifactId>
- <scope>test</scope>
- </dependency>
- <dependency>
- <groupId>org.apache.commons</groupId>
- <artifactId>commons-configuration2</artifactId>
- </dependency>
- </dependencies>
-
-</project>
diff --git a/server/blob/blob-union/src/main/java/org/apache/james/blob/union/HybridBlobStore.java b/server/blob/blob-union/src/main/java/org/apache/james/blob/union/HybridBlobStore.java
deleted file mode 100644
index 6dd06ad..0000000
--- a/server/blob/blob-union/src/main/java/org/apache/james/blob/union/HybridBlobStore.java
+++ /dev/null
@@ -1,239 +0,0 @@
-/****************************************************************
- * Licensed to the Apache Software Foundation (ASF) under one *
- * or more contributor license agreements. See the NOTICE file *
- * distributed with this work for additional information *
- * regarding copyright ownership. The ASF licenses this file *
- * to you under the Apache License, Version 2.0 (the *
- * "License"); you may not use this file except in compliance *
- * with the License. You may obtain a copy of the License at *
- * *
- * http://www.apache.org/licenses/LICENSE-2.0 *
- * *
- * Unless required by applicable law or agreed to in writing, *
- * software distributed under the License is distributed on an *
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
- * KIND, either express or implied. See the License for the *
- * specific language governing permissions and limitations *
- * under the License. *
- ****************************************************************/
-
-package org.apache.james.blob.union;
-
-import java.io.BufferedInputStream;
-import java.io.IOException;
-import java.io.InputStream;
-import java.util.Objects;
-import java.util.Optional;
-
-import org.apache.james.blob.api.BlobId;
-import org.apache.james.blob.api.BlobStore;
-import org.apache.james.blob.api.BucketName;
-import org.apache.james.blob.api.ObjectNotFoundException;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import com.google.common.base.MoreObjects;
-import com.google.common.base.Preconditions;
-
-import reactor.core.publisher.Mono;
-
-/**
- * Will be removed in future release (3.6.0).
- * Prefer using CachedBlobStore.
- *
- * Introduced to fasten small blob access, its usage could be compared to a cache, but with a sub-optimal
- * implementation (no eviction, default replication factor, no circuit breaking).
- */
-@Deprecated
-public class HybridBlobStore implements BlobStore {
- @FunctionalInterface
- public interface RequireLowCost {
- RequireHighPerformance lowCost(BlobStore blobStore);
- }
-
- @FunctionalInterface
- public interface RequireHighPerformance {
- RequireConfiguration highPerformance(BlobStore blobStore);
- }
-
- @FunctionalInterface
- public interface RequireConfiguration {
- Builder configuration(Configuration configuration);
- }
-
- public static class Builder {
- private final BlobStore lowCostBlobStore;
- private final BlobStore highPerformanceBlobStore;
- private final Configuration configuration;
-
- Builder(BlobStore lowCostBlobStore, BlobStore highPerformanceBlobStore, Configuration configuration) {
- this.lowCostBlobStore = lowCostBlobStore;
- this.highPerformanceBlobStore = highPerformanceBlobStore;
- this.configuration = configuration;
- }
-
- public HybridBlobStore build() {
- return new HybridBlobStore(
- lowCostBlobStore,
- highPerformanceBlobStore,
- configuration);
- }
- }
-
- public static class Configuration {
- public static final int DEFAULT_SIZE_THRESHOLD = 32 * 1024;
- public static final Configuration DEFAULT = new Configuration(DEFAULT_SIZE_THRESHOLD);
- private static final String PROPERTY_NAME = "hybrid.size.threshold";
-
- public static Configuration from(org.apache.commons.configuration2.Configuration propertiesConfiguration) {
- return new Configuration(Optional.ofNullable(propertiesConfiguration.getInteger(PROPERTY_NAME, null))
- .orElse(DEFAULT_SIZE_THRESHOLD));
- }
-
- private final int sizeThreshold;
-
- public Configuration(int sizeThreshold) {
- Preconditions.checkArgument(sizeThreshold >= 0, "'" + PROPERTY_NAME + "' needs to be positive");
-
- this.sizeThreshold = sizeThreshold;
- }
-
- public int getSizeThreshold() {
- return sizeThreshold;
- }
-
- @Override
- public final boolean equals(Object o) {
- if (o instanceof Configuration) {
- Configuration that = (Configuration) o;
-
- return Objects.equals(this.sizeThreshold, that.sizeThreshold);
- }
- return false;
- }
-
- @Override
- public final int hashCode() {
- return Objects.hash(sizeThreshold);
- }
- }
-
- private static final Logger LOGGER = LoggerFactory.getLogger(HybridBlobStore.class);
-
- public static RequireLowCost builder() {
- return lowCost -> highPerformance -> configuration -> new Builder(lowCost, highPerformance, configuration);
- }
-
- private final BlobStore lowCostBlobStore;
- private final BlobStore highPerformanceBlobStore;
- private final Configuration configuration;
-
- private HybridBlobStore(BlobStore lowCostBlobStore, BlobStore highPerformanceBlobStore, Configuration configuration) {
- this.lowCostBlobStore = lowCostBlobStore;
- this.highPerformanceBlobStore = highPerformanceBlobStore;
- this.configuration = configuration;
- }
-
- @Override
- public Mono<BlobId> save(BucketName bucketName, byte[] data, StoragePolicy storagePolicy) {
- return selectBlobStore(storagePolicy, Mono.just(data.length > configuration.getSizeThreshold()))
- .flatMap(blobStore -> Mono.from(blobStore.save(bucketName, data, storagePolicy)));
- }
-
- @Override
- public Mono<BlobId> save(BucketName bucketName, InputStream data, StoragePolicy storagePolicy) {
- Preconditions.checkNotNull(data);
-
- BufferedInputStream bufferedInputStream = new BufferedInputStream(data, configuration.getSizeThreshold() + 1);
- return selectBlobStore(storagePolicy, Mono.fromCallable(() -> isItABigStream(bufferedInputStream)))
- .flatMap(blobStore -> Mono.from(blobStore.save(bucketName, bufferedInputStream, storagePolicy)));
- }
-
- private Mono<BlobStore> selectBlobStore(StoragePolicy storagePolicy, Mono<Boolean> largeData) {
- switch (storagePolicy) {
- case LOW_COST:
- return Mono.just(lowCostBlobStore);
- case SIZE_BASED:
- return largeData.map(isLarge -> {
- if (isLarge) {
- return lowCostBlobStore;
- }
- return highPerformanceBlobStore;
- });
- case HIGH_PERFORMANCE:
- return Mono.just(highPerformanceBlobStore);
- default:
- throw new RuntimeException("Unknown storage policy: " + storagePolicy);
- }
- }
-
- private boolean isItABigStream(InputStream bufferedData) throws IOException {
- bufferedData.mark(0);
- bufferedData.skip(configuration.getSizeThreshold());
- boolean isItABigStream = bufferedData.read() != -1;
- bufferedData.reset();
- return isItABigStream;
- }
-
- @Override
- public BucketName getDefaultBucketName() {
- Preconditions.checkState(
- lowCostBlobStore.getDefaultBucketName()
- .equals(highPerformanceBlobStore.getDefaultBucketName()),
- "lowCostBlobStore and highPerformanceBlobStore doen't have same defaultBucketName which could lead to " +
- "unexpected result when interact with other APIs");
-
- return lowCostBlobStore.getDefaultBucketName();
- }
-
- @Override
- public Mono<byte[]> readBytes(BucketName bucketName, BlobId blobId) {
- return Mono.defer(() -> Mono.from(highPerformanceBlobStore.readBytes(bucketName, blobId)))
- .onErrorResume(this::logAndReturnEmpty)
- .switchIfEmpty(Mono.defer(() -> Mono.from(lowCostBlobStore.readBytes(bucketName, blobId))));
- }
-
- @Override
- public InputStream read(BucketName bucketName, BlobId blobId) {
- try {
- return highPerformanceBlobStore.read(bucketName, blobId);
- } catch (ObjectNotFoundException e) {
- return lowCostBlobStore.read(bucketName, blobId);
- } catch (Exception e) {
- LOGGER.error("Error reading {} {} in {}, falling back to {}", bucketName, blobId, highPerformanceBlobStore, lowCostBlobStore);
- return lowCostBlobStore.read(bucketName, blobId);
- }
- }
-
- @Override
- public Mono<Void> deleteBucket(BucketName bucketName) {
- return Mono.defer(() -> Mono.from(lowCostBlobStore.deleteBucket(bucketName)))
- .and(highPerformanceBlobStore.deleteBucket(bucketName))
- .onErrorResume(this::logDeleteFailureAndReturnEmpty);
- }
-
- @Override
- public Mono<Void> delete(BucketName bucketName, BlobId blobId) {
- return Mono.defer(() -> Mono.from(lowCostBlobStore.delete(bucketName, blobId)))
- .and(highPerformanceBlobStore.delete(bucketName, blobId))
- .onErrorResume(this::logDeleteFailureAndReturnEmpty);
- }
-
- private <T> Mono<T> logAndReturnEmpty(Throwable throwable) {
- LOGGER.error("error happens from current blob store, fall back to lowCost blob store", throwable);
- return Mono.empty();
- }
-
- private <T> Mono<T> logDeleteFailureAndReturnEmpty(Throwable throwable) {
- LOGGER.error("Cannot delete from either lowCost or highPerformance blob store", throwable);
- return Mono.empty();
- }
-
- @Override
- public String toString() {
- return MoreObjects.toStringHelper(this)
- .add("lowCostBlobStore", lowCostBlobStore)
- .add("highPerformanceBlobStore", highPerformanceBlobStore)
- .toString();
- }
-}
diff --git a/server/blob/blob-union/src/test/java/org/apache/james/blob/union/HybridBlobStoreTest.java b/server/blob/blob-union/src/test/java/org/apache/james/blob/union/HybridBlobStoreTest.java
deleted file mode 100644
index ee36d52..0000000
--- a/server/blob/blob-union/src/test/java/org/apache/james/blob/union/HybridBlobStoreTest.java
+++ /dev/null
@@ -1,536 +0,0 @@
-/****************************************************************
- * Licensed to the Apache Software Foundation (ASF) under one *
- * or more contributor license agreements. See the NOTICE file *
- * distributed with this work for additional information *
- * regarding copyright ownership. The ASF licenses this file *
- * to you under the Apache License, Version 2.0 (the *
- * "License"); you may not use this file except in compliance *
- * with the License. You may obtain a copy of the License at *
- * *
- * http://www.apache.org/licenses/LICENSE-2.0 *
- * *
- * Unless required by applicable law or agreed to in writing, *
- * software distributed under the License is distributed on an *
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
- * KIND, either express or implied. See the License for the *
- * specific language governing permissions and limitations *
- * under the License. *
- ****************************************************************/
-
-package org.apache.james.blob.union;
-
-import static org.apache.james.blob.api.BlobStore.StoragePolicy.HIGH_PERFORMANCE;
-import static org.apache.james.blob.api.BlobStore.StoragePolicy.LOW_COST;
-import static org.apache.james.blob.api.BlobStore.StoragePolicy.SIZE_BASED;
-import static org.assertj.core.api.Assertions.assertThat;
-import static org.assertj.core.api.Assertions.assertThatCode;
-import static org.assertj.core.api.Assertions.assertThatThrownBy;
-
-import java.io.ByteArrayInputStream;
-import java.io.InputStream;
-
-import org.apache.james.blob.api.BlobId;
-import org.apache.james.blob.api.BlobStore;
-import org.apache.james.blob.api.BlobStoreContract;
-import org.apache.james.blob.api.BucketName;
-import org.apache.james.blob.api.HashBlobId;
-import org.apache.james.blob.api.ObjectNotFoundException;
-import org.apache.james.blob.api.ObjectStoreException;
-import org.apache.james.blob.memory.MemoryBlobStoreFactory;
-import org.assertj.core.api.SoftAssertions;
-import org.junit.jupiter.api.BeforeEach;
-import org.junit.jupiter.api.Nested;
-import org.junit.jupiter.api.Test;
-
-import com.github.fge.lambdas.Throwing;
-import com.google.common.base.MoreObjects;
-
-import nl.jqno.equalsverifier.EqualsVerifier;
-import reactor.core.publisher.Mono;
-
-class HybridBlobStoreTest implements BlobStoreContract {
-
- private static class FailingBlobStore implements BlobStore {
- @Override
- public Mono<BlobId> save(BucketName bucketName, InputStream data, StoragePolicy storagePolicy) {
- return Mono.error(new RuntimeException("broken everywhere"));
- }
-
- @Override
- public Mono<BlobId> save(BucketName bucketName, byte[] data, StoragePolicy storagePolicy) {
- return Mono.error(new RuntimeException("broken everywhere"));
- }
-
- @Override
- public Mono<BlobId> save(BucketName bucketName, String data, StoragePolicy storagePolicy) {
- return Mono.error(new RuntimeException("broken everywhere"));
- }
-
- @Override
- public BucketName getDefaultBucketName() {
- return BucketName.DEFAULT;
- }
-
- @Override
- public Mono<byte[]> readBytes(BucketName bucketName, BlobId blobId) {
- return Mono.error(new RuntimeException("broken everywhere"));
- }
-
- @Override
- public InputStream read(BucketName bucketName, BlobId blobId) {
- throw new RuntimeException("broken everywhere");
- }
-
- @Override
- public Mono<Void> deleteBucket(BucketName bucketName) {
- return Mono.error(new RuntimeException("broken everywhere"));
- }
-
- @Override
- public Mono<Void> delete(BucketName bucketName, BlobId blobId) {
- return Mono.error(new RuntimeException("broken everywhere"));
- }
-
- @Override
- public String toString() {
- return MoreObjects.toStringHelper(this)
- .toString();
- }
- }
-
- private static class ThrowingBlobStore implements BlobStore {
-
- @Override
- public Mono<BlobId> save(BucketName bucketName, byte[] data, StoragePolicy storagePolicy) {
- throw new RuntimeException("broken everywhere");
- }
-
- @Override
- public Mono<BlobId> save(BucketName bucketName, String data, StoragePolicy storagePolicy) {
- throw new RuntimeException("broken everywhere");
- }
-
- @Override
- public BucketName getDefaultBucketName() {
- return BucketName.DEFAULT;
- }
-
- @Override
- public Mono<BlobId> save(BucketName bucketName, InputStream data, StoragePolicy storagePolicy) {
- throw new RuntimeException("broken everywhere");
- }
-
- @Override
- public Mono<byte[]> readBytes(BucketName bucketName, BlobId blobId) {
- throw new RuntimeException("broken everywhere");
- }
-
- @Override
- public InputStream read(BucketName bucketName, BlobId blobId) {
- throw new RuntimeException("broken everywhere");
- }
-
- @Override
- public Mono<Void> deleteBucket(BucketName bucketName) {
- return Mono.error(new RuntimeException("broken everywhere"));
- }
-
- @Override
- public Mono<Void> delete(BucketName bucketName, BlobId blobId) {
- return Mono.error(new RuntimeException("broken everywhere"));
- }
-
- @Override
- public String toString() {
- return MoreObjects.toStringHelper(this)
- .toString();
- }
- }
-
- private static final HashBlobId.Factory BLOB_ID_FACTORY = new HashBlobId.Factory();
- private static final String STRING_CONTENT = "blob content";
- private static final byte [] BLOB_CONTENT = STRING_CONTENT.getBytes();
-
- private BlobStore lowCostBlobStore;
- private BlobStore highPerformanceBlobStore;
- private HybridBlobStore hybridBlobStore;
-
- @BeforeEach
- void setup() {
- lowCostBlobStore = MemoryBlobStoreFactory.create(BLOB_ID_FACTORY);
- highPerformanceBlobStore = MemoryBlobStoreFactory.create(BLOB_ID_FACTORY);
- hybridBlobStore = HybridBlobStore.builder()
- .lowCost(lowCostBlobStore)
- .highPerformance(highPerformanceBlobStore)
- .configuration(HybridBlobStore.Configuration.DEFAULT)
- .build();
- }
-
- @Override
- public BlobStore testee() {
- return hybridBlobStore;
- }
-
- @Override
- public BlobId.Factory blobIdFactory() {
- return BLOB_ID_FACTORY;
- }
-
- @Nested
- class StoragePolicyTests {
- @Test
- void saveShouldRelyOnLowCostWhenLowCost() {
- BlobId blobId = hybridBlobStore.save(BucketName.DEFAULT, BLOB_CONTENT, LOW_COST).block();
-
- SoftAssertions.assertSoftly(softly -> {
- softly.assertThat(lowCostBlobStore.read(BucketName.DEFAULT, blobId))
- .hasSameContentAs(new ByteArrayInputStream(BLOB_CONTENT));
- softly.assertThatThrownBy(() -> highPerformanceBlobStore.read(BucketName.DEFAULT, blobId))
- .isInstanceOf(ObjectNotFoundException.class);
- });
- }
-
- @Test
- void saveShouldRelyOnPerformingWhenPerforming() {
- BlobId blobId = hybridBlobStore.save(BucketName.DEFAULT, BLOB_CONTENT, HIGH_PERFORMANCE).block();
-
- SoftAssertions.assertSoftly(softly -> {
- softly.assertThat(highPerformanceBlobStore.read(BucketName.DEFAULT, blobId))
- .hasSameContentAs(new ByteArrayInputStream(BLOB_CONTENT));
- softly.assertThatThrownBy(() -> lowCostBlobStore.read(BucketName.DEFAULT, blobId))
- .isInstanceOf(ObjectNotFoundException.class);
- });
- }
-
- @Test
- void saveShouldRelyOnPerformingWhenSizeBasedAndSmall() {
- BlobId blobId = hybridBlobStore.save(BucketName.DEFAULT, BLOB_CONTENT, SIZE_BASED).block();
-
- SoftAssertions.assertSoftly(softly -> {
- softly.assertThat(highPerformanceBlobStore.read(BucketName.DEFAULT, blobId))
- .hasSameContentAs(new ByteArrayInputStream(BLOB_CONTENT));
- softly.assertThatThrownBy(() -> lowCostBlobStore.read(BucketName.DEFAULT, blobId))
- .isInstanceOf(ObjectNotFoundException.class);
- });
- }
-
- @Test
- void saveShouldRelyOnLowCostWhenSizeBasedAndBig() {
- BlobId blobId = hybridBlobStore.save(BucketName.DEFAULT, TWELVE_MEGABYTES, SIZE_BASED).block();
-
- SoftAssertions.assertSoftly(softly -> {
- softly.assertThat(lowCostBlobStore.read(BucketName.DEFAULT, blobId))
- .satisfies(Throwing.consumer(inputStream -> assertThat(inputStream.read()).isGreaterThan(0)));
- softly.assertThatThrownBy(() -> highPerformanceBlobStore.read(BucketName.DEFAULT, blobId))
- .isInstanceOf(ObjectNotFoundException.class);
- });
- }
-
- @Test
- void saveInputStreamShouldRelyOnLowCostWhenLowCost() {
- BlobId blobId = hybridBlobStore.save(BucketName.DEFAULT, new ByteArrayInputStream(BLOB_CONTENT), LOW_COST).block();
-
- SoftAssertions.assertSoftly(softly -> {
- softly.assertThat(lowCostBlobStore.read(BucketName.DEFAULT, blobId))
- .hasSameContentAs(new ByteArrayInputStream(BLOB_CONTENT));
- softly.assertThatThrownBy(() -> highPerformanceBlobStore.read(BucketName.DEFAULT, blobId))
- .isInstanceOf(ObjectNotFoundException.class);
- });
- }
-
- @Test
- void saveInputStreamShouldRelyOnPerformingWhenPerforming() {
- BlobId blobId = hybridBlobStore.save(BucketName.DEFAULT, new ByteArrayInputStream(BLOB_CONTENT), HIGH_PERFORMANCE).block();
-
- SoftAssertions.assertSoftly(softly -> {
- softly.assertThat(highPerformanceBlobStore.read(BucketName.DEFAULT, blobId))
- .hasSameContentAs(new ByteArrayInputStream(BLOB_CONTENT));
- softly.assertThatThrownBy(() -> lowCostBlobStore.read(BucketName.DEFAULT, blobId))
- .isInstanceOf(ObjectNotFoundException.class);
- });
- }
-
- @Test
- void saveInputStreamShouldRelyOnPerformingWhenSizeBasedAndSmall() {
- BlobId blobId = hybridBlobStore.save(BucketName.DEFAULT, new ByteArrayInputStream(BLOB_CONTENT), SIZE_BASED).block();
-
- SoftAssertions.assertSoftly(softly -> {
- softly.assertThat(highPerformanceBlobStore.read(BucketName.DEFAULT, blobId))
- .hasSameContentAs(new ByteArrayInputStream(BLOB_CONTENT));
- softly.assertThatThrownBy(() -> lowCostBlobStore.read(BucketName.DEFAULT, blobId))
- .isInstanceOf(ObjectNotFoundException.class);
- });
- }
-
- @Test
- void saveInputStreamShouldRelyOnLowCostWhenSizeBasedAndBig() {
- BlobId blobId = hybridBlobStore.save(BucketName.DEFAULT, new ByteArrayInputStream(TWELVE_MEGABYTES), SIZE_BASED).block();
-
- SoftAssertions.assertSoftly(softly -> {
- softly.assertThat(lowCostBlobStore.read(BucketName.DEFAULT, blobId))
- .satisfies(Throwing.consumer(inputStream -> assertThat(inputStream.read()).isGreaterThan(0)));
- softly.assertThatThrownBy(() -> highPerformanceBlobStore.read(BucketName.DEFAULT, blobId))
- .isInstanceOf(ObjectNotFoundException.class);
- });
- }
- }
-
- @Nested
- class LowCostSaveThrowsExceptionDirectly {
- @Test
- void saveShouldFailWhenException() {
- BlobStore highPerformanceBlobStore = MemoryBlobStoreFactory.create(BLOB_ID_FACTORY);
- HybridBlobStore hybridBlobStore = HybridBlobStore.builder()
- .lowCost(new ThrowingBlobStore())
- .highPerformance(highPerformanceBlobStore)
- .configuration(HybridBlobStore.Configuration.DEFAULT)
- .build();
-
- assertThatThrownBy(() -> hybridBlobStore.save(hybridBlobStore.getDefaultBucketName(), BLOB_CONTENT, LOW_COST).block())
- .isInstanceOf(RuntimeException.class);
- }
-
- @Test
- void saveInputStreamShouldFailWhenException() {
- BlobStore highPerformanceBlobStore = MemoryBlobStoreFactory.create(BLOB_ID_FACTORY);
- HybridBlobStore hybridBlobStore = HybridBlobStore.builder()
- .lowCost(new ThrowingBlobStore())
- .highPerformance(highPerformanceBlobStore)
- .configuration(HybridBlobStore.Configuration.DEFAULT)
- .build();
-
- assertThatThrownBy(() -> hybridBlobStore.save(hybridBlobStore.getDefaultBucketName(), new ByteArrayInputStream(BLOB_CONTENT), LOW_COST).block())
- .isInstanceOf(RuntimeException.class);
- }
- }
-
- @Nested
- class LowCostSaveCompletesExceptionally {
-
- @Test
- void saveShouldFailWhenLowCostCompletedExceptionally() {
- BlobStore highPerformanceBlobStore = MemoryBlobStoreFactory.create(BLOB_ID_FACTORY);
- HybridBlobStore hybridBlobStore = HybridBlobStore.builder()
- .lowCost(new FailingBlobStore())
- .highPerformance(highPerformanceBlobStore)
- .configuration(HybridBlobStore.Configuration.DEFAULT)
- .build();
-
- assertThatThrownBy(() -> hybridBlobStore.save(hybridBlobStore.getDefaultBucketName(), BLOB_CONTENT, LOW_COST).block())
- .isInstanceOf(RuntimeException.class);
- }
-
- @Test
- void saveInputStreamShouldFallBackToPerformingWhenLowCostCompletedExceptionally() {
- BlobStore highPerformanceBlobStore = MemoryBlobStoreFactory.create(BLOB_ID_FACTORY);
- HybridBlobStore hybridBlobStore = HybridBlobStore.builder()
- .lowCost(new FailingBlobStore())
- .highPerformance(highPerformanceBlobStore)
- .configuration(HybridBlobStore.Configuration.DEFAULT)
- .build();
-
- assertThatThrownBy(() -> hybridBlobStore.save(hybridBlobStore.getDefaultBucketName(), new ByteArrayInputStream(BLOB_CONTENT), LOW_COST).block())
- .isInstanceOf(RuntimeException.class);
- }
-
- }
-
- @Nested
- class LowCostReadThrowsExceptionDirectly {
-
- @Test
- void readShouldReturnFallbackToPerformingWhenLowCostGotException() {
- BlobStore highPerformanceBlobStore = MemoryBlobStoreFactory.create(BLOB_ID_FACTORY);
- HybridBlobStore hybridBlobStore = HybridBlobStore.builder()
- .lowCost(new ThrowingBlobStore())
- .highPerformance(highPerformanceBlobStore)
- .configuration(HybridBlobStore.Configuration.DEFAULT)
- .build();
- BlobId blobId = Mono.from(highPerformanceBlobStore.save(hybridBlobStore.getDefaultBucketName(), BLOB_CONTENT, LOW_COST)).block();
-
- assertThat(hybridBlobStore.read(hybridBlobStore.getDefaultBucketName(), blobId))
- .hasSameContentAs(new ByteArrayInputStream(BLOB_CONTENT));
- }
-
- @Test
- void readBytesShouldReturnFallbackToPerformingWhenLowCostGotException() {
- BlobStore highPerformanceBlobStore = MemoryBlobStoreFactory.create(BLOB_ID_FACTORY);
-
- HybridBlobStore hybridBlobStore = HybridBlobStore.builder()
- .lowCost(new ThrowingBlobStore())
- .highPerformance(highPerformanceBlobStore)
- .configuration(HybridBlobStore.Configuration.DEFAULT)
- .build();
- BlobId blobId = Mono.from(highPerformanceBlobStore.save(hybridBlobStore.getDefaultBucketName(), BLOB_CONTENT, LOW_COST)).block();
-
- assertThat(hybridBlobStore.readBytes(hybridBlobStore.getDefaultBucketName(), blobId).block())
- .isEqualTo(BLOB_CONTENT);
- }
-
- }
-
- @Nested
- class LowCostReadCompletesExceptionally {
-
- @Test
- void readShouldReturnFallbackToPerformingWhenLowCostCompletedExceptionally() {
- BlobStore highPerformanceBlobStore = MemoryBlobStoreFactory.create(BLOB_ID_FACTORY);
- HybridBlobStore hybridBlobStore = HybridBlobStore.builder()
- .lowCost(new FailingBlobStore())
- .highPerformance(highPerformanceBlobStore)
- .configuration(HybridBlobStore.Configuration.DEFAULT)
- .build();
- BlobId blobId = Mono.from(highPerformanceBlobStore.save(hybridBlobStore.getDefaultBucketName(), BLOB_CONTENT, LOW_COST)).block();
-
- assertThat(hybridBlobStore.read(hybridBlobStore.getDefaultBucketName(), blobId))
- .hasSameContentAs(new ByteArrayInputStream(BLOB_CONTENT));
- }
-
- @Test
- void readBytesShouldReturnFallbackToPerformingWhenLowCostCompletedExceptionally() {
- BlobStore highPerformanceBlobStore = MemoryBlobStoreFactory.create(BLOB_ID_FACTORY);
- HybridBlobStore hybridBlobStore = HybridBlobStore.builder()
- .lowCost(new FailingBlobStore())
- .highPerformance(highPerformanceBlobStore)
- .configuration(HybridBlobStore.Configuration.DEFAULT)
- .build();
- BlobId blobId = Mono.from(highPerformanceBlobStore.save(hybridBlobStore.getDefaultBucketName(), BLOB_CONTENT, LOW_COST)).block();
-
- assertThat(hybridBlobStore.readBytes(hybridBlobStore.getDefaultBucketName(), blobId).block())
- .isEqualTo(BLOB_CONTENT);
- }
- }
-
- @Test
- void readShouldReturnFromLowCostWhenAvailable() {
- BlobId blobId = Mono.from(lowCostBlobStore.save(lowCostBlobStore.getDefaultBucketName(), BLOB_CONTENT, LOW_COST)).block();
-
- assertThat(hybridBlobStore.read(hybridBlobStore.getDefaultBucketName(), blobId))
- .hasSameContentAs(new ByteArrayInputStream(BLOB_CONTENT));
- }
-
- @Test
- void readShouldReturnFromPerformingWhenLowCostNotAvailable() {
- BlobId blobId = Mono.from(highPerformanceBlobStore.save(hybridBlobStore.getDefaultBucketName(), BLOB_CONTENT, LOW_COST)).block();
-
- assertThat(hybridBlobStore.read(hybridBlobStore.getDefaultBucketName(), blobId))
- .hasSameContentAs(new ByteArrayInputStream(BLOB_CONTENT));
- }
-
- @Test
- void readBytesShouldReturnFromLowCostWhenAvailable() {
- BlobId blobId = Mono.from(lowCostBlobStore.save(lowCostBlobStore.getDefaultBucketName(), BLOB_CONTENT, LOW_COST)).block();
-
- assertThat(hybridBlobStore.readBytes(lowCostBlobStore.getDefaultBucketName(), blobId).block())
- .isEqualTo(BLOB_CONTENT);
- }
-
- @Test
- void readBytesShouldReturnFromPerformingWhenLowCostNotAvailable() {
- BlobId blobId = Mono.from(highPerformanceBlobStore.save(hybridBlobStore.getDefaultBucketName(), BLOB_CONTENT, LOW_COST)).block();
-
- assertThat(hybridBlobStore.readBytes(hybridBlobStore.getDefaultBucketName(), blobId).block())
- .isEqualTo(BLOB_CONTENT);
- }
-
- @Test
- void deleteBucketShouldDeleteBothLowCostAndPerformingBuckets() {
- BlobId blobId1 = Mono.from(highPerformanceBlobStore.save(BucketName.DEFAULT, BLOB_CONTENT, LOW_COST)).block();
- BlobId blobId2 = Mono.from(lowCostBlobStore.save(BucketName.DEFAULT, BLOB_CONTENT, LOW_COST)).block();
-
- hybridBlobStore.deleteBucket(BucketName.DEFAULT).block();
-
- assertThatThrownBy(() -> Mono.from(highPerformanceBlobStore.readBytes(BucketName.DEFAULT, blobId1)).block())
- .isInstanceOf(ObjectStoreException.class);
- assertThatThrownBy(() -> Mono.from(lowCostBlobStore.readBytes(BucketName.DEFAULT, blobId2)).block())
- .isInstanceOf(ObjectStoreException.class);
- }
-
- @Test
- void deleteBucketShouldDeleteLowCostBucketEvenWhenPerformingDoesNotExist() {
- BlobId blobId = Mono.from(lowCostBlobStore.save(BucketName.DEFAULT, BLOB_CONTENT, LOW_COST)).block();
-
- hybridBlobStore.deleteBucket(BucketName.DEFAULT).block();
-
- assertThatThrownBy(() -> Mono.from(lowCostBlobStore.readBytes(BucketName.DEFAULT, blobId)).block())
- .isInstanceOf(ObjectStoreException.class);
- }
-
- @Test
- void deleteBucketShouldDeletePerformingBucketEvenWhenLowCostDoesNotExist() {
- BlobId blobId = Mono.from(highPerformanceBlobStore.save(BucketName.DEFAULT, BLOB_CONTENT, LOW_COST)).block();
-
- hybridBlobStore.deleteBucket(BucketName.DEFAULT).block();
-
- assertThatThrownBy(() -> Mono.from(highPerformanceBlobStore.readBytes(BucketName.DEFAULT, blobId)).block())
- .isInstanceOf(ObjectStoreException.class);
- }
-
- @Test
- void deleteBucketShouldNotThrowWhenLowCostAndPerformingBucketsDoNotExist() {
- assertThatCode(() -> hybridBlobStore.deleteBucket(BucketName.DEFAULT).block())
- .doesNotThrowAnyException();
- }
-
- @Test
- void getDefaultBucketNameShouldThrowWhenBlobStoreDontShareTheSameDefaultBucketName() {
- lowCostBlobStore = MemoryBlobStoreFactory.create(BLOB_ID_FACTORY, BucketName.of("lowCost"));
- highPerformanceBlobStore = MemoryBlobStoreFactory.create(BLOB_ID_FACTORY, BucketName.of("highPerformance"));
- hybridBlobStore = HybridBlobStore.builder()
- .lowCost(lowCostBlobStore)
- .highPerformance(highPerformanceBlobStore)
- .configuration(HybridBlobStore.Configuration.DEFAULT)
- .build();
-
- assertThatThrownBy(() -> hybridBlobStore.getDefaultBucketName())
- .isInstanceOf(IllegalStateException.class);
- }
-
- @Test
- void deleteShouldDeleteBothLowCostAndPerformingBlob() {
- BlobId blobId1 = hybridBlobStore.save(BucketName.DEFAULT, BLOB_CONTENT, LOW_COST).block();
- BlobId blobId2 = hybridBlobStore.save(BucketName.DEFAULT, BLOB_CONTENT, HIGH_PERFORMANCE).block();
-
- hybridBlobStore.delete(BucketName.DEFAULT, blobId1).block();
-
- assertThatThrownBy(() -> Mono.from(highPerformanceBlobStore.readBytes(BucketName.DEFAULT, blobId1)).block())
- .isInstanceOf(ObjectStoreException.class);
- assertThatThrownBy(() -> Mono.from(lowCostBlobStore.readBytes(BucketName.DEFAULT, blobId2)).block())
- .isInstanceOf(ObjectStoreException.class);
- }
-
- @Test
- void deleteShouldDeleteLowCostBlobEvenWhenPerformingDoesNotExist() {
- BlobId blobId = Mono.from(lowCostBlobStore.save(BucketName.DEFAULT, BLOB_CONTENT, LOW_COST)).block();
-
- hybridBlobStore.delete(BucketName.DEFAULT, blobId).block();
-
- assertThatThrownBy(() -> Mono.from(lowCostBlobStore.readBytes(BucketName.DEFAULT, blobId)).block())
- .isInstanceOf(ObjectStoreException.class);
- }
-
- @Test
- void deleteShouldDeletePerformingBlobEvenWhenLowCostDoesNotExist() {
- BlobId blobId = Mono.from(highPerformanceBlobStore.save(BucketName.DEFAULT, BLOB_CONTENT, LOW_COST)).block();
-
- hybridBlobStore.delete(BucketName.DEFAULT, blobId).block();
-
- assertThatThrownBy(() -> Mono.from(highPerformanceBlobStore.readBytes(BucketName.DEFAULT, blobId)).block())
- .isInstanceOf(ObjectStoreException.class);
- }
-
- @Test
- void deleteShouldNotThrowWhenLowCostAndPerformingBlobsDoNotExist() {
- assertThatCode(() -> hybridBlobStore.delete(BucketName.DEFAULT, blobIdFactory().randomId()).block())
- .doesNotThrowAnyException();
- }
-
- @Nested
- class ConfigurationTest {
- @Test
- void shouldMatchBeanContract() {
- EqualsVerifier.forClass(HybridBlobStore.Configuration.class)
- .verify();
- }
- }
-}
diff --git a/server/blob/pom.xml b/server/blob/pom.xml
index f62e027..a4d6742 100644
--- a/server/blob/pom.xml
+++ b/server/blob/pom.xml
@@ -42,7 +42,6 @@
<module>blob-gc</module>
<module>blob-memory</module>
<module>blob-objectstorage</module>
- <module>blob-union</module>
<module>mail-store</module>
</modules>
diff --git a/server/container/guice/cassandra-rabbitmq-guice/pom.xml b/server/container/guice/cassandra-rabbitmq-guice/pom.xml
index 6edd29c..c0fefec 100644
--- a/server/container/guice/cassandra-rabbitmq-guice/pom.xml
+++ b/server/container/guice/cassandra-rabbitmq-guice/pom.xml
@@ -128,10 +128,6 @@
</dependency>
<dependency>
<groupId>${james.groupId}</groupId>
- <artifactId>blob-union</artifactId>
- </dependency>
- <dependency>
- <groupId>${james.groupId}</groupId>
<artifactId>james-server-cassandra-guice</artifactId>
</dependency>
<dependency>
diff --git a/server/container/guice/cassandra-rabbitmq-guice/src/main/java/org/apache/james/modules/blobstore/BlobStoreChoosingModule.java b/server/container/guice/cassandra-rabbitmq-guice/src/main/java/org/apache/james/modules/blobstore/BlobStoreChoosingModule.java
index 5b30d21..99e9ea2 100644
--- a/server/container/guice/cassandra-rabbitmq-guice/src/main/java/org/apache/james/modules/blobstore/BlobStoreChoosingModule.java
+++ b/server/container/guice/cassandra-rabbitmq-guice/src/main/java/org/apache/james/modules/blobstore/BlobStoreChoosingModule.java
@@ -19,22 +19,11 @@
package org.apache.james.modules.blobstore;
-import java.io.FileNotFoundException;
-
-import javax.inject.Singleton;
-
-import org.apache.commons.configuration2.Configuration;
-import org.apache.commons.configuration2.ex.ConfigurationException;
import org.apache.james.backends.cassandra.components.CassandraModule;
import org.apache.james.blob.cassandra.CassandraBlobModule;
-import org.apache.james.blob.union.HybridBlobStore;
-import org.apache.james.modules.mailbox.ConfigurationComponent;
import org.apache.james.modules.objectstorage.ObjectStorageDependenciesModule;
-import org.apache.james.utils.PropertiesProvider;
-import com.google.common.annotations.VisibleForTesting;
import com.google.inject.AbstractModule;
-import com.google.inject.Provides;
import com.google.inject.multibindings.Multibinder;
public class BlobStoreChoosingModule extends AbstractModule {
@@ -45,16 +34,4 @@ public class BlobStoreChoosingModule extends AbstractModule {
Multibinder<CassandraModule> cassandraDataDefinitions = Multibinder.newSetBinder(binder(), CassandraModule.class);
cassandraDataDefinitions.addBinding().toInstance(CassandraBlobModule.MODULE);
}
-
- @Provides
- @Singleton
- @VisibleForTesting
- HybridBlobStore.Configuration providesHybridBlobStoreConfiguration(PropertiesProvider propertiesProvider) {
- try {
- Configuration configuration = propertiesProvider.getConfigurations(ConfigurationComponent.NAMES);
- return HybridBlobStore.Configuration.from(configuration);
- } catch (FileNotFoundException | ConfigurationException e) {
- return HybridBlobStore.Configuration.DEFAULT;
- }
- }
}
diff --git a/server/container/guice/cassandra-rabbitmq-guice/src/main/java/org/apache/james/modules/blobstore/BlobStoreConfiguration.java b/server/container/guice/cassandra-rabbitmq-guice/src/main/java/org/apache/james/modules/blobstore/BlobStoreConfiguration.java
index c6044fa..789813b 100644
--- a/server/container/guice/cassandra-rabbitmq-guice/src/main/java/org/apache/james/modules/blobstore/BlobStoreConfiguration.java
+++ b/server/container/guice/cassandra-rabbitmq-guice/src/main/java/org/apache/james/modules/blobstore/BlobStoreConfiguration.java
@@ -57,8 +57,7 @@ public class BlobStoreConfiguration {
public enum BlobStoreImplName {
CASSANDRA("cassandra"),
- OBJECTSTORAGE("objectstorage"),
- HYBRID("hybrid");
+ OBJECTSTORAGE("objectstorage");
static String supportedImplNames() {
return Stream.of(BlobStoreImplName.values())
@@ -127,10 +126,6 @@ public class BlobStoreConfiguration {
return new CacheChoice(BlobStoreImplName.OBJECTSTORAGE);
}
- public static BlobStoreConfiguration hybrid() {
- return new BlobStoreConfiguration(BlobStoreImplName.HYBRID, !CACHE_ENABLED);
- }
-
private final BlobStoreImplName implementation;
private final boolean cacheEnabled;
diff --git a/server/container/guice/cassandra-rabbitmq-guice/src/main/java/org/apache/james/modules/blobstore/BlobStoreModulesChooser.java b/server/container/guice/cassandra-rabbitmq-guice/src/main/java/org/apache/james/modules/blobstore/BlobStoreModulesChooser.java
index b38ea7a..d790968 100644
--- a/server/container/guice/cassandra-rabbitmq-guice/src/main/java/org/apache/james/modules/blobstore/BlobStoreModulesChooser.java
+++ b/server/container/guice/cassandra-rabbitmq-guice/src/main/java/org/apache/james/modules/blobstore/BlobStoreModulesChooser.java
@@ -19,29 +19,21 @@
package org.apache.james.modules.blobstore;
-import java.io.FileNotFoundException;
import java.util.List;
-import javax.inject.Named;
-import javax.inject.Singleton;
-
-import org.apache.commons.configuration2.Configuration;
-import org.apache.commons.configuration2.ex.ConfigurationException;
import org.apache.james.blob.api.BlobStore;
-import org.apache.james.blob.cassandra.CassandraBlobStore;
+import org.apache.james.blob.api.DumbBlobStore;
+import org.apache.james.blob.cassandra.CassandraDumbBlobStore;
import org.apache.james.blob.cassandra.cache.CachedBlobStore;
import org.apache.james.blob.objectstorage.ObjectStorageBlobStore;
-import org.apache.james.blob.union.HybridBlobStore;
import org.apache.james.modules.mailbox.CassandraBlobStoreDependenciesModule;
-import org.apache.james.modules.mailbox.ConfigurationComponent;
import org.apache.james.modules.objectstorage.ObjectStorageDependenciesModule;
-import org.apache.james.utils.PropertiesProvider;
+import org.apache.james.server.blob.deduplication.DeDuplicationBlobStore;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableList;
import com.google.inject.AbstractModule;
import com.google.inject.Module;
-import com.google.inject.Provides;
import com.google.inject.name.Names;
public class BlobStoreModulesChooser {
@@ -68,39 +60,6 @@ public class BlobStoreModulesChooser {
}
}
- static class HybridDeclarationModule extends AbstractModule {
- @Override
- protected void configure() {
- install(new ObjectStorageDependenciesModule());
- install(new CassandraBlobStoreDependenciesModule());
- }
-
- @Provides
- @Singleton
- @VisibleForTesting
- HybridBlobStore.Configuration providesHybridBlobStoreConfiguration(PropertiesProvider propertiesProvider) {
- try {
- Configuration configuration = propertiesProvider.getConfigurations(ConfigurationComponent.NAMES);
- return HybridBlobStore.Configuration.from(configuration);
- } catch (FileNotFoundException | ConfigurationException e) {
- return HybridBlobStore.Configuration.DEFAULT;
- }
- }
-
- @Provides
- @Named(CachedBlobStore.BACKEND)
- @Singleton
- BlobStore providesHybridBlobStore(HybridBlobStore.Configuration hybridBlobStoreConfiguration,
- CassandraBlobStore cassandraBlobStore,
- ObjectStorageBlobStore objectStorageBlobStore) {
- return HybridBlobStore.builder()
- .lowCost(objectStorageBlobStore)
- .highPerformance(cassandraBlobStore)
- .configuration(hybridBlobStoreConfiguration)
- .build();
- }
- }
-
@VisibleForTesting
public static List<Module> chooseModules(BlobStoreConfiguration choosingConfiguration) {
switch (choosingConfiguration.getImplementation()) {
@@ -108,8 +67,6 @@ public class BlobStoreModulesChooser {
return ImmutableList.of(new CassandraDeclarationModule());
case OBJECTSTORAGE:
return ImmutableList.of(new ObjectStorageDeclarationModule());
- case HYBRID:
- return ImmutableList.of(new HybridDeclarationModule());
default:
throw new RuntimeException("Unsuported blobStore implementation " + choosingConfiguration.getImplementation());
}
diff --git a/server/container/guice/cassandra-rabbitmq-guice/src/test/java/org/apache/james/modules/blobstore/BlobStoreConfigurationTest.java b/server/container/guice/cassandra-rabbitmq-guice/src/test/java/org/apache/james/modules/blobstore/BlobStoreConfigurationTest.java
index a1ae0c7..e870fb7 100644
--- a/server/container/guice/cassandra-rabbitmq-guice/src/test/java/org/apache/james/modules/blobstore/BlobStoreConfigurationTest.java
+++ b/server/container/guice/cassandra-rabbitmq-guice/src/test/java/org/apache/james/modules/blobstore/BlobStoreConfigurationTest.java
@@ -34,7 +34,6 @@ class BlobStoreConfigurationTest {
private static final String OBJECT_STORAGE = "objectstorage";
private static final String CASSANDRA = "cassandra";
- private static final String HYBRID = "hybrid";
@Test
void shouldMatchBeanContract() {
@@ -101,18 +100,6 @@ class BlobStoreConfigurationTest {
}
@Test
- void provideChoosingConfigurationShouldReturnHybridConfigurationWhenConfigurationImplIsHybrid() throws Exception {
- PropertiesConfiguration configuration = new PropertiesConfiguration();
- configuration.addProperty("implementation", BlobStoreConfiguration.BlobStoreImplName.HYBRID.getName());
- FakePropertiesProvider propertyProvider = FakePropertiesProvider.builder()
- .register(ConfigurationComponent.NAME, configuration)
- .build();
-
- assertThat(parse(propertyProvider))
- .isEqualTo(BlobStoreConfiguration.hybrid());
- }
-
- @Test
void provideChoosingConfigurationShouldReturnCassandraFactoryWhenConfigurationImplIsCassandra() throws Exception {
PropertiesConfiguration configuration = new PropertiesConfiguration();
configuration.addProperty("implementation", BlobStoreConfiguration.BlobStoreImplName.CASSANDRA.getName());
@@ -131,7 +118,7 @@ class BlobStoreConfigurationTest {
assertThatThrownBy(() -> BlobStoreConfiguration.from(configuration))
.isInstanceOf(IllegalStateException.class)
- .hasMessage("implementation property is missing please use one of supported values in: cassandra, objectstorage, hybrid");
+ .hasMessage("implementation property is missing please use one of supported values in: cassandra, objectstorage");
}
@Test
@@ -141,7 +128,7 @@ class BlobStoreConfigurationTest {
assertThatThrownBy(() -> BlobStoreConfiguration.from(configuration))
.isInstanceOf(IllegalStateException.class)
- .hasMessage("implementation property is missing please use one of supported values in: cassandra, objectstorage, hybrid");
+ .hasMessage("implementation property is missing please use one of supported values in: cassandra, objectstorage");
}
@Test
@@ -151,7 +138,7 @@ class BlobStoreConfigurationTest {
assertThatThrownBy(() -> BlobStoreConfiguration.from(configuration))
.isInstanceOf(IllegalStateException.class)
- .hasMessage("implementation property is missing please use one of supported values in: cassandra, objectstorage, hybrid");
+ .hasMessage("implementation property is missing please use one of supported values in: cassandra, objectstorage");
}
@Test
@@ -161,7 +148,7 @@ class BlobStoreConfigurationTest {
assertThatThrownBy(() -> BlobStoreConfiguration.from(configuration))
.isInstanceOf(IllegalArgumentException.class)
- .hasMessage("un_supported is not a valid name of BlobStores, please use one of supported values in: cassandra, objectstorage, hybrid");
+ .hasMessage("un_supported is not a valid name of BlobStores, please use one of supported values in: cassandra, objectstorage");
}
@Test
@@ -177,18 +164,6 @@ class BlobStoreConfigurationTest {
}
@Test
- void fromShouldReturnConfigurationWhenBlobStoreImplIsUnion() {
- PropertiesConfiguration configuration = new PropertiesConfiguration();
- configuration.addProperty("implementation", HYBRID);
-
- assertThat(
- BlobStoreConfiguration.from(configuration)
- .getImplementation()
- .getName())
- .isEqualTo(HYBRID);
- }
-
- @Test
void fromShouldReturnConfigurationWhenBlobStoreImplIsObjectStorage() {
PropertiesConfiguration configuration = new PropertiesConfiguration();
configuration.addProperty("implementation", OBJECT_STORAGE);
diff --git a/server/container/guice/cassandra-rabbitmq-guice/src/test/java/org/apache/james/modules/blobstore/BlobStoreModulesChooserTest.java b/server/container/guice/cassandra-rabbitmq-guice/src/test/java/org/apache/james/modules/blobstore/BlobStoreModulesChooserTest.java
index 2b77046..da99509 100644
--- a/server/container/guice/cassandra-rabbitmq-guice/src/test/java/org/apache/james/modules/blobstore/BlobStoreModulesChooserTest.java
+++ b/server/container/guice/cassandra-rabbitmq-guice/src/test/java/org/apache/james/modules/blobstore/BlobStoreModulesChooserTest.java
@@ -19,68 +19,11 @@
package org.apache.james.modules.blobstore;
-import static org.apache.james.modules.blobstore.BlobStoreModulesChooser.HybridDeclarationModule;
import static org.assertj.core.api.Assertions.assertThat;
-import static org.assertj.core.api.Assertions.assertThatThrownBy;
-import org.apache.commons.configuration2.PropertiesConfiguration;
-import org.apache.james.FakePropertiesProvider;
-import org.apache.james.blob.union.HybridBlobStore;
-import org.apache.james.modules.mailbox.ConfigurationComponent;
import org.junit.jupiter.api.Test;
class BlobStoreModulesChooserTest {
- @Test
- void providesHybridBlobStoreConfigurationShouldThrowWhenNegative() {
- HybridDeclarationModule module = new HybridDeclarationModule();
- PropertiesConfiguration configuration = new PropertiesConfiguration();
- configuration.addProperty("hybrid.size.threshold", -1);
- FakePropertiesProvider propertyProvider = FakePropertiesProvider.builder()
- .register(ConfigurationComponent.NAME, configuration)
- .build();
-
- assertThatThrownBy(() -> module.providesHybridBlobStoreConfiguration(propertyProvider))
- .isInstanceOf(IllegalArgumentException.class);
- }
-
- @Test
- void providesHybridBlobStoreConfigurationShouldNotThrowWhenZero() {
- HybridDeclarationModule module = new HybridDeclarationModule();
- PropertiesConfiguration configuration = new PropertiesConfiguration();
- configuration.addProperty("hybrid.size.threshold", 0);
- FakePropertiesProvider propertyProvider = FakePropertiesProvider.builder()
- .register(ConfigurationComponent.NAME, configuration)
- .build();
-
- assertThat(module.providesHybridBlobStoreConfiguration(propertyProvider))
- .isEqualTo(new HybridBlobStore.Configuration(0));
- }
-
- @Test
- void providesHybridBlobStoreConfigurationShouldReturnConfiguration() {
- HybridDeclarationModule module = new HybridDeclarationModule();
- PropertiesConfiguration configuration = new PropertiesConfiguration();
- configuration.addProperty("hybrid.size.threshold", 36);
- FakePropertiesProvider propertyProvider = FakePropertiesProvider.builder()
- .register(ConfigurationComponent.NAME, configuration)
- .build();
-
- assertThat(module.providesHybridBlobStoreConfiguration(propertyProvider))
- .isEqualTo(new HybridBlobStore.Configuration(36));
- }
-
- @Test
- void providesHybridBlobStoreConfigurationShouldReturnConfigurationWhenLegacyFile() {
- HybridDeclarationModule module = new HybridDeclarationModule();
- PropertiesConfiguration configuration = new PropertiesConfiguration();
- configuration.addProperty("hybrid.size.threshold", 36);
- FakePropertiesProvider propertyProvider = FakePropertiesProvider.builder()
- .register(ConfigurationComponent.LEGACY, configuration)
- .build();
-
- assertThat(module.providesHybridBlobStoreConfiguration(propertyProvider))
- .isEqualTo(new HybridBlobStore.Configuration(36));
- }
@Test
void provideBlobStoreShouldReturnObjectStoreBlobStoreWhenObjectStoreConfigured() {
@@ -95,11 +38,4 @@ class BlobStoreModulesChooserTest {
.first()
.isInstanceOf(BlobStoreModulesChooser.CassandraDeclarationModule.class);
}
-
- @Test
- void provideBlobStoreShouldReturnHybridBlobStoreWhenHybridConfigured() {
- assertThat(BlobStoreModulesChooser.chooseModules(BlobStoreConfiguration.hybrid()))
- .first()
- .isInstanceOf(BlobStoreModulesChooser.HybridDeclarationModule.class);
- }
}
\ No newline at end of file
diff --git a/src/site/xdoc/server/config-blobstore.xml b/src/site/xdoc/server/config-blobstore.xml
index 389aee8..299d764 100644
--- a/src/site/xdoc/server/config-blobstore.xml
+++ b/src/site/xdoc/server/config-blobstore.xml
@@ -46,8 +46,6 @@
<dt><strong>implementation</strong></dt>
<dd>cassandra: use cassandra based BlobStore</dd>
<dd>objectstorage: use Swift/AWS S3 based BlobStore</dd>
- <dd>hybrid (deprecated, use CachedBlobStore instead): Using both objectstorage for unfrequently read or big blobs & cassandra for small, often read blobs.
- Introduced to fasten small blob access, its usage could be compared to a cache, but with a sub-optimal implementation (no eviction, default replication factor, no circuit breaking).</dd>
</dl>
@@ -79,13 +77,6 @@
</dl>
</subsection>
- <subsection name="Hybrid BlobStore size threshold (deprecated)">
- <dl>
- <dt><strong>hybrid.size.threshold</strong></dt>
- <dd>DEFAULT: 32768 bytes (32KB), must be positive. Size threshold for considering a blob as 'big', causing it to be saved in the low cost blobStore.</dd>
- </dl>
- </subsection>
-
<subsection name="ObjectStorage BlobStore Codec Configuration">
<dl>
<dt><strong>objectstorage.payload.codec</strong></dt>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org