You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@james.apache.org by rc...@apache.org on 2023/09/18 07:22:13 UTC
[james-project] branch master updated: JAMES-3940 Set up TTL on mailbox_change and email_change tables (#1729)
This is an automated email from the ASF dual-hosted git repository.
rcordier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
The following commit(s) were added to refs/heads/master by this push:
new 21d6b95152 JAMES-3940 Set up TTL on mailbox_change and email_change tables (#1729)
21d6b95152 is described below
commit 21d6b95152fa2e745a691c63540855589a005a12
Author: Trần Hồng Quân <55...@users.noreply.github.com>
AuthorDate: Mon Sep 18 14:22:08 2023 +0700
JAMES-3940 Set up TTL on mailbox_change and email_change tables (#1729)
---
.../modules/ROOT/pages/configure/cassandra.adoc | 6 +
.../modules/mailbox/CassandraMailboxModule.java | 17 +++
.../change/CassandraChangesConfiguration.java | 122 +++++++++++++++++++++
.../cassandra/change/EmailChangeRepositoryDAO.java | 12 +-
.../change/MailboxChangeRepositoryDAO.java | 12 +-
.../change/CassandraChangesConfigurationTest.java | 83 ++++++++++++++
.../change/CassandraEmailChangeRepositoryTest.java | 59 +++++++++-
.../CassandraMailboxChangeRepositoryTest.java | 61 ++++++++++-
src/site/xdoc/server/config-cassandra.xml | 6 +
upgrade-instructions.md | 19 ++++
10 files changed, 391 insertions(+), 6 deletions(-)
diff --git a/server/apps/distributed-app/docs/modules/ROOT/pages/configure/cassandra.adoc b/server/apps/distributed-app/docs/modules/ROOT/pages/configure/cassandra.adoc
index 0f26f36efa..0bdb101ace 100644
--- a/server/apps/distributed-app/docs/modules/ROOT/pages/configure/cassandra.adoc
+++ b/server/apps/distributed-app/docs/modules/ROOT/pages/configure/cassandra.adoc
@@ -152,4 +152,10 @@ by turning it off. Reads performed as part of write transaction are also perform
have access to the mailboxes they own. ACLs can represent a high volume of requests. If you do not propose mailbox sharing
features to your users, you can consider disabling them in order to improve performance.
+| email.change.ttl
+| Optional. Duration, default to 60 days. Cassandra Time-to-live for Email change records. Setting time-to-live to zero means refusing to use time-to-live on email changes.
+
+| mailbox.change.ttl
+| Optional. Duration, default to 60 days. Cassandra Time-to-live for Mailbox change records. Setting time-to-live to zero means refusing to use time-to-live on mailbox changes.
+
|===
\ No newline at end of file
diff --git a/server/container/guice/cassandra/src/main/java/org/apache/james/modules/mailbox/CassandraMailboxModule.java b/server/container/guice/cassandra/src/main/java/org/apache/james/modules/mailbox/CassandraMailboxModule.java
index bd43acf845..63260e5a11 100644
--- a/server/container/guice/cassandra/src/main/java/org/apache/james/modules/mailbox/CassandraMailboxModule.java
+++ b/server/container/guice/cassandra/src/main/java/org/apache/james/modules/mailbox/CassandraMailboxModule.java
@@ -20,8 +20,12 @@ package org.apache.james.modules.mailbox;
import static org.apache.james.modules.Names.MAILBOXMANAGER_NAME;
+import java.io.FileNotFoundException;
+
import javax.inject.Singleton;
+import org.apache.commons.configuration2.Configuration;
+import org.apache.commons.configuration2.ex.ConfigurationException;
import org.apache.james.adapter.mailbox.ACLUsernameChangeTaskStep;
import org.apache.james.adapter.mailbox.MailboxUserDeletionTaskStep;
import org.apache.james.adapter.mailbox.MailboxUsernameChangeTaskStep;
@@ -41,6 +45,7 @@ import org.apache.james.jmap.api.change.EmailChangeRepository;
import org.apache.james.jmap.api.change.Limit;
import org.apache.james.jmap.api.change.MailboxChangeRepository;
import org.apache.james.jmap.api.change.State;
+import org.apache.james.jmap.cassandra.change.CassandraChangesConfiguration;
import org.apache.james.jmap.cassandra.change.CassandraEmailChangeRepository;
import org.apache.james.jmap.cassandra.change.CassandraMailboxChangeRepository;
import org.apache.james.jmap.cassandra.change.CassandraStateFactory;
@@ -118,6 +123,7 @@ import org.apache.james.mailbox.store.user.SubscriptionMapperFactory;
import org.apache.james.user.api.DeleteUserDataTaskStep;
import org.apache.james.user.api.UsernameChangeTaskStep;
import org.apache.james.utils.MailboxManagerDefinition;
+import org.apache.james.utils.PropertiesProvider;
import org.apache.mailbox.tools.indexer.MessageIdReIndexerImpl;
import org.apache.mailbox.tools.indexer.ReIndexerImpl;
@@ -272,4 +278,15 @@ public class CassandraMailboxModule extends AbstractModule {
super("cassandra-mailboxmanager", manager);
}
}
+
+ @Provides
+ @Singleton
+ CassandraChangesConfiguration providesCassandraChangesConfiguration(PropertiesProvider propertiesProvider) throws ConfigurationException {
+ try {
+ Configuration configuration = propertiesProvider.getConfiguration("cassandra");
+ return CassandraChangesConfiguration.from(configuration);
+ } catch (FileNotFoundException e) {
+ return CassandraChangesConfiguration.DEFAULT;
+ }
+ }
}
diff --git a/server/data/data-jmap-cassandra/src/main/java/org/apache/james/jmap/cassandra/change/CassandraChangesConfiguration.java b/server/data/data-jmap-cassandra/src/main/java/org/apache/james/jmap/cassandra/change/CassandraChangesConfiguration.java
new file mode 100644
index 0000000000..6670be8831
--- /dev/null
+++ b/server/data/data-jmap-cassandra/src/main/java/org/apache/james/jmap/cassandra/change/CassandraChangesConfiguration.java
@@ -0,0 +1,122 @@
+/****************************************************************
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information *
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the *
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the *
+ * specific language governing permissions and limitations *
+ * under the License. *
+ ****************************************************************/
+
+package org.apache.james.jmap.cassandra.change;
+
+import java.time.Duration;
+import java.time.temporal.ChronoUnit;
+import java.util.Objects;
+import java.util.Optional;
+
+import org.apache.commons.configuration2.Configuration;
+import org.apache.james.util.DurationParser;
+
+import com.google.common.base.Preconditions;
+
+public class CassandraChangesConfiguration {
+ public static final Duration DEFAULT_TTL = Duration.ofDays(60);
+ public static final CassandraChangesConfiguration DEFAULT = builder().build();
+
+ public static class Builder {
+ private Optional<Duration> emailChangeTtl = Optional.empty();
+ private Optional<Duration> mailboxChangeTtl = Optional.empty();
+
+ public Builder emailChangeTtl(Duration emailChangeTtl) {
+ this.emailChangeTtl = Optional.of(emailChangeTtl);
+ return this;
+ }
+
+ public Builder emailChangeTtl(Optional<Duration> emailChangeTtl) {
+ emailChangeTtl.ifPresent(this::emailChangeTtl);
+ return this;
+ }
+
+ public Builder mailboxChangeTtl(Duration mailboxChangeTtl) {
+ this.mailboxChangeTtl = Optional.of(mailboxChangeTtl);
+ return this;
+ }
+
+ public Builder mailboxChangeTtl(Optional<Duration> mailboxChangeTtl) {
+ mailboxChangeTtl.ifPresent(this::mailboxChangeTtl);
+ return this;
+ }
+
+ public CassandraChangesConfiguration build() {
+ return new CassandraChangesConfiguration(emailChangeTtl.orElse(DEFAULT_TTL),
+ mailboxChangeTtl.orElse(DEFAULT_TTL));
+ }
+ }
+
+ public static Builder builder() {
+ return new Builder();
+ }
+
+ public static CassandraChangesConfiguration from(Configuration configuration) {
+ Optional<Duration> emailChangeTtl = Optional.ofNullable(configuration.getString("email.change.ttl", null))
+ .map(value -> DurationParser.parse(value, ChronoUnit.SECONDS));
+
+ Optional<Duration> mailboxChangeTtl = Optional.ofNullable(configuration.getString("mailbox.change.ttl", null))
+ .map(value -> DurationParser.parse(value, ChronoUnit.SECONDS));
+
+ return new Builder()
+ .emailChangeTtl(emailChangeTtl)
+ .mailboxChangeTtl(mailboxChangeTtl)
+ .build();
+ }
+
+ private final Duration emailChangeTtl;
+ private final Duration mailboxChangeTtl;
+
+ private CassandraChangesConfiguration(Duration emailChangeTtl, Duration mailboxChangeTtl) {
+ Preconditions.checkArgument(emailChangeTtl.getSeconds() >= 0, "'TTL' needs to be positive");
+ Preconditions.checkArgument(emailChangeTtl.getSeconds() < Integer.MAX_VALUE,
+ "'TTL' must not greater than %s sec", Integer.MAX_VALUE);
+
+ Preconditions.checkArgument(mailboxChangeTtl.getSeconds() >= 0, "'TTL' needs to be positive");
+ Preconditions.checkArgument(mailboxChangeTtl.getSeconds() < Integer.MAX_VALUE,
+ "'TTL' must not greater than %s sec", Integer.MAX_VALUE);
+
+ this.emailChangeTtl = emailChangeTtl;
+ this.mailboxChangeTtl = mailboxChangeTtl;
+ }
+
+ public Duration getEmailChangeTtl() {
+ return emailChangeTtl;
+ }
+
+ public Duration getMailboxChangeTtl() {
+ return mailboxChangeTtl;
+ }
+
+ @Override
+ public final boolean equals(Object o) {
+ if (o instanceof CassandraChangesConfiguration) {
+ CassandraChangesConfiguration that = (CassandraChangesConfiguration) o;
+
+ return Objects.equals(this.emailChangeTtl, that.emailChangeTtl)
+ && Objects.equals(this.mailboxChangeTtl, that.mailboxChangeTtl);
+ }
+ return false;
+ }
+
+ @Override
+ public final int hashCode() {
+ return Objects.hash(emailChangeTtl, mailboxChangeTtl);
+ }
+}
diff --git a/server/data/data-jmap-cassandra/src/main/java/org/apache/james/jmap/cassandra/change/EmailChangeRepositoryDAO.java b/server/data/data-jmap-cassandra/src/main/java/org/apache/james/jmap/cassandra/change/EmailChangeRepositoryDAO.java
index 2c608d3921..112eded22d 100644
--- a/server/data/data-jmap-cassandra/src/main/java/org/apache/james/jmap/cassandra/change/EmailChangeRepositoryDAO.java
+++ b/server/data/data-jmap-cassandra/src/main/java/org/apache/james/jmap/cassandra/change/EmailChangeRepositoryDAO.java
@@ -49,6 +49,7 @@ import org.apache.james.jmap.api.model.AccountId;
import org.apache.james.mailbox.cassandra.ids.CassandraMessageId;
import org.apache.james.mailbox.model.MessageId;
+import com.datastax.oss.driver.api.core.CqlIdentifier;
import com.datastax.oss.driver.api.core.CqlSession;
import com.datastax.oss.driver.api.core.cql.PreparedStatement;
import com.datastax.oss.driver.api.core.cql.Row;
@@ -66,6 +67,7 @@ import reactor.core.publisher.Mono;
public class EmailChangeRepositoryDAO {
private static final TypeCodec<Set<UUID>> SET_OF_UUIDS_CODEC = CodecRegistry.DEFAULT.codecFor(DataTypes.frozenSetOf(DataTypes.UUID), GenericType.setOf(UUID.class));
+ private static final CqlIdentifier TTL_FOR_ROW = CqlIdentifier.fromCql("ttl");
private final CassandraAsyncExecutor executor;
private final UserDefinedType zonedDateTimeUserType;
@@ -74,9 +76,11 @@ public class EmailChangeRepositoryDAO {
private final PreparedStatement selectFromStatement;
private final PreparedStatement selectLatestStatement;
private final PreparedStatement selectLatestNotDelegatedStatement;
+ private final int timeToLive;
@Inject
- public EmailChangeRepositoryDAO(CqlSession session, CassandraTypesProvider cassandraTypesProvider) {
+ public EmailChangeRepositoryDAO(CqlSession session, CassandraTypesProvider cassandraTypesProvider,
+ CassandraChangesConfiguration cassandraChangesConfiguration) {
executor = new CassandraAsyncExecutor(session);
zonedDateTimeUserType = cassandraTypesProvider.getDefinedUserType(CassandraZonedDateTimeModule.ZONED_DATE_TIME);
@@ -88,6 +92,7 @@ public class EmailChangeRepositoryDAO {
.value(CREATED, bindMarker(CREATED))
.value(UPDATED, bindMarker(UPDATED))
.value(DESTROYED, bindMarker(DESTROYED))
+ .usingTtl(bindMarker(TTL_FOR_ROW))
.build());
selectAllStatement = session.prepare(selectFrom(TABLE_NAME)
@@ -118,6 +123,8 @@ public class EmailChangeRepositoryDAO {
.limit(1)
.allowFiltering()
.build());
+
+ timeToLive = Math.toIntExact(cassandraChangesConfiguration.getEmailChangeTtl().getSeconds());
}
Mono<Void> insert(EmailChange change) {
@@ -128,7 +135,8 @@ public class EmailChangeRepositoryDAO {
.set(CREATED, toUuidSet(change.getCreated()), SET_OF_UUIDS_CODEC)
.set(UPDATED, toUuidSet(change.getUpdated()), SET_OF_UUIDS_CODEC)
.set(DESTROYED, toUuidSet(change.getDestroyed()), SET_OF_UUIDS_CODEC)
- .setUdtValue(DATE, CassandraZonedDateTimeModule.toUDT(zonedDateTimeUserType, change.getDate())));
+ .setUdtValue(DATE, CassandraZonedDateTimeModule.toUDT(zonedDateTimeUserType, change.getDate()))
+ .setInt(TTL_FOR_ROW, timeToLive));
}
private ImmutableSet<UUID> toUuidSet(List<MessageId> idSet) {
diff --git a/server/data/data-jmap-cassandra/src/main/java/org/apache/james/jmap/cassandra/change/MailboxChangeRepositoryDAO.java b/server/data/data-jmap-cassandra/src/main/java/org/apache/james/jmap/cassandra/change/MailboxChangeRepositoryDAO.java
index d966d5194a..1fde9bf32a 100644
--- a/server/data/data-jmap-cassandra/src/main/java/org/apache/james/jmap/cassandra/change/MailboxChangeRepositoryDAO.java
+++ b/server/data/data-jmap-cassandra/src/main/java/org/apache/james/jmap/cassandra/change/MailboxChangeRepositoryDAO.java
@@ -49,6 +49,7 @@ import org.apache.james.jmap.api.model.AccountId;
import org.apache.james.mailbox.cassandra.ids.CassandraId;
import org.apache.james.mailbox.model.MailboxId;
+import com.datastax.oss.driver.api.core.CqlIdentifier;
import com.datastax.oss.driver.api.core.CqlSession;
import com.datastax.oss.driver.api.core.cql.PreparedStatement;
import com.datastax.oss.driver.api.core.cql.Row;
@@ -68,6 +69,7 @@ import reactor.core.publisher.Mono;
public class MailboxChangeRepositoryDAO {
private static final TypeCodec<Set<UUID>> SET_OF_UUIDS_CODEC = CodecRegistry.DEFAULT.codecFor(DataTypes.frozenSetOf(DataTypes.UUID), GenericType.setOf(UUID.class));
+ private static final CqlIdentifier TTL_FOR_ROW = CqlIdentifier.fromCql("ttl");
private final CassandraAsyncExecutor executor;
private final UserDefinedType zonedDateTimeUserType;
@@ -76,9 +78,11 @@ public class MailboxChangeRepositoryDAO {
private final PreparedStatement selectFromStatement;
private final PreparedStatement selectLatestStatement;
private final PreparedStatement selectLatestNotDelegatedStatement;
+ private final int timeToLive;
@Inject
- public MailboxChangeRepositoryDAO(CqlSession session, CassandraTypesProvider cassandraTypesProvider) {
+ public MailboxChangeRepositoryDAO(CqlSession session, CassandraTypesProvider cassandraTypesProvider,
+ CassandraChangesConfiguration cassandraChangesConfiguration) {
executor = new CassandraAsyncExecutor(session);
zonedDateTimeUserType = cassandraTypesProvider.getDefinedUserType(CassandraZonedDateTimeModule.ZONED_DATE_TIME);
@@ -91,6 +95,7 @@ public class MailboxChangeRepositoryDAO {
.value(CREATED, bindMarker(CREATED))
.value(UPDATED, bindMarker(UPDATED))
.value(DESTROYED, bindMarker(DESTROYED))
+ .usingTtl(bindMarker(TTL_FOR_ROW))
.build());
selectAllStatement = session.prepare(selectFrom(TABLE_NAME)
@@ -121,6 +126,8 @@ public class MailboxChangeRepositoryDAO {
.limit(1)
.allowFiltering()
.build());
+
+ timeToLive = Math.toIntExact(cassandraChangesConfiguration.getMailboxChangeTtl().getSeconds());
}
Mono<Void> insert(MailboxChange change) {
@@ -132,7 +139,8 @@ public class MailboxChangeRepositoryDAO {
.set(CREATED, toUuidSet(change.getCreated()), SET_OF_UUIDS_CODEC)
.set(UPDATED, toUuidSet(change.getUpdated()), SET_OF_UUIDS_CODEC)
.set(DESTROYED, toUuidSet(change.getDestroyed()), SET_OF_UUIDS_CODEC)
- .setUdtValue(DATE, CassandraZonedDateTimeModule.toUDT(zonedDateTimeUserType, change.getDate())));
+ .setUdtValue(DATE, CassandraZonedDateTimeModule.toUDT(zonedDateTimeUserType, change.getDate()))
+ .setInt(TTL_FOR_ROW, timeToLive));
}
private ImmutableSet<UUID> toUuidSet(List<MailboxId> idSet) {
diff --git a/server/data/data-jmap-cassandra/src/test/java/org/apache/james/jmap/cassandra/change/CassandraChangesConfigurationTest.java b/server/data/data-jmap-cassandra/src/test/java/org/apache/james/jmap/cassandra/change/CassandraChangesConfigurationTest.java
new file mode 100644
index 0000000000..f75a5d60af
--- /dev/null
+++ b/server/data/data-jmap-cassandra/src/test/java/org/apache/james/jmap/cassandra/change/CassandraChangesConfigurationTest.java
@@ -0,0 +1,83 @@
+/****************************************************************
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information *
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the *
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the *
+ * specific language governing permissions and limitations *
+ * under the License. *
+ ****************************************************************/
+
+package org.apache.james.jmap.cassandra.change;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+import java.time.Duration;
+
+import org.apache.commons.configuration2.PropertiesConfiguration;
+import org.junit.jupiter.api.Test;
+
+class CassandraChangesConfigurationTest {
+ private static final Duration TOO_BIG_TTL = Duration.ofSeconds(Integer.MAX_VALUE + 1L);
+
+ @Test
+ void fromShouldReturnValuesFromSuppliedConfiguration() {
+ PropertiesConfiguration configuration = new PropertiesConfiguration();
+ configuration.addProperty("email.change.ttl", "3 days");
+ configuration.addProperty("mailbox.change.ttl", "12 hours");
+
+ assertThat(CassandraChangesConfiguration.from(configuration))
+ .isEqualTo(new CassandraChangesConfiguration.Builder()
+ .emailChangeTtl(Duration.ofDays(3))
+ .mailboxChangeTtl(Duration.ofHours(12))
+ .build());
+ }
+
+ @Test
+ void fromShouldFallbackToDefaultValueWhenEmptySuppliedConfiguration() {
+ PropertiesConfiguration emptyConfiguration = new PropertiesConfiguration();
+
+ assertThat(CassandraChangesConfiguration.from(emptyConfiguration))
+ .isEqualTo(new CassandraChangesConfiguration.Builder()
+ .emailChangeTtl(Duration.ofDays(60))
+ .mailboxChangeTtl(Duration.ofDays(60))
+ .build());
+ }
+
+ @Test
+ void shouldThrowWhenConfiguredNegativeTTL() {
+ PropertiesConfiguration configuration = new PropertiesConfiguration();
+ configuration.addProperty("email.change.ttl", "-300");
+
+ assertThatThrownBy(() -> CassandraChangesConfiguration.from(configuration))
+ .isInstanceOf(IllegalArgumentException.class);
+ }
+
+ @Test
+ void shouldAcceptZeroTTL() {
+ PropertiesConfiguration configuration = new PropertiesConfiguration();
+ configuration.addProperty("email.change.ttl", "0");
+
+ assertThat(CassandraChangesConfiguration.from(configuration).getEmailChangeTtl().getSeconds())
+ .isEqualTo(0L);
+ }
+
+ @Test
+ void shouldThrowWhenConfiguredTooBigTTL() {
+ assertThatThrownBy(() -> new CassandraChangesConfiguration.Builder()
+ .emailChangeTtl(TOO_BIG_TTL)
+ .mailboxChangeTtl(TOO_BIG_TTL)
+ .build())
+ .isInstanceOf(IllegalArgumentException.class);
+ }
+}
diff --git a/server/data/data-jmap-cassandra/src/test/java/org/apache/james/jmap/cassandra/change/CassandraEmailChangeRepositoryTest.java b/server/data/data-jmap-cassandra/src/test/java/org/apache/james/jmap/cassandra/change/CassandraEmailChangeRepositoryTest.java
index 5741f80bd5..cadc92e00e 100644
--- a/server/data/data-jmap-cassandra/src/test/java/org/apache/james/jmap/cassandra/change/CassandraEmailChangeRepositoryTest.java
+++ b/server/data/data-jmap-cassandra/src/test/java/org/apache/james/jmap/cassandra/change/CassandraEmailChangeRepositoryTest.java
@@ -19,17 +19,25 @@
package org.apache.james.jmap.cassandra.change;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatCode;
+import static org.awaitility.Awaitility.await;
+
+import java.time.Duration;
+
import org.apache.james.backends.cassandra.CassandraCluster;
import org.apache.james.backends.cassandra.CassandraClusterExtension;
import org.apache.james.backends.cassandra.components.CassandraModule;
import org.apache.james.backends.cassandra.init.CassandraZonedDateTimeModule;
import org.apache.james.backends.cassandra.versions.CassandraSchemaVersionModule;
+import org.apache.james.jmap.api.change.EmailChange;
import org.apache.james.jmap.api.change.EmailChangeRepository;
import org.apache.james.jmap.api.change.EmailChangeRepositoryContract;
import org.apache.james.jmap.api.change.State;
import org.apache.james.mailbox.cassandra.ids.CassandraMessageId;
import org.apache.james.mailbox.model.MessageId;
import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
public class CassandraEmailChangeRepositoryTest implements EmailChangeRepositoryContract {
@@ -45,7 +53,8 @@ public class CassandraEmailChangeRepositoryTest implements EmailChangeRepository
@BeforeEach
public void setUp(CassandraCluster cassandra) {
- emailChangeRepositoryDAO = new EmailChangeRepositoryDAO(cassandra.getConf(), cassandra.getTypesProvider());
+ emailChangeRepositoryDAO = new EmailChangeRepositoryDAO(cassandra.getConf(), cassandra.getTypesProvider(),
+ CassandraChangesConfiguration.DEFAULT);
emailChangeRepository = new CassandraEmailChangeRepository(emailChangeRepositoryDAO, DEFAULT_NUMBER_OF_CHANGES);
}
@@ -63,4 +72,52 @@ public class CassandraEmailChangeRepositoryTest implements EmailChangeRepository
public MessageId generateNewMessageId() {
return new CassandraMessageId.Factory().generate();
}
+
+ @Test
+ void emailChangeRecordsShouldBeDeletedAfterTTL(CassandraCluster cassandra) {
+ emailChangeRepositoryDAO = new EmailChangeRepositoryDAO(cassandra.getConf(), cassandra.getTypesProvider(),
+ new CassandraChangesConfiguration.Builder()
+ .emailChangeTtl(Duration.ofSeconds(1))
+ .build());
+
+ EmailChange emailChange = EmailChange.builder()
+ .accountId(ACCOUNT_ID)
+ .state(generateNewState())
+ .date(DATE)
+ .isShared(false)
+ .created(generateNewMessageId())
+ .build();
+
+ assertThatCode(() -> emailChangeRepositoryDAO.insert(emailChange).block())
+ .doesNotThrowAnyException();
+
+ await().atMost(Duration.ofSeconds(3))
+ .await()
+ .untilAsserted(() -> assertThat(emailChangeRepositoryDAO.getAllChanges(ACCOUNT_ID).collectList().block())
+ .isEmpty());
+ }
+
+ @Test
+ void emailChangeRecordsShouldNotBeDeletedWhenTtlIsZero(CassandraCluster cassandra) throws InterruptedException {
+ emailChangeRepositoryDAO = new EmailChangeRepositoryDAO(cassandra.getConf(), cassandra.getTypesProvider(),
+ new CassandraChangesConfiguration.Builder()
+ .emailChangeTtl(Duration.ofSeconds(0))
+ .build());
+
+ EmailChange emailChange = EmailChange.builder()
+ .accountId(ACCOUNT_ID)
+ .state(generateNewState())
+ .date(DATE)
+ .isShared(false)
+ .created(generateNewMessageId())
+ .build();
+
+ assertThatCode(() -> emailChangeRepositoryDAO.insert(emailChange).block())
+ .doesNotThrowAnyException();
+
+ Thread.sleep(200L);
+
+ assertThat(emailChangeRepositoryDAO.getAllChanges(ACCOUNT_ID).collectList().block())
+ .isNotEmpty();
+ }
}
diff --git a/server/data/data-jmap-cassandra/src/test/java/org/apache/james/jmap/cassandra/change/CassandraMailboxChangeRepositoryTest.java b/server/data/data-jmap-cassandra/src/test/java/org/apache/james/jmap/cassandra/change/CassandraMailboxChangeRepositoryTest.java
index 2c7e5e2be0..57db1688df 100644
--- a/server/data/data-jmap-cassandra/src/test/java/org/apache/james/jmap/cassandra/change/CassandraMailboxChangeRepositoryTest.java
+++ b/server/data/data-jmap-cassandra/src/test/java/org/apache/james/jmap/cassandra/change/CassandraMailboxChangeRepositoryTest.java
@@ -19,19 +19,29 @@
package org.apache.james.jmap.cassandra.change;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatCode;
+import static org.awaitility.Awaitility.await;
+
+import java.time.Duration;
+
import org.apache.james.backends.cassandra.CassandraCluster;
import org.apache.james.backends.cassandra.CassandraClusterExtension;
import org.apache.james.backends.cassandra.components.CassandraModule;
import org.apache.james.backends.cassandra.init.CassandraZonedDateTimeModule;
import org.apache.james.backends.cassandra.versions.CassandraSchemaVersionModule;
+import org.apache.james.jmap.api.change.MailboxChange;
import org.apache.james.jmap.api.change.MailboxChangeRepository;
import org.apache.james.jmap.api.change.MailboxChangeRepositoryContract;
import org.apache.james.jmap.api.change.State;
import org.apache.james.mailbox.cassandra.ids.CassandraId;
import org.apache.james.mailbox.model.MailboxId;
import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
+import com.google.common.collect.ImmutableList;
+
public class CassandraMailboxChangeRepositoryTest implements MailboxChangeRepositoryContract {
@RegisterExtension
@@ -45,7 +55,8 @@ public class CassandraMailboxChangeRepositoryTest implements MailboxChangeReposi
@BeforeEach
public void setUp(CassandraCluster cassandra) {
- mailboxChangeRepositoryDAO = new MailboxChangeRepositoryDAO(cassandra.getConf(), cassandra.getTypesProvider());
+ mailboxChangeRepositoryDAO = new MailboxChangeRepositoryDAO(cassandra.getConf(), cassandra.getTypesProvider(),
+ CassandraChangesConfiguration.DEFAULT);
mailboxChangeRepository = new CassandraMailboxChangeRepository(mailboxChangeRepositoryDAO, DEFAULT_NUMBER_OF_CHANGES);
}
@@ -63,4 +74,52 @@ public class CassandraMailboxChangeRepositoryTest implements MailboxChangeReposi
public MailboxId generateNewMailboxId() {
return CassandraId.timeBased();
}
+
+ @Test
+ void mailboxChangeRecordsShouldBeDeletedAfterTTL(CassandraCluster cassandra) {
+ mailboxChangeRepositoryDAO = new MailboxChangeRepositoryDAO(cassandra.getConf(), cassandra.getTypesProvider(),
+ new CassandraChangesConfiguration.Builder()
+ .mailboxChangeTtl(Duration.ofSeconds(1))
+ .build());
+
+ MailboxChange mailboxChange = MailboxChange.builder()
+ .accountId(ACCOUNT_ID)
+ .state(stateFactory().generate())
+ .date(DATE)
+ .isCountChange(false)
+ .created(ImmutableList.of(generateNewMailboxId()))
+ .build();
+
+ assertThatCode(() -> mailboxChangeRepositoryDAO.insert(mailboxChange).block())
+ .doesNotThrowAnyException();
+
+ await().atMost(Duration.ofSeconds(3))
+ .await()
+ .untilAsserted(() -> assertThat(mailboxChangeRepositoryDAO.getAllChanges(ACCOUNT_ID).collectList().block())
+ .isEmpty());
+ }
+
+ @Test
+ void mailboxChangeRecordsShouldNotBeDeletedWhenTtlIsZero(CassandraCluster cassandra) throws InterruptedException {
+ mailboxChangeRepositoryDAO = new MailboxChangeRepositoryDAO(cassandra.getConf(), cassandra.getTypesProvider(),
+ new CassandraChangesConfiguration.Builder()
+ .mailboxChangeTtl(Duration.ofSeconds(0))
+ .build());
+
+ MailboxChange mailboxChange = MailboxChange.builder()
+ .accountId(ACCOUNT_ID)
+ .state(stateFactory().generate())
+ .date(DATE)
+ .isCountChange(false)
+ .created(ImmutableList.of(generateNewMailboxId()))
+ .build();
+
+ assertThatCode(() -> mailboxChangeRepositoryDAO.insert(mailboxChange).block())
+ .doesNotThrowAnyException();
+
+ Thread.sleep(200L);
+
+ assertThat(mailboxChangeRepositoryDAO.getAllChanges(ACCOUNT_ID).collectList().block())
+ .isNotEmpty();
+ }
}
diff --git a/src/site/xdoc/server/config-cassandra.xml b/src/site/xdoc/server/config-cassandra.xml
index 9909bd1d35..ec509dd51b 100644
--- a/src/site/xdoc/server/config-cassandra.xml
+++ b/src/site/xdoc/server/config-cassandra.xml
@@ -126,6 +126,12 @@
<dd>Optional. Boolean, defaults to true. Allows disabling ACLs: if set to false, delegation will fail and users will only
have access to the mailboxes they own. ACLs can represent a high volume of requests. If you do not propose mailbox sharing
features to your users, you can consider disabling them in order to improve performance.</dd>
+ <dt><strong>email.change.ttl</strong></dt>
+ <dd>Optional. Duration, default to 60 days. Cassandra Time-to-live for Email change records.
+ Setting time-to-live to zero means refusing to use time-to-live on email changes.</dd>
+ <dt><strong>mailbox.change.ttl</strong></dt>
+ <dd>Optional. Duration, default to 60 days. Cassandra Time-to-live for Mailbox change records.
+ Setting time-to-live to zero means refusing to use time-to-live on mailbox changes.</dd>
</dl>
diff --git a/upgrade-instructions.md b/upgrade-instructions.md
index ec868bc67f..d89b87dc66 100644
--- a/upgrade-instructions.md
+++ b/upgrade-instructions.md
@@ -22,6 +22,25 @@ Change list:
- [Mutualize quota table](#mutualize-quota-table)
- [Drop Legacy Cassandra migrations](#drop-legacy-cassandra-migrations)
- [Improve CassandraThreadIdGuessingAlgorithm](#improve-cassandrathreadidguessingalgorithm)
+ - [Set up TTL on the mailbox_change and email_change tables](#set-up-ttl-on-the-mailboxchange-and-emailchange-tables)
+
+### Set up TTL on the mailbox_change and email_change tables
+Date: 15/09/2023
+
+JIRA: https://issues.apache.org/jira/browse/JAMES-3940
+
+Concerned products: Distributed James, Cassandra James Server
+
+From now on, we set a default 60 days (configurable) Cassandra time-to-live on the `mailbox_change` and `email_change` tables
+to reduce considered outdated changes' storage usage.
+
+If you want to keep the old behavior which is not using Cassandra time-to-live on those JMAP change tables, please set the
+following configurations in `cassandra.properties`:
+
+```
+email.change.ttl=0 second
+mailbox.change.ttl=0 second
+```
### Improve CassandraThreadIdGuessingAlgorithm
Date: 14/09/2023
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org