You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@james.apache.org by bt...@apache.org on 2021/04/08 03:56:50 UTC
[james-project] branch master updated (fb92538 -> 4619763)
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git.
from fb92538 JAMES-3553 Disable read_repair_chance & read_repair_chance on table creation
new 43887c3 JAMES-3435 s/ReadConsistency/ConsistencyChoice
new d39460f JAMES-3435 Configuration option: message.write.strong.consistency
new eb44d31 JAMES-3435 CassandraMessageIdToImapUidDAO should rely on LWT only if strong write consistency is requested.
new c490eea JAMES-3435 Use writeConsistency to determine read consistency needs upon writes.
new e139282 JAMES-3435 Document message.write.strong.consistency.unsafe
new 2842ea8 JAMES-3435 Tests for relaxed consistency
new 14fb1fb JAMES-3435 Configuration: mailbox.read.strong.consistency
new 5ae8147 JAMES-3435 Serial read is not needed when listUserMailboxes
new 23131f2 JAMES-3435 Allow to avoid mailbox SERIAL reads when not linked to writes
new 14ec0f3 JAMES-3435 Factorize consistency choices
new db59e02 JAMES-3435 Document mailbox.read.strong.consistency
new 4619763 JAMES-3435 test for mailbox.read.strong.cst
The 12 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../init/configuration/CassandraConfiguration.java | 53 +++++++-
.../CassandraConsistenciesConfiguration.java | 16 +++
.../pages/distributed/configure/cassandra.adoc | 16 ++-
.../CassandraMailboxSessionMapperFactory.java | 2 +-
.../mailbox/cassandra/DeleteMessageListener.java | 21 +++-
.../cassandra/mail/CassandraMailboxMapper.java | 20 +++-
.../cassandra/mail/CassandraMailboxPathV3DAO.java | 17 ++-
.../cassandra/mail/CassandraMessageIdMapper.java | 19 ++-
.../mail/CassandraMessageIdToImapUidDAO.java | 133 +++++++++++----------
.../cassandra/mail/CassandraMessageMapper.java | 13 +-
.../mail/task/RecomputeMailboxCountersService.java | 2 +-
.../task/SolveMailboxInconsistenciesService.java | 3 +-
.../task/SolveMessageInconsistenciesService.java | 2 +-
.../cassandra/CassandraMailboxManagerTest.java | 3 +-
.../mail/CassandraMailboxMapperGenericTest.java | 23 ++++
.../mail/CassandraMailboxPathV3DAOTest.java | 3 +-
.../cassandra/mail/CassandraMapperProvider.java | 12 +-
...andraMessageIdMapperRelaxedConsistencyTest.java | 72 +++++++++++
.../mail/CassandraMessageIdMapperTest.java | 6 +-
.../mail/CassandraMessageIdToImapUidDAOTest.java | 4 +-
...ssandraMessageMapperRelaxedConsistencyTest.java | 73 +++++++++++
.../cassandra/mail/CassandraMessageMapperTest.java | 7 +-
.../cassandra/mail/CassandraMessageMoveTest.java | 4 +-
.../CassandraMessageWithAttachmentMapperTest.java | 4 +-
.../task/RecomputeMailboxCountersServiceTest.java | 4 +-
.../SolveMessageInconsistenciesServiceTest.java | 4 +-
.../store/mail/model/MessageIdMapperTest.java | 4 +-
.../store/mail/model/MessageMapperTest.java | 4 +-
src/site/xdoc/server/config-cassandra.xml | 16 ++-
29 files changed, 448 insertions(+), 112 deletions(-)
create mode 100644 mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapperRelaxedConsistencyTest.java
create mode 100644 mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapperRelaxedConsistencyTest.java
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 07/12: JAMES-3435 Configuration:
mailbox.read.strong.consistency
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 14fb1fb064a78f04567946d12fce41f665e02560
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Sun Mar 28 22:45:23 2021 +0700
JAMES-3435 Configuration: mailbox.read.strong.consistency
---
.../init/configuration/CassandraConfiguration.java | 23 +++++++++++++++++++++-
1 file changed, 22 insertions(+), 1 deletion(-)
diff --git a/backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/CassandraConfiguration.java b/backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/CassandraConfiguration.java
index 57e60a4..41284c7 100644
--- a/backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/CassandraConfiguration.java
+++ b/backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/CassandraConfiguration.java
@@ -73,6 +73,7 @@ public class CassandraConfiguration {
private static final String BLOB_PART_SIZE = "mailbox.blob.part.size";
private static final String ATTACHMENT_V2_MIGRATION_READ_TIMEOUT = "attachment.v2.migration.read.timeout";
private static final String MESSAGE_ATTACHMENTID_READ_TIMEOUT = "message.attachmentids.read.timeout";
+ private static final String MAILBOX_READ_STRONG_CONSISTENCY = "mailbox.read.strong.consistency";
private static final String MESSAGE_READ_STRONG_CONSISTENCY = "message.read.strong.consistency";
private static final String MESSAGE_WRITE_STRONG_CONSISTENCY = "message.write.strong.consistency.unsafe";
private static final String CONSISTENCY_LEVEL_REGULAR = "cassandra.consistency_level.regular";
@@ -97,9 +98,20 @@ public class CassandraConfiguration {
private Optional<Float> mailboxReadRepair = Optional.empty();
private Optional<Float> mailboxCountersReadRepairMax = Optional.empty();
private Optional<Float> mailboxCountersReadRepairChanceOneHundred = Optional.empty();
+ private Optional<Boolean> mailboxReadStrongConsistency = Optional.empty();
private Optional<Boolean> messageReadStrongConsistency = Optional.empty();
private Optional<Boolean> messageWriteStrongConsistency = Optional.empty();
+ public Builder mailboxReadStrongConsistency(boolean value) {
+ this.mailboxReadStrongConsistency = Optional.of(value);
+ return this;
+ }
+
+ public Builder mailboxReadStrongConsistency(Optional<Boolean> value) {
+ this.mailboxReadStrongConsistency = value;
+ return this;
+ }
+
public Builder messageReadStrongConsistency(boolean value) {
this.messageReadStrongConsistency = Optional.of(value);
return this;
@@ -326,6 +338,7 @@ public class CassandraConfiguration {
mailboxReadRepair.orElse(DEFAULT_MAILBOX_READ_REPAIR),
mailboxCountersReadRepairMax.orElse(DEFAULT_MAX_MAILBOX_COUNTERS_READ_REPAIR_CHANCE),
mailboxCountersReadRepairChanceOneHundred.orElse(DEFAULT_ONE_HUNDRED_MAILBOX_COUNTERS_READ_REPAIR_CHANCE),
+ mailboxReadStrongConsistency.orElse(DEFAULT_STRONG_CONSISTENCY),
messageReadStrongConsistency.orElse(DEFAULT_STRONG_CONSISTENCY),
messageWriteStrongConsistency.orElse(DEFAULT_STRONG_CONSISTENCY));
}
@@ -369,6 +382,8 @@ public class CassandraConfiguration {
propertiesConfiguration.getFloat(MAILBOX_MAX_COUNTERS_READ_REPAIR, null)))
.mailboxCountersReadRepairChanceOneHundred(Optional.ofNullable(
propertiesConfiguration.getFloat(MAILBOX_ONE_HUNDRED_COUNTERS_READ_REPAIR, null)))
+ .mailboxReadStrongConsistency(Optional.ofNullable(
+ propertiesConfiguration.getBoolean(MAILBOX_READ_STRONG_CONSISTENCY, null)))
.messageReadStrongConsistency(Optional.ofNullable(
propertiesConfiguration.getBoolean(MESSAGE_READ_STRONG_CONSISTENCY, null)))
.messageWriteStrongConsistency(Optional.ofNullable(
@@ -392,6 +407,7 @@ public class CassandraConfiguration {
private final float mailboxReadRepair;
private final float mailboxCountersReadRepairChanceMax;
private final float mailboxCountersReadRepairChanceOneHundred;
+ private final boolean mailboxReadStrongConsistency;
private final boolean messageReadStrongConsistency;
private final boolean messageWriteStrongConsistency;
@@ -402,7 +418,7 @@ public class CassandraConfiguration {
int blobPartSize, final int attachmentV2MigrationReadTimeout, int messageAttachmentIdsReadTimeout,
String consistencyLevelRegular, String consistencyLevelLightweightTransaction,
float mailboxReadRepair, float mailboxCountersReadRepairChanceMax,
- float mailboxCountersReadRepairChanceOneHundred, boolean messageReadStrongConsistency,
+ float mailboxCountersReadRepairChanceOneHundred, boolean mailboxReadStrongConsistency, boolean messageReadStrongConsistency,
boolean messageWriteStrongConsistency) {
this.aclMaxRetry = aclMaxRetry;
this.messageReadChunkSize = messageReadChunkSize;
@@ -420,10 +436,15 @@ public class CassandraConfiguration {
this.mailboxReadRepair = mailboxReadRepair;
this.mailboxCountersReadRepairChanceMax = mailboxCountersReadRepairChanceMax;
this.mailboxCountersReadRepairChanceOneHundred = mailboxCountersReadRepairChanceOneHundred;
+ this.mailboxReadStrongConsistency = mailboxReadStrongConsistency;
this.messageReadStrongConsistency = messageReadStrongConsistency;
this.messageWriteStrongConsistency = messageWriteStrongConsistency;
}
+ public boolean isMailboxReadStrongConsistency() {
+ return mailboxReadStrongConsistency;
+ }
+
public boolean isMessageWriteStrongConsistency() {
return messageWriteStrongConsistency;
}
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 04/12: JAMES-3435 Use writeConsistency to determine
read consistency needs upon writes.
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit c490eeaff0af77d65a35e90e1f6b4684b9e322ad
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Sun Mar 28 14:44:20 2021 +0700
JAMES-3435 Use writeConsistency to determine read consistency needs upon writes.
---
.../CassandraMailboxSessionMapperFactory.java | 2 +-
.../mailbox/cassandra/DeleteMessageListener.java | 20 +++++++++++++++++---
.../cassandra/mail/CassandraMessageIdMapper.java | 11 +++++++++--
.../cassandra/mail/CassandraMessageMapper.java | 10 +++++++++-
4 files changed, 36 insertions(+), 7 deletions(-)
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/CassandraMailboxSessionMapperFactory.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/CassandraMailboxSessionMapperFactory.java
index 26a8a04..671b894 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/CassandraMailboxSessionMapperFactory.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/CassandraMailboxSessionMapperFactory.java
@@ -217,6 +217,6 @@ public class CassandraMailboxSessionMapperFactory extends MailboxSessionMapperFa
public DeleteMessageListener deleteMessageListener() {
return new DeleteMessageListener(imapUidDAO, messageIdDAO, messageDAO, messageDAOV3, attachmentDAOV2, ownerDAO,
attachmentMessageIdDAO, aclMapper, userMailboxRightsDAO, applicableFlagDAO, firstUnseenDAO, deletedMessageDAO,
- mailboxCounterDAO, mailboxRecentsDAO, blobStore);
+ mailboxCounterDAO, mailboxRecentsDAO, blobStore, cassandraConfiguration);
}
}
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/DeleteMessageListener.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/DeleteMessageListener.java
index ddfe04b..d48623b 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/DeleteMessageListener.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/DeleteMessageListener.java
@@ -19,6 +19,8 @@
package org.apache.james.mailbox.cassandra;
+import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ConsistencyChoice.STRONG;
+import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ConsistencyChoice.WEAK;
import static org.apache.james.util.FunctionalUtils.negate;
import static org.apache.james.util.ReactorUtils.DEFAULT_CONCURRENCY;
@@ -27,6 +29,7 @@ import java.util.function.Predicate;
import javax.inject.Inject;
+import org.apache.james.backends.cassandra.init.configuration.CassandraConfiguration;
import org.apache.james.blob.api.BlobStore;
import org.apache.james.events.Event;
import org.apache.james.events.EventListener;
@@ -95,13 +98,16 @@ public class DeleteMessageListener implements EventListener.GroupEventListener {
private final CassandraMailboxCounterDAO counterDAO;
private final CassandraMailboxRecentsDAO recentsDAO;
private final BlobStore blobStore;
+ private final CassandraConfiguration cassandraConfiguration;
@Inject
public DeleteMessageListener(CassandraMessageIdToImapUidDAO imapUidDAO, CassandraMessageIdDAO messageIdDAO, CassandraMessageDAO messageDAO,
CassandraMessageDAOV3 messageDAOV3, CassandraAttachmentDAOV2 attachmentDAO, CassandraAttachmentOwnerDAO ownerDAO,
CassandraAttachmentMessageIdDAO attachmentMessageIdDAO, CassandraACLMapper aclMapper,
CassandraUserMailboxRightsDAO rightsDAO, CassandraApplicableFlagDAO applicableFlagDAO,
- CassandraFirstUnseenDAO firstUnseenDAO, CassandraDeletedMessageDAO deletedMessageDAO, CassandraMailboxCounterDAO counterDAO, CassandraMailboxRecentsDAO recentsDAO, BlobStore blobStore) {
+ CassandraFirstUnseenDAO firstUnseenDAO, CassandraDeletedMessageDAO deletedMessageDAO,
+ CassandraMailboxCounterDAO counterDAO, CassandraMailboxRecentsDAO recentsDAO, BlobStore blobStore,
+ CassandraConfiguration cassandraConfiguration) {
this.imapUidDAO = imapUidDAO;
this.messageIdDAO = messageIdDAO;
this.messageDAO = messageDAO;
@@ -117,6 +123,7 @@ public class DeleteMessageListener implements EventListener.GroupEventListener {
this.counterDAO = counterDAO;
this.recentsDAO = recentsDAO;
this.blobStore = blobStore;
+ this.cassandraConfiguration = cassandraConfiguration;
}
@Override
@@ -239,15 +246,22 @@ public class DeleteMessageListener implements EventListener.GroupEventListener {
}
private Mono<Boolean> isReferenced(CassandraMessageId id) {
- return imapUidDAO.retrieve(id, ALL_MAILBOXES, CassandraMessageIdToImapUidDAO.ConsistencyChoice.STRONG)
+ return imapUidDAO.retrieve(id, ALL_MAILBOXES, chooseReadConsistencyUponWrites())
.hasElements()
.map(negate());
}
private Mono<Boolean> isReferenced(CassandraMessageId id, CassandraId excludedId) {
- return imapUidDAO.retrieve(id, ALL_MAILBOXES, CassandraMessageIdToImapUidDAO.ConsistencyChoice.STRONG)
+ return imapUidDAO.retrieve(id, ALL_MAILBOXES, chooseReadConsistencyUponWrites())
.filter(metadata -> !metadata.getComposedMessageId().getMailboxId().equals(excludedId))
.hasElements()
.map(negate());
}
+
+ private CassandraMessageIdToImapUidDAO.ConsistencyChoice chooseReadConsistencyUponWrites() {
+ if (cassandraConfiguration.isMessageWriteStrongConsistency()) {
+ return STRONG;
+ }
+ return WEAK;
+ }
}
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapper.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapper.java
index 9cb0d5e..1b5f58e 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapper.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapper.java
@@ -155,6 +155,13 @@ public class CassandraMessageIdMapper implements MessageIdMapper {
}
}
+ private CassandraMessageIdToImapUidDAO.ConsistencyChoice chooseReadConsistencyUponWrites() {
+ if (cassandraConfiguration.isMessageWriteStrongConsistency()) {
+ return STRONG;
+ }
+ return WEAK;
+ }
+
@Override
public void save(MailboxMessage mailboxMessage) throws MailboxException {
CassandraId mailboxId = (CassandraId) mailboxMessage.getMailboxId();
@@ -220,7 +227,7 @@ public class CassandraMessageIdMapper implements MessageIdMapper {
}
private Mono<Void> retrieveAndDeleteIndices(CassandraMessageId messageId, Optional<CassandraId> mailboxId) {
- return imapUidDAO.retrieve(messageId, mailboxId, STRONG)
+ return imapUidDAO.retrieve(messageId, mailboxId, chooseReadConsistencyUponWrites())
.flatMap(this::deleteIds, ReactorUtils.DEFAULT_CONCURRENCY)
.then();
}
@@ -283,7 +290,7 @@ public class CassandraMessageIdMapper implements MessageIdMapper {
private Mono<List<Pair<Flags, ComposedMessageIdWithMetaData>>> updateFlags(MailboxId mailboxId, MessageId messageId, Flags newState, MessageManager.FlagsUpdateMode updateMode) {
CassandraId cassandraId = (CassandraId) mailboxId;
- return imapUidDAO.retrieve((CassandraMessageId) messageId, Optional.of(cassandraId), STRONG)
+ return imapUidDAO.retrieve((CassandraMessageId) messageId, Optional.of(cassandraId), chooseReadConsistencyUponWrites())
.flatMap(oldComposedId -> updateFlags(newState, updateMode, cassandraId, oldComposedId), ReactorUtils.DEFAULT_CONCURRENCY)
.switchIfEmpty(Mono.error(MailboxDeleteDuringUpdateException::new))
.collectList();
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapper.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapper.java
index 68a7e72..faa3464 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapper.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapper.java
@@ -20,6 +20,7 @@
package org.apache.james.mailbox.cassandra.mail;
import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ConsistencyChoice.STRONG;
+import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ConsistencyChoice.WEAK;
import static org.apache.james.util.ReactorUtils.DEFAULT_CONCURRENCY;
import java.security.SecureRandom;
@@ -409,7 +410,7 @@ public class CassandraMessageMapper implements MessageMapper {
private Mono<FlagsUpdateStageResult> retryUpdatesStage(CassandraId mailboxId, FlagsUpdateCalculator flagsUpdateCalculator, List<ComposedMessageId> failed) {
if (!failed.isEmpty()) {
Flux<ComposedMessageIdWithMetaData> toUpdate = Flux.fromIterable(failed)
- .flatMap(ids -> imapUidDAO.retrieve((CassandraMessageId) ids.getMessageId(), Optional.of((CassandraId) ids.getMailboxId()), STRONG),
+ .flatMap(ids -> imapUidDAO.retrieve((CassandraMessageId) ids.getMessageId(), Optional.of((CassandraId) ids.getMailboxId()), chooseReadConsistencyUponWrites()),
DEFAULT_CONCURRENCY);
return runUpdateStage(mailboxId, toUpdate, flagsUpdateCalculator);
} else {
@@ -417,6 +418,13 @@ public class CassandraMessageMapper implements MessageMapper {
}
}
+ private CassandraMessageIdToImapUidDAO.ConsistencyChoice chooseReadConsistencyUponWrites() {
+ if (cassandraConfiguration.isMessageWriteStrongConsistency()) {
+ return STRONG;
+ }
+ return WEAK;
+ }
+
private Mono<FlagsUpdateStageResult> runUpdateStage(CassandraId mailboxId, Flux<ComposedMessageIdWithMetaData> toBeUpdated, FlagsUpdateCalculator flagsUpdateCalculator) {
return computeNewModSeq(mailboxId)
.flatMapMany(newModSeq -> toBeUpdated
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 11/12: JAMES-3435 Document
mailbox.read.strong.consistency
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit db59e02645bee7d3b7e498d6420e866dce69ce11
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Sun Mar 28 23:08:21 2021 +0700
JAMES-3435 Document mailbox.read.strong.consistency
---
.../servers/pages/distributed/configure/cassandra.adoc | 11 +++++++++--
src/site/xdoc/server/config-cassandra.xml | 11 +++++++++--
2 files changed, 18 insertions(+), 4 deletions(-)
diff --git a/docs/modules/servers/pages/distributed/configure/cassandra.adoc b/docs/modules/servers/pages/distributed/configure/cassandra.adoc
index b9810d8..25c4789 100644
--- a/docs/modules/servers/pages/distributed/configure/cassandra.adoc
+++ b/docs/modules/servers/pages/distributed/configure/cassandra.adoc
@@ -185,16 +185,23 @@ Controls the number of messages to be retrieved in parallel.
| Optional. Defaults to 102400 (100KB).
Controls the size of blob parts used to store messages in the Cassandra blobStore.
+| mailbox.read.strong.consistency
+| Optional. Boolean, defaults to true. Disabling should be considered experimental.
+If enabled, regular consistency level is used for read transactions for mailbox. Not doing so might result
+in stale reads as the system.paxos table will not be checked for latest updates. Better performance are expected
+by turning it off. Note that reads performed as part of write transactions are always performed with a strong
+consistency.
+
| message.read.strong.consistency
| Optional. Boolean, defaults to true. Disabling should be considered experimental.
-If enabled, regular consistency level is used for read transactions for message. This might result
+If enabled, regular consistency level is used for read transactions for message. Not doing so might result
in stale reads as the system.paxos table will not be checked for latest updates. Better performance are expected
by turning it off. Note that reads performed as part of write transactions are always performed with a strong
consistency.
| message.write.strong.consistency.unsafe
| Optional. Boolean, defaults to true. Disabling should be considered experimental and unsafe.
-If enabled, Lightweight transactions will no longer be used upon messages operation (table `imapUidTable`).
+If disabled, Lightweight transactions will no longer be used upon messages operation (table `imapUidTable`).
As message flags updates relies so far on a read-before-write model, it exposes yourself to data races leading to
potentially update loss. Better performance are expected
by turning it off. Reads performed as part of write transaction are also performed with a relaxed consistency.
diff --git a/src/site/xdoc/server/config-cassandra.xml b/src/site/xdoc/server/config-cassandra.xml
index 904247f..f91ed83 100644
--- a/src/site/xdoc/server/config-cassandra.xml
+++ b/src/site/xdoc/server/config-cassandra.xml
@@ -149,16 +149,23 @@
<dt><strong>mailbox.blob.part.size</strong></dt>
<dd>Optional. Defaults to 102400 (100KB).<br/> Controls the size of blob parts used to store messages.</dd>
+ <dt><strong>mailbox.read.strong.consistency</strong></dt>
+ <dd>Optional. Boolean, defaults to true. Disabling should be considered experimental.
+ If enabled, regular consistency level is used for read transactions for mailbox. Not doing so might result
+ in stale reads as the system.paxos table will not be checked for latest updates. Better performance are expected
+ by turning it off. Note that reads performed as part of write transactions are always performed with a strong
+ consistency.</dd>
+
<dt><strong>message.read.strong.consistency</strong></dt>
<dd>Optional. Boolean, defaults to true. Disabling should be considered experimental.
- If enabled, regular consistency level is used for read transactions for message. This might result
+ If enabled, regular consistency level is used for read transactions for message. Not doing so might result
in stale reads as the system.paxos table will not be checked for latest updates. Better performance are expected
by turning it off. Note that reads performed as part of write transactions are always performed with a strong
consistency.</dd>
<dt><strong>message.write.strong.consistency.unsafe</strong></dt>
<dd>Optional. Boolean, defaults to true. Disabling should be considered experimental and unsafe.
- If enabled, Lightweight transactions will no longer be used upon messages operation (table `imapUidTable`).
+ If disabled, Lightweight transactions will no longer be used upon messages operation (table `imapUidTable`).
As message flags updates relies so far on a read-before-write model, it exposes yourself to data races leading to
potentially update loss. Better performance are expected
by turning it off. Reads performed as part of write transaction are also performed with a relaxed consistency.</dd>
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 03/12: JAMES-3435 CassandraMessageIdToImapUidDAO
should rely on LWT only if strong write consistency is requested.
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit eb44d318e3081dcfeb8d109d440429facd221ab5
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Sun Mar 28 14:38:31 2021 +0700
JAMES-3435 CassandraMessageIdToImapUidDAO should rely on LWT only if strong write consistency is requested.
---
.../mail/CassandraMessageIdToImapUidDAO.java | 70 +++++++++++++---------
.../cassandra/CassandraMailboxManagerTest.java | 3 +-
.../mail/CassandraMessageIdMapperTest.java | 3 +-
.../mail/CassandraMessageIdToImapUidDAOTest.java | 4 +-
.../cassandra/mail/CassandraMessageMapperTest.java | 4 +-
.../SolveMessageInconsistenciesServiceTest.java | 4 +-
6 files changed, 55 insertions(+), 33 deletions(-)
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdToImapUidDAO.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdToImapUidDAO.java
index e4bce1f..48f29d3 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdToImapUidDAO.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdToImapUidDAO.java
@@ -47,6 +47,7 @@ import javax.inject.Inject;
import javax.mail.Flags;
import javax.mail.Flags.Flag;
+import org.apache.james.backends.cassandra.init.configuration.CassandraConfiguration;
import org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration;
import org.apache.james.backends.cassandra.utils.CassandraAsyncExecutor;
import org.apache.james.mailbox.MessageUid;
@@ -63,7 +64,9 @@ import com.datastax.driver.core.PreparedStatement;
import com.datastax.driver.core.Row;
import com.datastax.driver.core.Session;
import com.datastax.driver.core.Statement;
+import com.datastax.driver.core.querybuilder.Insert;
import com.datastax.driver.core.querybuilder.QueryBuilder;
+import com.datastax.driver.core.querybuilder.Update;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.ImmutableSet;
@@ -97,11 +100,12 @@ public class CassandraMessageIdToImapUidDAO {
private final PreparedStatement selectAll;
private final PreparedStatement select;
private final PreparedStatement listStatement;
+ private final CassandraConfiguration cassandraConfiguration;
private final CassandraConsistenciesConfiguration consistenciesConfiguration;
@Inject
public CassandraMessageIdToImapUidDAO(Session session, CassandraConsistenciesConfiguration consistenciesConfiguration,
- CassandraMessageId.Factory messageIdFactory) {
+ CassandraMessageId.Factory messageIdFactory, CassandraConfiguration cassandraConfiguration) {
this.cassandraAsyncExecutor = new CassandraAsyncExecutor(session);
this.consistenciesConfiguration = consistenciesConfiguration;
this.cassandraConfiguration = cassandraConfiguration;
@@ -122,37 +126,47 @@ public class CassandraMessageIdToImapUidDAO {
}
private PreparedStatement prepareInsert(Session session) {
- return session.prepare(insertInto(TABLE_NAME)
- .value(MESSAGE_ID, bindMarker(MESSAGE_ID))
- .value(MAILBOX_ID, bindMarker(MAILBOX_ID))
- .value(IMAP_UID, bindMarker(IMAP_UID))
- .value(MOD_SEQ, bindMarker(MOD_SEQ))
- .value(ANSWERED, bindMarker(ANSWERED))
- .value(DELETED, bindMarker(DELETED))
- .value(DRAFT, bindMarker(DRAFT))
- .value(FLAGGED, bindMarker(FLAGGED))
- .value(RECENT, bindMarker(RECENT))
- .value(SEEN, bindMarker(SEEN))
- .value(USER, bindMarker(USER))
- .value(USER_FLAGS, bindMarker(USER_FLAGS))
- .ifNotExists());
+ Insert insert = insertInto(TABLE_NAME)
+ .value(MESSAGE_ID, bindMarker(MESSAGE_ID))
+ .value(MAILBOX_ID, bindMarker(MAILBOX_ID))
+ .value(IMAP_UID, bindMarker(IMAP_UID))
+ .value(MOD_SEQ, bindMarker(MOD_SEQ))
+ .value(ANSWERED, bindMarker(ANSWERED))
+ .value(DELETED, bindMarker(DELETED))
+ .value(DRAFT, bindMarker(DRAFT))
+ .value(FLAGGED, bindMarker(FLAGGED))
+ .value(RECENT, bindMarker(RECENT))
+ .value(SEEN, bindMarker(SEEN))
+ .value(USER, bindMarker(USER))
+ .value(USER_FLAGS, bindMarker(USER_FLAGS));
+ if (cassandraConfiguration.isMessageWriteStrongConsistency()) {
+ return session.prepare(insert.ifNotExists());
+ } else {
+ return session.prepare(insert);
+ }
}
private PreparedStatement prepareUpdate(Session session) {
- return session.prepare(update(TABLE_NAME)
- .with(set(MOD_SEQ, bindMarker(MOD_SEQ)))
- .and(set(ANSWERED, bindMarker(ANSWERED)))
- .and(set(DELETED, bindMarker(DELETED)))
- .and(set(DRAFT, bindMarker(DRAFT)))
- .and(set(FLAGGED, bindMarker(FLAGGED)))
- .and(set(RECENT, bindMarker(RECENT)))
- .and(set(SEEN, bindMarker(SEEN)))
- .and(set(USER, bindMarker(USER)))
- .and(set(USER_FLAGS, bindMarker(USER_FLAGS)))
- .where(eq(MESSAGE_ID, bindMarker(MESSAGE_ID)))
- .and(eq(MAILBOX_ID, bindMarker(MAILBOX_ID)))
- .and(eq(IMAP_UID, bindMarker(IMAP_UID)))
+ Update.Where update = update(TABLE_NAME)
+ .with(set(MOD_SEQ, bindMarker(MOD_SEQ)))
+ .and(set(ANSWERED, bindMarker(ANSWERED)))
+ .and(set(DELETED, bindMarker(DELETED)))
+ .and(set(DRAFT, bindMarker(DRAFT)))
+ .and(set(FLAGGED, bindMarker(FLAGGED)))
+ .and(set(RECENT, bindMarker(RECENT)))
+ .and(set(SEEN, bindMarker(SEEN)))
+ .and(set(USER, bindMarker(USER)))
+ .and(set(USER_FLAGS, bindMarker(USER_FLAGS)))
+ .where(eq(MESSAGE_ID, bindMarker(MESSAGE_ID)))
+ .and(eq(MAILBOX_ID, bindMarker(MAILBOX_ID)))
+ .and(eq(IMAP_UID, bindMarker(IMAP_UID)));
+
+ if (cassandraConfiguration.isMessageWriteStrongConsistency()) {
+ return session.prepare(update
.onlyIf(eq(MOD_SEQ, bindMarker(MOD_SEQ_CONDITION))));
+ } else {
+ return session.prepare(update);
+ }
}
private PreparedStatement prepareSelectAll(Session session) {
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/CassandraMailboxManagerTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/CassandraMailboxManagerTest.java
index 7012984..0acbb0c 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/CassandraMailboxManagerTest.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/CassandraMailboxManagerTest.java
@@ -836,7 +836,8 @@ public class CassandraMailboxManagerTest extends MailboxManagerTest<CassandraMai
return new CassandraMessageIdToImapUidDAO(
cassandraCluster.getConf(),
cassandra.getCassandraConsistenciesConfiguration(),
- new CassandraMessageId.Factory());
+ new CassandraMessageId.Factory(),
+ CassandraConfiguration.DEFAULT_CONFIGURATION);
}
private CassandraMessageDAOV3 messageDAO(CassandraCluster cassandraCluster) {
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapperTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapperTest.java
index 48d2a66..fc5885f 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapperTest.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapperTest.java
@@ -241,7 +241,8 @@ class CassandraMessageIdMapperTest extends MessageIdMapperTest {
CassandraMessageIdToImapUidDAO imapUidDAO = new CassandraMessageIdToImapUidDAO(
cassandra.getConf(),
cassandraCluster.getCassandraConsistenciesConfiguration(),
- new CassandraMessageId.Factory());
+ new CassandraMessageId.Factory(),
+ CassandraConfiguration.DEFAULT_CONFIGURATION);
SoftAssertions.assertSoftly(Throwing.consumer(softly -> {
softly.assertThat(sut.find(ImmutableList.of(message1.getMessageId()), MessageMapper.FetchType.Metadata))
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdToImapUidDAOTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdToImapUidDAOTest.java
index 9ce6eb9..6abb4b6 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdToImapUidDAOTest.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdToImapUidDAOTest.java
@@ -29,6 +29,7 @@ import javax.mail.Flags.Flag;
import org.apache.james.backends.cassandra.CassandraCluster;
import org.apache.james.backends.cassandra.CassandraClusterExtension;
import org.apache.james.backends.cassandra.components.CassandraModule;
+import org.apache.james.backends.cassandra.init.configuration.CassandraConfiguration;
import org.apache.james.backends.cassandra.versions.CassandraSchemaVersionModule;
import org.apache.james.mailbox.MessageUid;
import org.apache.james.mailbox.ModSeq;
@@ -63,7 +64,8 @@ class CassandraMessageIdToImapUidDAOTest {
testee = new CassandraMessageIdToImapUidDAO(
cassandra.getConf(),
cassandraCluster.getCassandraConsistenciesConfiguration(),
- messageIdFactory);
+ messageIdFactory,
+ CassandraConfiguration.DEFAULT_CONFIGURATION);
}
@Test
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapperTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapperTest.java
index 6965440..e113cb9 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapperTest.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapperTest.java
@@ -33,6 +33,7 @@ import org.apache.james.backends.cassandra.CassandraCluster;
import org.apache.james.backends.cassandra.CassandraClusterExtension;
import org.apache.james.backends.cassandra.StatementRecorder;
import org.apache.james.backends.cassandra.StatementRecorder.Selector;
+import org.apache.james.backends.cassandra.init.configuration.CassandraConfiguration;
import org.apache.james.mailbox.MessageManager;
import org.apache.james.mailbox.cassandra.ids.CassandraId;
import org.apache.james.mailbox.cassandra.ids.CassandraMessageId;
@@ -324,7 +325,8 @@ class CassandraMessageMapperTest extends MessageMapperTest {
CassandraMessageIdToImapUidDAO imapUidDAO = new CassandraMessageIdToImapUidDAO(
cassandra.getConf(),
cassandraCluster.getCassandraConsistenciesConfiguration(),
- new CassandraMessageId.Factory());
+ new CassandraMessageId.Factory(),
+ CassandraConfiguration.DEFAULT_CONFIGURATION);
SoftAssertions.assertSoftly(Throwing.consumer(softly -> {
softly.assertThat(messageMapper.findInMailbox(benwaInboxMailbox, MessageRange.all(), FetchType.Metadata, 1))
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/SolveMessageInconsistenciesServiceTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/SolveMessageInconsistenciesServiceTest.java
index 881822f..8273988 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/SolveMessageInconsistenciesServiceTest.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/SolveMessageInconsistenciesServiceTest.java
@@ -31,6 +31,7 @@ import org.apache.james.backends.cassandra.CassandraCluster;
import org.apache.james.backends.cassandra.CassandraClusterExtension;
import org.apache.james.backends.cassandra.Scenario;
import org.apache.james.backends.cassandra.components.CassandraModule;
+import org.apache.james.backends.cassandra.init.configuration.CassandraConfiguration;
import org.apache.james.backends.cassandra.versions.CassandraSchemaVersionModule;
import org.apache.james.junit.categories.Unstable;
import org.apache.james.mailbox.MessageUid;
@@ -104,7 +105,8 @@ public class SolveMessageInconsistenciesServiceTest {
imapUidDAO = new CassandraMessageIdToImapUidDAO(
cassandra.getConf(),
cassandraCluster.getCassandraConsistenciesConfiguration(),
- new CassandraMessageId.Factory());
+ new CassandraMessageId.Factory(),
+ CassandraConfiguration.DEFAULT_CONFIGURATION);
messageIdDAO = new CassandraMessageIdDAO(cassandra.getConf(), new CassandraMessageId.Factory());
testee = new SolveMessageInconsistenciesService(imapUidDAO, messageIdDAO);
}
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 01/12: JAMES-3435
s/ReadConsistency/ConsistencyChoice
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 43887c3f061e7cd39ab828a668b64fff5c0b9d88
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Sun Mar 28 14:29:45 2021 +0700
JAMES-3435 s/ReadConsistency/ConsistencyChoice
---
.../apache/james/mailbox/cassandra/DeleteMessageListener.java | 4 ++--
.../james/mailbox/cassandra/mail/CassandraMessageIdMapper.java | 8 ++++----
.../mailbox/cassandra/mail/CassandraMessageIdToImapUidDAO.java | 10 +++++-----
.../james/mailbox/cassandra/mail/CassandraMessageMapper.java | 2 +-
.../cassandra/mail/task/RecomputeMailboxCountersService.java | 2 +-
.../mail/task/SolveMessageInconsistenciesService.java | 2 +-
6 files changed, 14 insertions(+), 14 deletions(-)
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/DeleteMessageListener.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/DeleteMessageListener.java
index 728bd81..ddfe04b 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/DeleteMessageListener.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/DeleteMessageListener.java
@@ -239,13 +239,13 @@ public class DeleteMessageListener implements EventListener.GroupEventListener {
}
private Mono<Boolean> isReferenced(CassandraMessageId id) {
- return imapUidDAO.retrieve(id, ALL_MAILBOXES, CassandraMessageIdToImapUidDAO.ReadConsistency.STRONG)
+ return imapUidDAO.retrieve(id, ALL_MAILBOXES, CassandraMessageIdToImapUidDAO.ConsistencyChoice.STRONG)
.hasElements()
.map(negate());
}
private Mono<Boolean> isReferenced(CassandraMessageId id, CassandraId excludedId) {
- return imapUidDAO.retrieve(id, ALL_MAILBOXES, CassandraMessageIdToImapUidDAO.ReadConsistency.STRONG)
+ return imapUidDAO.retrieve(id, ALL_MAILBOXES, CassandraMessageIdToImapUidDAO.ConsistencyChoice.STRONG)
.filter(metadata -> !metadata.getComposedMessageId().getMailboxId().equals(excludedId))
.hasElements()
.map(negate());
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapper.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapper.java
index 3fae704..9cb0d5e 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapper.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapper.java
@@ -18,8 +18,8 @@
****************************************************************/
package org.apache.james.mailbox.cassandra.mail;
-import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ReadConsistency.STRONG;
-import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ReadConsistency.WEAK;
+import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ConsistencyChoice.STRONG;
+import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ConsistencyChoice.WEAK;
import static org.apache.james.util.ReactorUtils.DEFAULT_CONCURRENCY;
import java.time.Duration;
@@ -35,7 +35,7 @@ import org.apache.james.backends.cassandra.init.configuration.CassandraConfigura
import org.apache.james.mailbox.MessageManager;
import org.apache.james.mailbox.cassandra.ids.CassandraId;
import org.apache.james.mailbox.cassandra.ids.CassandraMessageId;
-import org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ReadConsistency;
+import org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ConsistencyChoice;
import org.apache.james.mailbox.exception.MailboxException;
import org.apache.james.mailbox.exception.MailboxNotFoundException;
import org.apache.james.mailbox.model.ComposedMessageId;
@@ -147,7 +147,7 @@ public class CassandraMessageIdMapper implements MessageIdMapper {
.block();
}
- public ReadConsistency chooseReadConsistency() {
+ public ConsistencyChoice chooseReadConsistency() {
if (cassandraConfiguration.isMessageReadStrongConsistency()) {
return STRONG;
} else {
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdToImapUidDAO.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdToImapUidDAO.java
index 2633fbf..0bb92ea 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdToImapUidDAO.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdToImapUidDAO.java
@@ -70,14 +70,14 @@ import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
public class CassandraMessageIdToImapUidDAO {
- public enum ReadConsistency {
+ public enum ConsistencyChoice {
STRONG(CassandraConsistenciesConfiguration::getLightweightTransaction),
WEAK(CassandraConsistenciesConfiguration::getRegular);
private final Function<CassandraConsistenciesConfiguration, ConsistencyLevel> consistencyLevelChoice;
- ReadConsistency(Function<CassandraConsistenciesConfiguration, ConsistencyLevel> consistencyLevelChoice) {
+ ConsistencyChoice(Function<CassandraConsistenciesConfiguration, ConsistencyLevel> consistencyLevelChoice) {
this.consistencyLevelChoice = consistencyLevelChoice;
}
@@ -213,16 +213,16 @@ public class CassandraMessageIdToImapUidDAO {
.setLong(MOD_SEQ_CONDITION, oldModSeq.asLong()));
}
- public Flux<ComposedMessageIdWithMetaData> retrieve(CassandraMessageId messageId, Optional<CassandraId> mailboxId, ReadConsistency readConsistency) {
+ public Flux<ComposedMessageIdWithMetaData> retrieve(CassandraMessageId messageId, Optional<CassandraId> mailboxId, ConsistencyChoice readConsistencyChoice) {
return cassandraAsyncExecutor.executeRows(
selectStatement(messageId, mailboxId)
- .setConsistencyLevel(readConsistency.choose(consistenciesConfiguration)))
+ .setConsistencyLevel(readConsistencyChoice.choose(consistenciesConfiguration)))
.map(this::toComposedMessageIdWithMetadata);
}
@VisibleForTesting
public Flux<ComposedMessageIdWithMetaData> retrieve(CassandraMessageId messageId, Optional<CassandraId> mailboxId) {
- return retrieve(messageId, mailboxId, ReadConsistency.STRONG);
+ return retrieve(messageId, mailboxId, ConsistencyChoice.STRONG);
}
public Flux<ComposedMessageIdWithMetaData> retrieveAllMessages() {
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapper.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapper.java
index 419b8f8..68a7e72 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapper.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapper.java
@@ -19,7 +19,7 @@
package org.apache.james.mailbox.cassandra.mail;
-import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ReadConsistency.STRONG;
+import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ConsistencyChoice.STRONG;
import static org.apache.james.util.ReactorUtils.DEFAULT_CONCURRENCY;
import java.security.SecureRandom;
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java
index aa88ef9..4245e78 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java
@@ -19,7 +19,7 @@
package org.apache.james.mailbox.cassandra.mail.task;
-import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ReadConsistency.STRONG;
+import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ConsistencyChoice.STRONG;
import java.util.Objects;
import java.util.Optional;
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/SolveMessageInconsistenciesService.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/SolveMessageInconsistenciesService.java
index facd5d6..a7ddc31 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/SolveMessageInconsistenciesService.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/SolveMessageInconsistenciesService.java
@@ -19,7 +19,7 @@
package org.apache.james.mailbox.cassandra.mail.task;
-import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ReadConsistency.STRONG;
+import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ConsistencyChoice.STRONG;
import static org.apache.james.util.ReactorUtils.publishIfPresent;
import java.time.Duration;
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 08/12: JAMES-3435 Serial read is not needed when
listUserMailboxes
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 5ae8147c1469bc2762cb8c836a016b995fa6f45e
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Sun Mar 28 22:48:09 2021 +0700
JAMES-3435 Serial read is not needed when listUserMailboxes
As it is not used in write transactions
---
.../mailbox/cassandra/mail/CassandraMailboxPathV3DAO.java | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxPathV3DAO.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxPathV3DAO.java
index c07c702..3a6d46b 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxPathV3DAO.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxPathV3DAO.java
@@ -46,7 +46,6 @@ import org.apache.james.mailbox.model.UidValidity;
import org.apache.james.util.FunctionalUtils;
import org.apache.james.util.ReactorUtils;
-import com.datastax.driver.core.ConsistencyLevel;
import com.datastax.driver.core.PreparedStatement;
import com.datastax.driver.core.Row;
import com.datastax.driver.core.Session;
@@ -63,13 +62,13 @@ public class CassandraMailboxPathV3DAO {
private final PreparedStatement select;
private final PreparedStatement selectUser;
private final PreparedStatement selectAll;
- private final ConsistencyLevel consistencyLevel;
+ private final CassandraConsistenciesConfiguration consistenciesConfiguration;
@Inject
public CassandraMailboxPathV3DAO(Session session, CassandraUtils cassandraUtils,
CassandraConsistenciesConfiguration consistenciesConfiguration) {
this.cassandraAsyncExecutor = new CassandraAsyncExecutor(session);
- this.consistencyLevel = consistenciesConfiguration.getLightweightTransaction();
+ this.consistenciesConfiguration = consistenciesConfiguration;
this.cassandraUtils = cassandraUtils;
this.insert = prepareInsert(session);
this.delete = prepareDelete(session);
@@ -123,7 +122,7 @@ public class CassandraMailboxPathV3DAO {
.setString(NAMESPACE, mailboxPath.getNamespace())
.setString(USER, sanitizeUser(mailboxPath.getUser()))
.setString(MAILBOX_NAME, mailboxPath.getName())
- .setConsistencyLevel(consistencyLevel))
+ .setConsistencyLevel(consistenciesConfiguration.getLightweightTransaction()))
.map(this::fromRowToCassandraIdAndPath)
.map(FunctionalUtils.toFunction(this::logGhostMailboxSuccess))
.switchIfEmpty(ReactorUtils.executeAndEmpty(() -> logGhostMailboxFailure(mailboxPath)));
@@ -133,8 +132,7 @@ public class CassandraMailboxPathV3DAO {
return cassandraAsyncExecutor.execute(
selectUser.bind()
.setString(NAMESPACE, namespace)
- .setString(USER, sanitizeUser(user))
- .setConsistencyLevel(consistencyLevel))
+ .setString(USER, sanitizeUser(user)))
.flatMapMany(cassandraUtils::convertToFlux)
.map(this::fromRowToCassandraIdAndPath)
.map(FunctionalUtils.toFunction(this::logReadSuccess));
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 06/12: JAMES-3435 Tests for relaxed consistency
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 2842ea88b21e24d5988ee09c35cc549e82d6b8e9
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Sun Mar 28 20:33:36 2021 +0700
JAMES-3435 Tests for relaxed consistency
---
.../cassandra/mail/CassandraMapperProvider.java | 12 ++--
...andraMessageIdMapperRelaxedConsistencyTest.java | 72 +++++++++++++++++++++
.../mail/CassandraMessageIdMapperTest.java | 3 +-
...ssandraMessageMapperRelaxedConsistencyTest.java | 73 ++++++++++++++++++++++
.../cassandra/mail/CassandraMessageMapperTest.java | 3 +-
.../cassandra/mail/CassandraMessageMoveTest.java | 4 +-
.../CassandraMessageWithAttachmentMapperTest.java | 4 +-
.../task/RecomputeMailboxCountersServiceTest.java | 4 +-
.../store/mail/model/MessageIdMapperTest.java | 4 +-
.../store/mail/model/MessageMapperTest.java | 4 +-
10 files changed, 169 insertions(+), 14 deletions(-)
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMapperProvider.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMapperProvider.java
index ca5a756..db279c3 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMapperProvider.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMapperProvider.java
@@ -56,14 +56,15 @@ public class CassandraMapperProvider implements MapperProvider {
private CassandraMailboxSessionMapperFactory mapperFactory;
public CassandraMapperProvider(CassandraCluster cassandra,
- CassandraConsistenciesConfiguration cassandraConsistenciesConfiguration) {
+ CassandraConsistenciesConfiguration cassandraConsistenciesConfiguration,
+ CassandraConfiguration cassandraConfiguration) {
this.cassandra = cassandra;
messageUidProvider = new MessageUidProvider();
cassandraModSeqProvider = new CassandraModSeqProvider(
this.cassandra.getConf(),
- CassandraConfiguration.DEFAULT_CONFIGURATION,
+ cassandraConfiguration,
cassandraConsistenciesConfiguration);
- mapperFactory = createMapperFactory();
+ mapperFactory = createMapperFactory(cassandraConfiguration);
}
@Override
@@ -86,9 +87,10 @@ public class CassandraMapperProvider implements MapperProvider {
return mapperFactory.getMessageIdMapper(mailboxSession);
}
- private CassandraMailboxSessionMapperFactory createMapperFactory() {
+ private CassandraMailboxSessionMapperFactory createMapperFactory(CassandraConfiguration cassandraConfiguration) {
return TestCassandraMailboxSessionMapperFactory.forTests(cassandra,
- new CassandraMessageId.Factory());
+ new CassandraMessageId.Factory(),
+ cassandraConfiguration);
}
@Override
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapperRelaxedConsistencyTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapperRelaxedConsistencyTest.java
new file mode 100644
index 0000000..48f58e0
--- /dev/null
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapperRelaxedConsistencyTest.java
@@ -0,0 +1,72 @@
+/****************************************************************
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information *
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the *
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the *
+ * specific language governing permissions and limitations *
+ * under the License. *
+ ****************************************************************/
+
+package org.apache.james.mailbox.cassandra.mail;
+
+import org.apache.james.backends.cassandra.CassandraClusterExtension;
+import org.apache.james.backends.cassandra.init.configuration.CassandraConfiguration;
+import org.apache.james.mailbox.store.mail.model.MessageIdMapperTest;
+import org.junit.jupiter.api.Disabled;
+import org.junit.jupiter.api.Nested;
+import org.junit.jupiter.api.extension.RegisterExtension;
+
+class CassandraMessageIdMapperRelaxedConsistencyTest {
+ @RegisterExtension
+ static CassandraClusterExtension cassandraCluster = new CassandraClusterExtension(MailboxAggregateModule.MODULE);
+
+ @Nested
+ class WeakReadConsistency extends MessageIdMapperTest {
+ @Override
+ protected CassandraMapperProvider provideMapper() {
+ return new CassandraMapperProvider(
+ cassandraCluster.getCassandraCluster(),
+ cassandraCluster.getCassandraConsistenciesConfiguration(),
+ CassandraConfiguration.builder()
+ .messageReadStrongConsistency(false)
+ .messageWriteStrongConsistency(true)
+ .build());
+ }
+ }
+
+ @Nested
+ class WeakWriteConsistency extends MessageIdMapperTest {
+ @Override
+ protected CassandraMapperProvider provideMapper() {
+ return new CassandraMapperProvider(
+ cassandraCluster.getCassandraCluster(),
+ cassandraCluster.getCassandraConsistenciesConfiguration(),
+ CassandraConfiguration.builder()
+ .messageReadStrongConsistency(false)
+ .messageWriteStrongConsistency(false)
+ .build());
+ }
+
+ @Disabled("JAMES-3435 Without strong consistency flags update is not thread safe as long as it follows a read-before-write pattern")
+ @Override
+ public void setFlagsShouldWorkWithConcurrencyWithAdd() throws Exception {
+ super.setFlagsShouldWorkWithConcurrencyWithAdd();
+ }
+
+ @Disabled("JAMES-3435 Without strong consistency flags update is not thread safe as long as it follows a read-before-write pattern")
+ @Override
+ public void setFlagsShouldWorkWithConcurrencyWithRemove() throws Exception {
+ super.setFlagsShouldWorkWithConcurrencyWithRemove();
+ }
+ }
+}
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapperTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapperTest.java
index fc5885f..85dd03d 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapperTest.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapperTest.java
@@ -64,7 +64,8 @@ class CassandraMessageIdMapperTest extends MessageIdMapperTest {
protected CassandraMapperProvider provideMapper() {
return new CassandraMapperProvider(
cassandraCluster.getCassandraCluster(),
- cassandraCluster.getCassandraConsistenciesConfiguration());
+ cassandraCluster.getCassandraConsistenciesConfiguration(),
+ CassandraConfiguration.DEFAULT_CONFIGURATION);
}
@Test
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapperRelaxedConsistencyTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapperRelaxedConsistencyTest.java
new file mode 100644
index 0000000..7850d1c
--- /dev/null
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapperRelaxedConsistencyTest.java
@@ -0,0 +1,73 @@
+/****************************************************************
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information *
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the *
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the *
+ * specific language governing permissions and limitations *
+ * under the License. *
+ ****************************************************************/
+
+package org.apache.james.mailbox.cassandra.mail;
+
+import org.apache.james.backends.cassandra.CassandraClusterExtension;
+import org.apache.james.backends.cassandra.init.configuration.CassandraConfiguration;
+import org.apache.james.mailbox.store.mail.model.MapperProvider;
+import org.apache.james.mailbox.store.mail.model.MessageMapperTest;
+import org.junit.jupiter.api.Disabled;
+import org.junit.jupiter.api.Nested;
+import org.junit.jupiter.api.extension.RegisterExtension;
+
+class CassandraMessageMapperRelaxedConsistencyTest {
+ @RegisterExtension
+ static CassandraClusterExtension cassandraCluster = new CassandraClusterExtension(MailboxAggregateModule.MODULE);
+
+ @Nested
+ class WeakReadConsistency extends MessageMapperTest {
+ @Override
+ protected MapperProvider createMapperProvider() {
+ return new CassandraMapperProvider(
+ cassandraCluster.getCassandraCluster(),
+ cassandraCluster.getCassandraConsistenciesConfiguration(),
+ CassandraConfiguration.builder()
+ .messageReadStrongConsistency(false)
+ .messageWriteStrongConsistency(true)
+ .build());
+ }
+ }
+
+ @Nested
+ class WeakWriteConsistency extends MessageMapperTest {
+ @Override
+ protected MapperProvider createMapperProvider() {
+ return new CassandraMapperProvider(
+ cassandraCluster.getCassandraCluster(),
+ cassandraCluster.getCassandraConsistenciesConfiguration(),
+ CassandraConfiguration.builder()
+ .messageReadStrongConsistency(false)
+ .messageWriteStrongConsistency(false)
+ .build());
+ }
+
+ @Disabled("JAMES-3435 Without strong consistency flags update is not thread safe as long as it follows a read-before-write pattern")
+ @Override
+ public void setFlagsShouldWorkWithConcurrencyWithRemove() throws Exception {
+ super.setFlagsShouldWorkWithConcurrencyWithRemove();
+ }
+
+ @Disabled("JAMES-3435 Without strong consistency flags update is not thread safe as long as it follows a read-before-write pattern")
+ @Override
+ public void userFlagsUpdateShouldWorkInConcurrentEnvironment() throws Exception {
+ super.userFlagsUpdateShouldWorkInConcurrentEnvironment();
+ }
+ }
+}
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapperTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapperTest.java
index e113cb9..2734fd7 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapperTest.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapperTest.java
@@ -62,7 +62,8 @@ class CassandraMessageMapperTest extends MessageMapperTest {
protected MapperProvider createMapperProvider() {
return new CassandraMapperProvider(
cassandraCluster.getCassandraCluster(),
- cassandraCluster.getCassandraConsistenciesConfiguration());
+ cassandraCluster.getCassandraConsistenciesConfiguration(),
+ CassandraConfiguration.DEFAULT_CONFIGURATION);
}
@Nested
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMoveTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMoveTest.java
index f6fb9c1..2b00db0 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMoveTest.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMoveTest.java
@@ -20,6 +20,7 @@
package org.apache.james.mailbox.cassandra.mail;
import org.apache.james.backends.cassandra.CassandraClusterExtension;
+import org.apache.james.backends.cassandra.init.configuration.CassandraConfiguration;
import org.apache.james.mailbox.store.mail.model.MapperProvider;
import org.apache.james.mailbox.store.mail.model.MessageMoveTest;
import org.junit.jupiter.api.extension.RegisterExtension;
@@ -32,6 +33,7 @@ class CassandraMessageMoveTest extends MessageMoveTest {
protected MapperProvider createMapperProvider() {
return new CassandraMapperProvider(
cassandraCluster.getCassandraCluster(),
- cassandraCluster.getCassandraConsistenciesConfiguration());
+ cassandraCluster.getCassandraConsistenciesConfiguration(),
+ CassandraConfiguration.DEFAULT_CONFIGURATION);
}
}
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageWithAttachmentMapperTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageWithAttachmentMapperTest.java
index 157e3ec..cc5fb23 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageWithAttachmentMapperTest.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageWithAttachmentMapperTest.java
@@ -20,6 +20,7 @@
package org.apache.james.mailbox.cassandra.mail;
import org.apache.james.backends.cassandra.CassandraClusterExtension;
+import org.apache.james.backends.cassandra.init.configuration.CassandraConfiguration;
import org.apache.james.mailbox.store.mail.model.MapperProvider;
import org.apache.james.mailbox.store.mail.model.MessageWithAttachmentMapperTest;
import org.junit.jupiter.api.extension.RegisterExtension;
@@ -32,6 +33,7 @@ class CassandraMessageWithAttachmentMapperTest extends MessageWithAttachmentMapp
protected MapperProvider createMapperProvider() {
return new CassandraMapperProvider(
cassandraCluster.getCassandraCluster(),
- cassandraCluster.getCassandraConsistenciesConfiguration());
+ cassandraCluster.getCassandraConsistenciesConfiguration(),
+ CassandraConfiguration.DEFAULT_CONFIGURATION);
}
}
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersServiceTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersServiceTest.java
index 8023f11..0813247 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersServiceTest.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersServiceTest.java
@@ -28,6 +28,7 @@ import javax.mail.Flags;
import org.apache.james.backends.cassandra.CassandraCluster;
import org.apache.james.backends.cassandra.CassandraClusterExtension;
import org.apache.james.backends.cassandra.components.CassandraModule;
+import org.apache.james.backends.cassandra.init.configuration.CassandraConfiguration;
import org.apache.james.backends.cassandra.versions.CassandraSchemaVersionModule;
import org.apache.james.core.Username;
import org.apache.james.mailbox.MessageUid;
@@ -95,7 +96,8 @@ class RecomputeMailboxCountersServiceTest {
messageIdToImapUidDAO = new CassandraMessageIdToImapUidDAO(
cassandra.getConf(),
cassandraCluster.getCassandraConsistenciesConfiguration(),
- MESSAGE_ID_FACTORY);
+ MESSAGE_ID_FACTORY,
+ CassandraConfiguration.DEFAULT_CONFIGURATION);
counterDAO = new CassandraMailboxCounterDAO(cassandra.getConf());
testee = new RecomputeMailboxCountersService(mailboxDAO, imapUidToMessageIdDAO, messageIdToImapUidDAO, counterDAO);
}
diff --git a/mailbox/store/src/test/java/org/apache/james/mailbox/store/mail/model/MessageIdMapperTest.java b/mailbox/store/src/test/java/org/apache/james/mailbox/store/mail/model/MessageIdMapperTest.java
index 82f20c8..e30a2ac 100644
--- a/mailbox/store/src/test/java/org/apache/james/mailbox/store/mail/model/MessageIdMapperTest.java
+++ b/mailbox/store/src/test/java/org/apache/james/mailbox/store/mail/model/MessageIdMapperTest.java
@@ -661,7 +661,7 @@ public abstract class MessageIdMapperTest {
}
@Test
- void setFlagsShouldWorkWithConcurrencyWithAdd() throws Exception {
+ public void setFlagsShouldWorkWithConcurrencyWithAdd() throws Exception {
Assume.assumeTrue(mapperProvider.getSupportedCapabilities().contains(MapperProvider.Capabilities.THREAD_SAFE_FLAGS_UPDATE));
message1.setUid(mapperProvider.generateMessageUid());
message1.setModSeq(mapperProvider.generateModSeq(benwaInboxMailbox));
@@ -684,7 +684,7 @@ public abstract class MessageIdMapperTest {
}
@Test
- void setFlagsShouldWorkWithConcurrencyWithRemove() throws Exception {
+ public void setFlagsShouldWorkWithConcurrencyWithRemove() throws Exception {
Assume.assumeTrue(mapperProvider.getSupportedCapabilities().contains(MapperProvider.Capabilities.THREAD_SAFE_FLAGS_UPDATE));
message1.setUid(mapperProvider.generateMessageUid());
message1.setModSeq(mapperProvider.generateModSeq(benwaInboxMailbox));
diff --git a/mailbox/store/src/test/java/org/apache/james/mailbox/store/mail/model/MessageMapperTest.java b/mailbox/store/src/test/java/org/apache/james/mailbox/store/mail/model/MessageMapperTest.java
index b790dc4..eec632a 100644
--- a/mailbox/store/src/test/java/org/apache/james/mailbox/store/mail/model/MessageMapperTest.java
+++ b/mailbox/store/src/test/java/org/apache/james/mailbox/store/mail/model/MessageMapperTest.java
@@ -910,7 +910,7 @@ public abstract class MessageMapperTest {
}
@Test
- void userFlagsUpdateShouldWorkInConcurrentEnvironment() throws Exception {
+ public void userFlagsUpdateShouldWorkInConcurrentEnvironment() throws Exception {
Assume.assumeTrue(mapperProvider.getSupportedCapabilities().contains(MapperProvider.Capabilities.THREAD_SAFE_FLAGS_UPDATE));
saveMessages();
@@ -931,7 +931,7 @@ public abstract class MessageMapperTest {
}
@Test
- void setFlagsShouldWorkWithConcurrencyWithRemove() throws Exception {
+ public void setFlagsShouldWorkWithConcurrencyWithRemove() throws Exception {
Assume.assumeTrue(mapperProvider.getSupportedCapabilities().contains(MapperProvider.Capabilities.THREAD_SAFE_FLAGS_UPDATE));
saveMessages();
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 05/12: JAMES-3435 Document
message.write.strong.consistency.unsafe
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit e13928232df7a66b9e057f76c642766a13ff122f
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Sun Mar 28 16:28:17 2021 +0700
JAMES-3435 Document message.write.strong.consistency.unsafe
---
docs/modules/servers/pages/distributed/configure/cassandra.adoc | 7 +++++++
src/site/xdoc/server/config-cassandra.xml | 7 +++++++
2 files changed, 14 insertions(+)
diff --git a/docs/modules/servers/pages/distributed/configure/cassandra.adoc b/docs/modules/servers/pages/distributed/configure/cassandra.adoc
index 019b14b..b9810d8 100644
--- a/docs/modules/servers/pages/distributed/configure/cassandra.adoc
+++ b/docs/modules/servers/pages/distributed/configure/cassandra.adoc
@@ -191,4 +191,11 @@ If enabled, regular consistency level is used for read transactions for message.
in stale reads as the system.paxos table will not be checked for latest updates. Better performance are expected
by turning it off. Note that reads performed as part of write transactions are always performed with a strong
consistency.
+
+| message.write.strong.consistency.unsafe
+| Optional. Boolean, defaults to true. Disabling should be considered experimental and unsafe.
+If enabled, Lightweight transactions will no longer be used upon messages operation (table `imapUidTable`).
+As message flags updates relies so far on a read-before-write model, it exposes yourself to data races leading to
+potentially update loss. Better performance are expected
+by turning it off. Reads performed as part of write transaction are also performed with a relaxed consistency.
|===
\ No newline at end of file
diff --git a/src/site/xdoc/server/config-cassandra.xml b/src/site/xdoc/server/config-cassandra.xml
index 6271984..904247f 100644
--- a/src/site/xdoc/server/config-cassandra.xml
+++ b/src/site/xdoc/server/config-cassandra.xml
@@ -156,6 +156,13 @@
by turning it off. Note that reads performed as part of write transactions are always performed with a strong
consistency.</dd>
+ <dt><strong>message.write.strong.consistency.unsafe</strong></dt>
+ <dd>Optional. Boolean, defaults to true. Disabling should be considered experimental and unsafe.
+ If enabled, Lightweight transactions will no longer be used upon messages operation (table `imapUidTable`).
+ As message flags updates relies so far on a read-before-write model, it exposes yourself to data races leading to
+ potentially update loss. Better performance are expected
+ by turning it off. Reads performed as part of write transaction are also performed with a relaxed consistency.</dd>
+
<dt><strong>Allows specifying the driver default consistency level.</strong></dt>
<dt><strong>cassandra.consistency_level.regular</strong></dt>
<dd>Optional. Defaults to QUORUM.<br/> <a href="https://docs.datastax.com/en/cassandra-oss/3.x/cassandra/dml/dmlConfigConsistency.html">QUORUM, LOCAL_QUORUM, or EACH_QUORUM</a>.</dd>
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 10/12: JAMES-3435 Factorize consistency choices
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 14ec0f30507648902c221f6e38a4d9e9d341c6f3
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Sun Mar 28 23:05:22 2021 +0700
JAMES-3435 Factorize consistency choices
As choosing SERIAL or QUORUM upon reads is a
shared concern between mailbox and message
---
.../CassandraConsistenciesConfiguration.java | 16 ++++++++++++++++
.../mailbox/cassandra/DeleteMessageListener.java | 7 ++++---
.../cassandra/mail/CassandraMailboxMapper.java | 7 ++++---
.../cassandra/mail/CassandraMailboxPathV3DAO.java | 19 +------------------
.../cassandra/mail/CassandraMessageIdMapper.java | 8 ++++----
.../mail/CassandraMessageIdToImapUidDAO.java | 19 +------------------
.../cassandra/mail/CassandraMessageMapper.java | 7 ++++---
.../mail/task/RecomputeMailboxCountersService.java | 2 +-
.../mail/task/SolveMailboxInconsistenciesService.java | 2 +-
.../mail/task/SolveMessageInconsistenciesService.java | 2 +-
10 files changed, 37 insertions(+), 52 deletions(-)
diff --git a/backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/CassandraConsistenciesConfiguration.java b/backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/CassandraConsistenciesConfiguration.java
index 8a70b45..a65620c 100644
--- a/backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/CassandraConsistenciesConfiguration.java
+++ b/backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/CassandraConsistenciesConfiguration.java
@@ -20,11 +20,27 @@
package org.apache.james.backends.cassandra.init.configuration;
import java.util.Objects;
+import java.util.function.Function;
import com.datastax.driver.core.ConsistencyLevel;
import com.google.common.base.MoreObjects;
public class CassandraConsistenciesConfiguration {
+ public enum ConsistencyChoice {
+ WEAK(CassandraConsistenciesConfiguration::getRegular),
+ STRONG(CassandraConsistenciesConfiguration::getLightweightTransaction);
+
+ private final Function<CassandraConsistenciesConfiguration, ConsistencyLevel> choice;
+
+ ConsistencyChoice(Function<CassandraConsistenciesConfiguration, ConsistencyLevel> choice) {
+ this.choice = choice;
+ }
+
+ public ConsistencyLevel choose(CassandraConsistenciesConfiguration configuration) {
+ return choice.apply(configuration);
+ }
+ }
+
public static final CassandraConsistenciesConfiguration DEFAULT = new CassandraConsistenciesConfiguration(ConsistencyLevel.QUORUM, ConsistencyLevel.SERIAL);
public static ConsistencyLevel fromString(String value) {
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/DeleteMessageListener.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/DeleteMessageListener.java
index d48623b..2deefad 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/DeleteMessageListener.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/DeleteMessageListener.java
@@ -19,8 +19,9 @@
package org.apache.james.mailbox.cassandra;
-import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ConsistencyChoice.STRONG;
-import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ConsistencyChoice.WEAK;
+import static org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration.ConsistencyChoice;
+import static org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration.ConsistencyChoice.STRONG;
+import static org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration.ConsistencyChoice.WEAK;
import static org.apache.james.util.FunctionalUtils.negate;
import static org.apache.james.util.ReactorUtils.DEFAULT_CONCURRENCY;
@@ -258,7 +259,7 @@ public class DeleteMessageListener implements EventListener.GroupEventListener {
.map(negate());
}
- private CassandraMessageIdToImapUidDAO.ConsistencyChoice chooseReadConsistencyUponWrites() {
+ private ConsistencyChoice chooseReadConsistencyUponWrites() {
if (cassandraConfiguration.isMessageWriteStrongConsistency()) {
return STRONG;
}
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxMapper.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxMapper.java
index 4ee3a75..d372af7 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxMapper.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxMapper.java
@@ -19,8 +19,8 @@
package org.apache.james.mailbox.cassandra.mail;
-import static org.apache.james.mailbox.cassandra.mail.CassandraMailboxPathV3DAO.ConsistencyChoice.STRONG;
-import static org.apache.james.mailbox.cassandra.mail.CassandraMailboxPathV3DAO.ConsistencyChoice.WEAK;
+import static org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration.ConsistencyChoice.STRONG;
+import static org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration.ConsistencyChoice.WEAK;
import java.security.SecureRandom;
import java.time.Duration;
@@ -29,6 +29,7 @@ import javax.inject.Inject;
import org.apache.commons.lang3.tuple.Pair;
import org.apache.james.backends.cassandra.init.configuration.CassandraConfiguration;
+import org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration;
import org.apache.james.backends.cassandra.versions.CassandraSchemaVersionManager;
import org.apache.james.backends.cassandra.versions.SchemaVersion;
import org.apache.james.core.Username;
@@ -120,7 +121,7 @@ public class CassandraMailboxMapper implements MailboxMapper {
return mailboxPathV3DAO.retrieve(path, consistencyChoice());
}
- private CassandraMailboxPathV3DAO.ConsistencyChoice consistencyChoice() {
+ private CassandraConsistenciesConfiguration.ConsistencyChoice consistencyChoice() {
if (cassandraConfiguration.isMailboxReadStrongConsistency()) {
return STRONG;
}
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxPathV3DAO.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxPathV3DAO.java
index af34896..1228691 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxPathV3DAO.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxPathV3DAO.java
@@ -32,11 +32,10 @@ import static org.apache.james.mailbox.cassandra.table.CassandraMailboxPathV3Tab
import static org.apache.james.mailbox.cassandra.table.CassandraMailboxPathV3Table.UIDVALIDITY;
import static org.apache.james.mailbox.cassandra.table.CassandraMailboxPathV3Table.USER;
-import java.util.function.Function;
-
import javax.inject.Inject;
import org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration;
+import org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration.ConsistencyChoice;
import org.apache.james.backends.cassandra.utils.CassandraAsyncExecutor;
import org.apache.james.backends.cassandra.utils.CassandraUtils;
import org.apache.james.core.Username;
@@ -58,22 +57,6 @@ import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
public class CassandraMailboxPathV3DAO {
- // todo factorize me in CassandraConsistenciesConfiguration
- public enum ConsistencyChoice {
- WEAK(CassandraConsistenciesConfiguration::getRegular),
- STRONG(CassandraConsistenciesConfiguration::getLightweightTransaction);
-
- private final Function<CassandraConsistenciesConfiguration, ConsistencyLevel> choice;
-
- ConsistencyChoice(Function<CassandraConsistenciesConfiguration, ConsistencyLevel> choice) {
- this.choice = choice;
- }
-
- public ConsistencyLevel choose(CassandraConsistenciesConfiguration configuration) {
- return choice.apply(configuration);
- }
- }
-
private final CassandraAsyncExecutor cassandraAsyncExecutor;
private final CassandraUtils cassandraUtils;
private final PreparedStatement delete;
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapper.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapper.java
index 1b5f58e..1224932 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapper.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdMapper.java
@@ -18,8 +18,8 @@
****************************************************************/
package org.apache.james.mailbox.cassandra.mail;
-import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ConsistencyChoice.STRONG;
-import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ConsistencyChoice.WEAK;
+import static org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration.ConsistencyChoice.STRONG;
+import static org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration.ConsistencyChoice.WEAK;
import static org.apache.james.util.ReactorUtils.DEFAULT_CONCURRENCY;
import java.time.Duration;
@@ -32,10 +32,10 @@ import javax.mail.Flags;
import org.apache.commons.lang3.tuple.Pair;
import org.apache.james.backends.cassandra.init.configuration.CassandraConfiguration;
+import org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration.ConsistencyChoice;
import org.apache.james.mailbox.MessageManager;
import org.apache.james.mailbox.cassandra.ids.CassandraId;
import org.apache.james.mailbox.cassandra.ids.CassandraMessageId;
-import org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ConsistencyChoice;
import org.apache.james.mailbox.exception.MailboxException;
import org.apache.james.mailbox.exception.MailboxNotFoundException;
import org.apache.james.mailbox.model.ComposedMessageId;
@@ -155,7 +155,7 @@ public class CassandraMessageIdMapper implements MessageIdMapper {
}
}
- private CassandraMessageIdToImapUidDAO.ConsistencyChoice chooseReadConsistencyUponWrites() {
+ private ConsistencyChoice chooseReadConsistencyUponWrites() {
if (cassandraConfiguration.isMessageWriteStrongConsistency()) {
return STRONG;
}
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdToImapUidDAO.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdToImapUidDAO.java
index 48f29d3..470b477 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdToImapUidDAO.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdToImapUidDAO.java
@@ -41,7 +41,6 @@ import static org.apache.james.mailbox.cassandra.table.MessageIdToImapUid.MOD_SE
import static org.apache.james.mailbox.cassandra.table.MessageIdToImapUid.TABLE_NAME;
import java.util.Optional;
-import java.util.function.Function;
import javax.inject.Inject;
import javax.mail.Flags;
@@ -49,6 +48,7 @@ import javax.mail.Flags.Flag;
import org.apache.james.backends.cassandra.init.configuration.CassandraConfiguration;
import org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration;
+import org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration.ConsistencyChoice;
import org.apache.james.backends.cassandra.utils.CassandraAsyncExecutor;
import org.apache.james.mailbox.MessageUid;
import org.apache.james.mailbox.ModSeq;
@@ -59,7 +59,6 @@ import org.apache.james.mailbox.model.ComposedMessageId;
import org.apache.james.mailbox.model.ComposedMessageIdWithMetaData;
import com.datastax.driver.core.BoundStatement;
-import com.datastax.driver.core.ConsistencyLevel;
import com.datastax.driver.core.PreparedStatement;
import com.datastax.driver.core.Row;
import com.datastax.driver.core.Session;
@@ -74,22 +73,6 @@ import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
public class CassandraMessageIdToImapUidDAO {
- public enum ConsistencyChoice {
- STRONG(CassandraConsistenciesConfiguration::getLightweightTransaction),
- WEAK(CassandraConsistenciesConfiguration::getRegular);
-
- private final Function<CassandraConsistenciesConfiguration, ConsistencyLevel> consistencyLevelChoice;
-
-
- ConsistencyChoice(Function<CassandraConsistenciesConfiguration, ConsistencyLevel> consistencyLevelChoice) {
- this.consistencyLevelChoice = consistencyLevelChoice;
- }
-
- public ConsistencyLevel choose(CassandraConsistenciesConfiguration configuration) {
- return consistencyLevelChoice.apply(configuration);
- }
- }
-
private static final String MOD_SEQ_CONDITION = "modSeqCondition";
private final CassandraAsyncExecutor cassandraAsyncExecutor;
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapper.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapper.java
index faa3464..0aa353a 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapper.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapper.java
@@ -19,8 +19,9 @@
package org.apache.james.mailbox.cassandra.mail;
-import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ConsistencyChoice.STRONG;
-import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ConsistencyChoice.WEAK;
+import static org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration.ConsistencyChoice;
+import static org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration.ConsistencyChoice.STRONG;
+import static org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration.ConsistencyChoice.WEAK;
import static org.apache.james.util.ReactorUtils.DEFAULT_CONCURRENCY;
import java.security.SecureRandom;
@@ -418,7 +419,7 @@ public class CassandraMessageMapper implements MessageMapper {
}
}
- private CassandraMessageIdToImapUidDAO.ConsistencyChoice chooseReadConsistencyUponWrites() {
+ private ConsistencyChoice chooseReadConsistencyUponWrites() {
if (cassandraConfiguration.isMessageWriteStrongConsistency()) {
return STRONG;
}
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java
index 4245e78..62ff851 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java
@@ -19,7 +19,7 @@
package org.apache.james.mailbox.cassandra.mail.task;
-import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ConsistencyChoice.STRONG;
+import static org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration.ConsistencyChoice.STRONG;
import java.util.Objects;
import java.util.Optional;
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/SolveMailboxInconsistenciesService.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/SolveMailboxInconsistenciesService.java
index 5f5f695..2643acc 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/SolveMailboxInconsistenciesService.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/SolveMailboxInconsistenciesService.java
@@ -19,7 +19,7 @@
package org.apache.james.mailbox.cassandra.mail.task;
-import static org.apache.james.mailbox.cassandra.mail.CassandraMailboxPathV3DAO.ConsistencyChoice.STRONG;
+import static org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration.ConsistencyChoice.STRONG;
import static org.apache.james.util.ReactorUtils.DEFAULT_CONCURRENCY;
import java.util.Collection;
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/SolveMessageInconsistenciesService.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/SolveMessageInconsistenciesService.java
index a7ddc31..45564b8 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/SolveMessageInconsistenciesService.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/SolveMessageInconsistenciesService.java
@@ -19,7 +19,7 @@
package org.apache.james.mailbox.cassandra.mail.task;
-import static org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO.ConsistencyChoice.STRONG;
+import static org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration.ConsistencyChoice.STRONG;
import static org.apache.james.util.ReactorUtils.publishIfPresent;
import java.time.Duration;
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 12/12: JAMES-3435 test for mailbox.read.strong.cst
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 4619763a6641a5621991ac28e2ce7b0cc38e1f49
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Mon Mar 29 09:33:06 2021 +0700
JAMES-3435 test for mailbox.read.strong.cst
---
.../init/configuration/CassandraConfiguration.java | 6 +++++-
.../mail/CassandraMailboxMapperGenericTest.java | 23 ++++++++++++++++++++++
2 files changed, 28 insertions(+), 1 deletion(-)
diff --git a/backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/CassandraConfiguration.java b/backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/CassandraConfiguration.java
index 41284c7..0519b13 100644
--- a/backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/CassandraConfiguration.java
+++ b/backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/CassandraConfiguration.java
@@ -535,8 +535,10 @@ public class CassandraConfiguration {
&& Objects.equals(this.fetchNextPageInAdvanceRow, that.fetchNextPageInAdvanceRow)
&& Objects.equals(this.blobPartSize, that.blobPartSize)
&& Objects.equals(this.attachmentV2MigrationReadTimeout, that.attachmentV2MigrationReadTimeout)
+ && Objects.equals(this.mailboxReadStrongConsistency, that.mailboxReadStrongConsistency)
&& Objects.equals(this.messageAttachmentIdsReadTimeout, that.messageAttachmentIdsReadTimeout)
&& Objects.equals(this.messageReadStrongConsistency, that.messageReadStrongConsistency)
+ && Objects.equals(this.messageWriteStrongConsistency, that.messageWriteStrongConsistency)
&& Objects.equals(this.consistencyLevelRegular, that.consistencyLevelRegular)
&& Objects.equals(this.consistencyLevelLightweightTransaction, that.consistencyLevelLightweightTransaction);
}
@@ -550,7 +552,7 @@ public class CassandraConfiguration {
mailboxCountersReadRepairChanceOneHundred, mailboxCountersReadRepairChanceMax,
blobPartSize, attachmentV2MigrationReadTimeout, messageAttachmentIdsReadTimeout,
consistencyLevelRegular, consistencyLevelLightweightTransaction, mailboxReadRepair,
- messageReadStrongConsistency);
+ messageReadStrongConsistency, mailboxReadStrongConsistency, messageWriteStrongConsistency);
}
@Override
@@ -571,6 +573,8 @@ public class CassandraConfiguration {
.add("attachmentV2MigrationReadTimeout", attachmentV2MigrationReadTimeout)
.add("messageAttachmentIdsReadTimeout", messageAttachmentIdsReadTimeout)
.add("messageReadStrongConsistency", messageReadStrongConsistency)
+ .add("messageWriteStrongConsistency", messageWriteStrongConsistency)
+ .add("mailboxReadStrongConsistency", mailboxReadStrongConsistency)
.add("consistencyLevelRegular", consistencyLevelRegular)
.add("consistencyLevelLightweightTransaction", consistencyLevelLightweightTransaction)
.toString();
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxMapperGenericTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxMapperGenericTest.java
index 04ca6aa..b5177db 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxMapperGenericTest.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxMapperGenericTest.java
@@ -21,12 +21,14 @@ package org.apache.james.mailbox.cassandra.mail;
import org.apache.james.backends.cassandra.CassandraClusterExtension;
import org.apache.james.backends.cassandra.components.CassandraModule;
+import org.apache.james.backends.cassandra.init.configuration.CassandraConfiguration;
import org.apache.james.backends.cassandra.versions.CassandraSchemaVersionDAO;
import org.apache.james.backends.cassandra.versions.CassandraSchemaVersionModule;
import org.apache.james.backends.cassandra.versions.SchemaVersion;
import org.apache.james.blob.cassandra.CassandraBlobModule;
import org.apache.james.eventsourcing.eventstore.cassandra.CassandraEventStoreModule;
import org.apache.james.mailbox.cassandra.ids.CassandraId;
+import org.apache.james.mailbox.cassandra.ids.CassandraMessageId;
import org.apache.james.mailbox.cassandra.mail.utils.GuiceUtils;
import org.apache.james.mailbox.cassandra.modules.CassandraAclModule;
import org.apache.james.mailbox.cassandra.modules.CassandraMailboxModule;
@@ -98,4 +100,25 @@ class CassandraMailboxMapperGenericTest {
return CassandraId.timeBased();
}
}
+
+ @Nested
+ class V10RelaxedConsistency extends MailboxMapperTest {
+ @Override
+ protected MailboxMapper createMailboxMapper() {
+ new CassandraSchemaVersionDAO(cassandraCluster.getCassandraCluster().getConf())
+ .updateVersion(new SchemaVersion(10))
+ .block();
+ return GuiceUtils.testInjector(cassandraCluster.getCassandraCluster().getConf(),
+ cassandraCluster.getCassandraCluster().getTypesProvider(), new CassandraMessageId.Factory(),
+ CassandraConfiguration.builder()
+ .mailboxReadStrongConsistency(false)
+ .build())
+ .getInstance(CassandraMailboxMapper.class);
+ }
+
+ @Override
+ protected MailboxId generateId() {
+ return CassandraId.timeBased();
+ }
+ }
}
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 09/12: JAMES-3435 Allow to avoid mailbox SERIAL
reads when not linked to writes
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 23131f2dd6fdf4d45f4940bbcc5524424437d64e
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Sun Mar 28 22:57:33 2021 +0700
JAMES-3435 Allow to avoid mailbox SERIAL reads when not linked to writes
Let's consider now read-repairs as writes thus requires strong consistency.
---
.../cassandra/mail/CassandraMailboxMapper.java | 19 ++++++++----
.../cassandra/mail/CassandraMailboxPathV3DAO.java | 34 ++++++++++++++++++++--
.../task/SolveMailboxInconsistenciesService.java | 3 +-
.../mail/CassandraMailboxPathV3DAOTest.java | 3 +-
4 files changed, 49 insertions(+), 10 deletions(-)
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxMapper.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxMapper.java
index f4dbd1d..4ee3a75 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxMapper.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxMapper.java
@@ -19,6 +19,9 @@
package org.apache.james.mailbox.cassandra.mail;
+import static org.apache.james.mailbox.cassandra.mail.CassandraMailboxPathV3DAO.ConsistencyChoice.STRONG;
+import static org.apache.james.mailbox.cassandra.mail.CassandraMailboxPathV3DAO.ConsistencyChoice.WEAK;
+
import java.security.SecureRandom;
import java.time.Duration;
@@ -101,7 +104,7 @@ public class CassandraMailboxMapper implements MailboxMapper {
return mailboxDAO.retrieveMailbox(id)
.flatMap(mailboxEntry -> SolveMailboxInconsistenciesService.Inconsistency
.detectMailboxDaoInconsistency(mailboxEntry,
- mailboxPathV3DAO.retrieve(mailboxEntry.generateAssociatedPath()))
+ mailboxPathV3DAO.retrieve(mailboxEntry.generateAssociatedPath(), STRONG))
.flatMap(inconsistency ->
inconsistency.fix(new SolveMailboxInconsistenciesService.Context(), mailboxDAO, mailboxPathV3DAO)
.then(Mono.just(mailboxEntry))));
@@ -111,11 +114,17 @@ public class CassandraMailboxMapper implements MailboxMapper {
private Mono<Mailbox> performReadRepair(MailboxPath path) {
if (shouldReadRepair()) {
- return mailboxPathV3DAO.retrieve(path)
+ return mailboxPathV3DAO.retrieve(path, STRONG)
.flatMap(this::performPathReadRepair);
}
- return mailboxPathV3DAO.retrieve(path);
+ return mailboxPathV3DAO.retrieve(path, consistencyChoice());
+ }
+ private CassandraMailboxPathV3DAO.ConsistencyChoice consistencyChoice() {
+ if (cassandraConfiguration.isMailboxReadStrongConsistency()) {
+ return STRONG;
+ }
+ return WEAK;
}
private Flux<Mailbox> performReadRepair(Flux<Mailbox> pathEntries) {
@@ -250,13 +259,13 @@ public class CassandraMailboxMapper implements MailboxMapper {
.flatMapMany(needSupport -> {
if (needSupport) {
return Flux.concat(
- mailboxPathV3DAO.listUserMailboxes(fixedNamespace, fixedUser),
+ mailboxPathV3DAO.listUserMailboxes(fixedNamespace, fixedUser, consistencyChoice()),
Flux.concat(
mailboxPathV2DAO.listUserMailboxes(fixedNamespace, fixedUser),
mailboxPathDAO.listUserMailboxes(fixedNamespace, fixedUser))
.flatMap(this::retrieveMailbox, CONCURRENCY));
}
- return mailboxPathV3DAO.listUserMailboxes(fixedNamespace, fixedUser);
+ return mailboxPathV3DAO.listUserMailboxes(fixedNamespace, fixedUser, consistencyChoice());
});
}
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxPathV3DAO.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxPathV3DAO.java
index 3a6d46b..af34896 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxPathV3DAO.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxPathV3DAO.java
@@ -32,6 +32,8 @@ import static org.apache.james.mailbox.cassandra.table.CassandraMailboxPathV3Tab
import static org.apache.james.mailbox.cassandra.table.CassandraMailboxPathV3Table.UIDVALIDITY;
import static org.apache.james.mailbox.cassandra.table.CassandraMailboxPathV3Table.USER;
+import java.util.function.Function;
+
import javax.inject.Inject;
import org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration;
@@ -46,6 +48,7 @@ import org.apache.james.mailbox.model.UidValidity;
import org.apache.james.util.FunctionalUtils;
import org.apache.james.util.ReactorUtils;
+import com.datastax.driver.core.ConsistencyLevel;
import com.datastax.driver.core.PreparedStatement;
import com.datastax.driver.core.Row;
import com.datastax.driver.core.Session;
@@ -55,6 +58,22 @@ import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
public class CassandraMailboxPathV3DAO {
+ // todo factorize me in CassandraConsistenciesConfiguration
+ public enum ConsistencyChoice {
+ WEAK(CassandraConsistenciesConfiguration::getRegular),
+ STRONG(CassandraConsistenciesConfiguration::getLightweightTransaction);
+
+ private final Function<CassandraConsistenciesConfiguration, ConsistencyLevel> choice;
+
+ ConsistencyChoice(Function<CassandraConsistenciesConfiguration, ConsistencyLevel> choice) {
+ this.choice = choice;
+ }
+
+ public ConsistencyLevel choose(CassandraConsistenciesConfiguration configuration) {
+ return choice.apply(configuration);
+ }
+ }
+
private final CassandraAsyncExecutor cassandraAsyncExecutor;
private final CassandraUtils cassandraUtils;
private final PreparedStatement delete;
@@ -117,22 +136,31 @@ public class CassandraMailboxPathV3DAO {
}
public Mono<Mailbox> retrieve(MailboxPath mailboxPath) {
+ return retrieve(mailboxPath, consistenciesConfiguration.getLightweightTransaction());
+ }
+
+ public Mono<Mailbox> retrieve(MailboxPath mailboxPath, ConsistencyChoice consistencyChoice) {
+ return retrieve(mailboxPath, consistencyChoice.choose(consistenciesConfiguration));
+ }
+
+ private Mono<Mailbox> retrieve(MailboxPath mailboxPath, ConsistencyLevel consistencyLevel) {
return cassandraAsyncExecutor.executeSingleRow(
select.bind()
.setString(NAMESPACE, mailboxPath.getNamespace())
.setString(USER, sanitizeUser(mailboxPath.getUser()))
.setString(MAILBOX_NAME, mailboxPath.getName())
- .setConsistencyLevel(consistenciesConfiguration.getLightweightTransaction()))
+ .setConsistencyLevel(consistencyLevel))
.map(this::fromRowToCassandraIdAndPath)
.map(FunctionalUtils.toFunction(this::logGhostMailboxSuccess))
.switchIfEmpty(ReactorUtils.executeAndEmpty(() -> logGhostMailboxFailure(mailboxPath)));
}
- public Flux<Mailbox> listUserMailboxes(String namespace, Username user) {
+ public Flux<Mailbox> listUserMailboxes(String namespace, Username user, ConsistencyChoice consistencyChoice) {
return cassandraAsyncExecutor.execute(
selectUser.bind()
.setString(NAMESPACE, namespace)
- .setString(USER, sanitizeUser(user)))
+ .setString(USER, sanitizeUser(user))
+ .setConsistencyLevel(consistencyChoice.choose(consistenciesConfiguration)))
.flatMapMany(cassandraUtils::convertToFlux)
.map(this::fromRowToCassandraIdAndPath)
.map(FunctionalUtils.toFunction(this::logReadSuccess));
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/SolveMailboxInconsistenciesService.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/SolveMailboxInconsistenciesService.java
index cef2c29..5f5f695 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/SolveMailboxInconsistenciesService.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/SolveMailboxInconsistenciesService.java
@@ -19,6 +19,7 @@
package org.apache.james.mailbox.cassandra.mail.task;
+import static org.apache.james.mailbox.cassandra.mail.CassandraMailboxPathV3DAO.ConsistencyChoice.STRONG;
import static org.apache.james.util.ReactorUtils.DEFAULT_CONCURRENCY;
import java.util.Collection;
@@ -423,7 +424,7 @@ public class SolveMailboxInconsistenciesService {
}
private Mono<Inconsistency> detectMailboxDaoInconsistency(Mailbox mailboxEntry) {
- Mono<Mailbox> pathEntry = mailboxPathV3DAO.retrieve(mailboxEntry.generateAssociatedPath());
+ Mono<Mailbox> pathEntry = mailboxPathV3DAO.retrieve(mailboxEntry.generateAssociatedPath(), STRONG);
return Inconsistency.detectMailboxDaoInconsistency(mailboxEntry, pathEntry);
}
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxPathV3DAOTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxPathV3DAOTest.java
index 92d1aa9..a59c759 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxPathV3DAOTest.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxPathV3DAOTest.java
@@ -19,6 +19,7 @@
package org.apache.james.mailbox.cassandra.mail;
+import static org.apache.james.backends.cassandra.init.configuration.CassandraConsistenciesConfiguration.ConsistencyChoice.STRONG;
import static org.apache.james.mailbox.cassandra.mail.MailboxFixture.MAILBOX_1;
import static org.apache.james.mailbox.cassandra.mail.MailboxFixture.MAILBOX_2;
import static org.apache.james.mailbox.cassandra.mail.MailboxFixture.MAILBOX_3;
@@ -95,7 +96,7 @@ class CassandraMailboxPathV3DAOTest {
testee.save(MAILBOX_3).block();
List<Mailbox> cassandraIds = testee
- .listUserMailboxes(USER_INBOX_MAILBOXPATH.getNamespace(), USER_INBOX_MAILBOXPATH.getUser())
+ .listUserMailboxes(USER_INBOX_MAILBOXPATH.getNamespace(), USER_INBOX_MAILBOXPATH.getUser(), STRONG)
.collectList()
.block();
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 02/12: JAMES-3435 Configuration option:
message.write.strong.consistency
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit d39460fcc063a4757e18663c9e2efe8673d96866
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Sun Mar 28 14:27:47 2021 +0700
JAMES-3435 Configuration option: message.write.strong.consistency
---
.../init/configuration/CassandraConfiguration.java | 26 +++++++++++++--
.../mail/CassandraMessageIdToImapUidDAO.java | 38 ++++++++++++++--------
2 files changed, 48 insertions(+), 16 deletions(-)
diff --git a/backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/CassandraConfiguration.java b/backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/CassandraConfiguration.java
index ad46dd9..57e60a4 100644
--- a/backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/CassandraConfiguration.java
+++ b/backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/CassandraConfiguration.java
@@ -74,6 +74,7 @@ public class CassandraConfiguration {
private static final String ATTACHMENT_V2_MIGRATION_READ_TIMEOUT = "attachment.v2.migration.read.timeout";
private static final String MESSAGE_ATTACHMENTID_READ_TIMEOUT = "message.attachmentids.read.timeout";
private static final String MESSAGE_READ_STRONG_CONSISTENCY = "message.read.strong.consistency";
+ private static final String MESSAGE_WRITE_STRONG_CONSISTENCY = "message.write.strong.consistency.unsafe";
private static final String CONSISTENCY_LEVEL_REGULAR = "cassandra.consistency_level.regular";
private static final String CONSISTENCY_LEVEL_LIGHTWEIGHT_TRANSACTION = "cassandra.consistency_level.lightweight_transaction";
@@ -97,6 +98,7 @@ public class CassandraConfiguration {
private Optional<Float> mailboxCountersReadRepairMax = Optional.empty();
private Optional<Float> mailboxCountersReadRepairChanceOneHundred = Optional.empty();
private Optional<Boolean> messageReadStrongConsistency = Optional.empty();
+ private Optional<Boolean> messageWriteStrongConsistency = Optional.empty();
public Builder messageReadStrongConsistency(boolean value) {
this.messageReadStrongConsistency = Optional.of(value);
@@ -108,6 +110,16 @@ public class CassandraConfiguration {
return this;
}
+ public Builder messageWriteStrongConsistency(boolean value) {
+ this.messageWriteStrongConsistency = Optional.of(value);
+ return this;
+ }
+
+ public Builder messageWriteStrongConsistency(Optional<Boolean> value) {
+ this.messageWriteStrongConsistency = value;
+ return this;
+ }
+
public Builder messageReadChunkSize(int value) {
Preconditions.checkArgument(value > 0, "messageReadChunkSize needs to be strictly positive");
this.messageReadChunkSize = Optional.of(value);
@@ -314,7 +326,8 @@ public class CassandraConfiguration {
mailboxReadRepair.orElse(DEFAULT_MAILBOX_READ_REPAIR),
mailboxCountersReadRepairMax.orElse(DEFAULT_MAX_MAILBOX_COUNTERS_READ_REPAIR_CHANCE),
mailboxCountersReadRepairChanceOneHundred.orElse(DEFAULT_ONE_HUNDRED_MAILBOX_COUNTERS_READ_REPAIR_CHANCE),
- messageReadStrongConsistency.orElse(DEFAULT_STRONG_CONSISTENCY));
+ messageReadStrongConsistency.orElse(DEFAULT_STRONG_CONSISTENCY),
+ messageWriteStrongConsistency.orElse(DEFAULT_STRONG_CONSISTENCY));
}
}
@@ -358,6 +371,8 @@ public class CassandraConfiguration {
propertiesConfiguration.getFloat(MAILBOX_ONE_HUNDRED_COUNTERS_READ_REPAIR, null)))
.messageReadStrongConsistency(Optional.ofNullable(
propertiesConfiguration.getBoolean(MESSAGE_READ_STRONG_CONSISTENCY, null)))
+ .messageWriteStrongConsistency(Optional.ofNullable(
+ propertiesConfiguration.getBoolean(MESSAGE_WRITE_STRONG_CONSISTENCY, null)))
.build();
}
@@ -378,6 +393,7 @@ public class CassandraConfiguration {
private final float mailboxCountersReadRepairChanceMax;
private final float mailboxCountersReadRepairChanceOneHundred;
private final boolean messageReadStrongConsistency;
+ private final boolean messageWriteStrongConsistency;
@VisibleForTesting
CassandraConfiguration(int aclMaxRetry, int messageReadChunkSize, int expungeChunkSize,
@@ -386,7 +402,8 @@ public class CassandraConfiguration {
int blobPartSize, final int attachmentV2MigrationReadTimeout, int messageAttachmentIdsReadTimeout,
String consistencyLevelRegular, String consistencyLevelLightweightTransaction,
float mailboxReadRepair, float mailboxCountersReadRepairChanceMax,
- float mailboxCountersReadRepairChanceOneHundred, boolean messageReadStrongConsistency) {
+ float mailboxCountersReadRepairChanceOneHundred, boolean messageReadStrongConsistency,
+ boolean messageWriteStrongConsistency) {
this.aclMaxRetry = aclMaxRetry;
this.messageReadChunkSize = messageReadChunkSize;
this.expungeChunkSize = expungeChunkSize;
@@ -404,6 +421,11 @@ public class CassandraConfiguration {
this.mailboxCountersReadRepairChanceMax = mailboxCountersReadRepairChanceMax;
this.mailboxCountersReadRepairChanceOneHundred = mailboxCountersReadRepairChanceOneHundred;
this.messageReadStrongConsistency = messageReadStrongConsistency;
+ this.messageWriteStrongConsistency = messageWriteStrongConsistency;
+ }
+
+ public boolean isMessageWriteStrongConsistency() {
+ return messageWriteStrongConsistency;
}
public boolean isMessageReadStrongConsistency() {
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdToImapUidDAO.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdToImapUidDAO.java
index 0bb92ea..e4bce1f 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdToImapUidDAO.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageIdToImapUidDAO.java
@@ -57,6 +57,7 @@ import org.apache.james.mailbox.cassandra.ids.CassandraMessageId.Factory;
import org.apache.james.mailbox.model.ComposedMessageId;
import org.apache.james.mailbox.model.ComposedMessageIdWithMetaData;
+import com.datastax.driver.core.BoundStatement;
import com.datastax.driver.core.ConsistencyLevel;
import com.datastax.driver.core.PreparedStatement;
import com.datastax.driver.core.Row;
@@ -103,6 +104,7 @@ public class CassandraMessageIdToImapUidDAO {
CassandraMessageId.Factory messageIdFactory) {
this.cassandraAsyncExecutor = new CassandraAsyncExecutor(session);
this.consistenciesConfiguration = consistenciesConfiguration;
+ this.cassandraConfiguration = cassandraConfiguration;
this.messageIdFactory = messageIdFactory;
this.delete = prepareDelete(session);
this.insert = prepareInsert(session);
@@ -197,20 +199,28 @@ public class CassandraMessageIdToImapUidDAO {
public Mono<Boolean> updateMetadata(ComposedMessageIdWithMetaData composedMessageIdWithMetaData, ModSeq oldModSeq) {
ComposedMessageId composedMessageId = composedMessageIdWithMetaData.getComposedMessageId();
Flags flags = composedMessageIdWithMetaData.getFlags();
- return cassandraAsyncExecutor.executeReturnApplied(update.bind()
- .setLong(MOD_SEQ, composedMessageIdWithMetaData.getModSeq().asLong())
- .setBool(ANSWERED, flags.contains(Flag.ANSWERED))
- .setBool(DELETED, flags.contains(Flag.DELETED))
- .setBool(DRAFT, flags.contains(Flag.DRAFT))
- .setBool(FLAGGED, flags.contains(Flag.FLAGGED))
- .setBool(RECENT, flags.contains(Flag.RECENT))
- .setBool(SEEN, flags.contains(Flag.SEEN))
- .setBool(USER, flags.contains(Flag.USER))
- .setSet(USER_FLAGS, ImmutableSet.copyOf(flags.getUserFlags()))
- .setUUID(MESSAGE_ID, ((CassandraMessageId) composedMessageId.getMessageId()).get())
- .setUUID(MAILBOX_ID, ((CassandraId) composedMessageId.getMailboxId()).asUuid())
- .setLong(IMAP_UID, composedMessageId.getUid().asLong())
- .setLong(MOD_SEQ_CONDITION, oldModSeq.asLong()));
+ return cassandraAsyncExecutor.executeReturnApplied(updateBoundStatement(composedMessageIdWithMetaData, composedMessageId, flags, oldModSeq));
+ }
+
+ private BoundStatement updateBoundStatement(ComposedMessageIdWithMetaData composedMessageIdWithMetaData, ComposedMessageId composedMessageId, Flags flags,
+ ModSeq oldModSeq) {
+ final BoundStatement boundStatement = update.bind()
+ .setLong(MOD_SEQ, composedMessageIdWithMetaData.getModSeq().asLong())
+ .setBool(ANSWERED, flags.contains(Flag.ANSWERED))
+ .setBool(DELETED, flags.contains(Flag.DELETED))
+ .setBool(DRAFT, flags.contains(Flag.DRAFT))
+ .setBool(FLAGGED, flags.contains(Flag.FLAGGED))
+ .setBool(RECENT, flags.contains(Flag.RECENT))
+ .setBool(SEEN, flags.contains(Flag.SEEN))
+ .setBool(USER, flags.contains(Flag.USER))
+ .setSet(USER_FLAGS, ImmutableSet.copyOf(flags.getUserFlags()))
+ .setUUID(MESSAGE_ID, ((CassandraMessageId) composedMessageId.getMessageId()).get())
+ .setUUID(MAILBOX_ID, ((CassandraId) composedMessageId.getMailboxId()).asUuid())
+ .setLong(IMAP_UID, composedMessageId.getUid().asLong());
+ if (cassandraConfiguration.isMessageWriteStrongConsistency()) {
+ return boundStatement.setLong(MOD_SEQ_CONDITION, oldModSeq.asLong());
+ }
+ return boundStatement;
}
public Flux<ComposedMessageIdWithMetaData> retrieve(CassandraMessageId messageId, Optional<CassandraId> mailboxId, ConsistencyChoice readConsistencyChoice) {
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org