You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@james.apache.org by bt...@apache.org on 2020/10/16 03:54:32 UTC
[james-project] branch master updated (3653e5d -> f0d359a)
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git.
from 3653e5d JAMES-3410 Email/set destroy error handling
new 3638011 JAMES-3428 Implement read-repairs for mailbox counters
new 3146cde JAMES-3428 Configure read-repairs for mailbox counters
new 01d79e3 JAMES-3428 Document read-repairs for mailbox counters
new db2dc80 JAMES-3407 Switch checks as overwise possible on the first check can make the second one fail
new 1a3e494 JAMES-3415 Modified ADR and README.md for James Webadmin CLI
new f0d359a JAMES-3028 define implementation in blob properties for cassandra docker images
The 6 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
CHANGELOG.md | 1 +
.../init/configuration/CassandraConfiguration.java | 59 +++++++++++++++++++-
.../destination/conf/blob.properties | 41 +++++++++++---
.../destination/conf/cassandra.properties | 6 +++
.../destination/conf/cassandra.properties | 6 +++
.../destination/conf/cassandra.properties | 6 +++
.../cassandra/destination/conf/blob.properties | 41 +++++++++++---
.../destination/conf/cassandra.properties | 6 +++
.../pages/distributed/configure/cassandra.adoc | 14 +++++
.../james/mailbox/model/MailboxCounters.java | 2 +-
.../CassandraMailboxSessionMapperFactory.java | 6 ++-
.../cassandra/mail/CassandraMessageMapper.java | 43 ++++++++++++++-
.../mail/task/RecomputeMailboxCountersService.java | 6 +--
.../CassandraSubscriptionManagerTest.java | 3 ++
.../cassandra/mail/CassandraMailboxMapperTest.java | 4 +-
.../cassandra/mail/CassandraMessageMapperTest.java | 62 ++++++++++++++++++++++
server/protocols/webadmin-cli/README.md | 23 ++++----
src/adr/0042-james-cli-based-on-webadmin.md | 6 +--
src/site/xdoc/server/config-cassandra.xml | 10 ++++
19 files changed, 308 insertions(+), 37 deletions(-)
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 04/06: JAMES-3407 Switch checks as overwise
possible on the first check can make the second one fail
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit db2dc80f3bd5f30be9ed62e1b67d5ac138e0e071
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Thu Oct 15 08:46:09 2020 +0700
JAMES-3407 Switch checks as overwise possible on the first check can make the second one fail
---
.../james/mailbox/cassandra/mail/CassandraMailboxMapperTest.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxMapperTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxMapperTest.java
index 2b9d1d7..a2a8f30 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxMapperTest.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxMapperTest.java
@@ -291,11 +291,11 @@ class CassandraMailboxMapperTest {
.block());
SoftAssertions.assertSoftly(Throwing.consumer(softly -> {
+ softly.assertThat(MailboxReactorUtils.blockOptional(testee.findMailboxByPath(MAILBOX_PATH)))
+ .isEmpty();
softly(softly)
.assertThat(testee.findMailboxById(MAILBOX_ID).block())
.isEqualTo(MAILBOX);
- softly.assertThat(MailboxReactorUtils.blockOptional(testee.findMailboxByPath(MAILBOX_PATH)))
- .isEmpty();
}));
}
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 01/06: JAMES-3428 Implement read-repairs for
mailbox counters
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 363801128637e23bd85ce2c6e105145ee405c0f5
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Wed Oct 14 13:46:35 2020 +0700
JAMES-3428 Implement read-repairs for mailbox counters
---
.../james/mailbox/model/MailboxCounters.java | 2 +-
.../CassandraMailboxSessionMapperFactory.java | 6 ++-
.../cassandra/mail/CassandraMessageMapper.java | 43 ++++++++++++++-
.../mail/task/RecomputeMailboxCountersService.java | 6 +--
.../CassandraSubscriptionManagerTest.java | 3 ++
.../cassandra/mail/CassandraMessageMapperTest.java | 62 ++++++++++++++++++++++
6 files changed, 116 insertions(+), 6 deletions(-)
diff --git a/mailbox/api/src/main/java/org/apache/james/mailbox/model/MailboxCounters.java b/mailbox/api/src/main/java/org/apache/james/mailbox/model/MailboxCounters.java
index 3cceed5..854264d 100644
--- a/mailbox/api/src/main/java/org/apache/james/mailbox/model/MailboxCounters.java
+++ b/mailbox/api/src/main/java/org/apache/james/mailbox/model/MailboxCounters.java
@@ -113,7 +113,7 @@ public class MailboxCounters {
return Sanitized.of(mailboxId, sanitizedCount, sanitizedUnseen);
}
- private boolean isValid() {
+ public boolean isValid() {
return count >= 0
&& unseen >= 0
&& count >= unseen;
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/CassandraMailboxSessionMapperFactory.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/CassandraMailboxSessionMapperFactory.java
index 3cd91fd..f0e5609 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/CassandraMailboxSessionMapperFactory.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/CassandraMailboxSessionMapperFactory.java
@@ -51,6 +51,7 @@ import org.apache.james.mailbox.cassandra.mail.CassandraMessageMapper;
import org.apache.james.mailbox.cassandra.mail.CassandraModSeqProvider;
import org.apache.james.mailbox.cassandra.mail.CassandraUidProvider;
import org.apache.james.mailbox.cassandra.mail.CassandraUserMailboxRightsDAO;
+import org.apache.james.mailbox.cassandra.mail.task.RecomputeMailboxCountersService;
import org.apache.james.mailbox.cassandra.user.CassandraSubscriptionMapper;
import org.apache.james.mailbox.store.MailboxSessionMapperFactory;
import org.apache.james.mailbox.store.mail.AnnotationMapper;
@@ -94,6 +95,7 @@ public class CassandraMailboxSessionMapperFactory extends MailboxSessionMapperFa
private final CassandraUserMailboxRightsDAO userMailboxRightsDAO;
private final CassandraSchemaVersionManager versionManager;
private final CassandraUtils cassandraUtils;
+ private final RecomputeMailboxCountersService recomputeMailboxCountersService;
private final CassandraConfiguration cassandraConfiguration;
@Inject
@@ -107,6 +109,7 @@ public class CassandraMailboxSessionMapperFactory extends MailboxSessionMapperFa
CassandraAttachmentOwnerDAO ownerDAO, CassandraACLMapper aclMapper,
CassandraUserMailboxRightsDAO userMailboxRightsDAO,
CassandraSchemaVersionManager versionManager,
+ RecomputeMailboxCountersService recomputeMailboxCountersService,
CassandraUtils cassandraUtils, CassandraConfiguration cassandraConfiguration) {
this.uidProvider = uidProvider;
this.modSeqProvider = modSeqProvider;
@@ -131,6 +134,7 @@ public class CassandraMailboxSessionMapperFactory extends MailboxSessionMapperFa
this.versionManager = versionManager;
this.cassandraUtils = cassandraUtils;
this.ownerDAO = ownerDAO;
+ this.recomputeMailboxCountersService = recomputeMailboxCountersService;
this.cassandraConfiguration = cassandraConfiguration;
this.indexTableHandler = new CassandraIndexTableHandler(
mailboxRecentsDAO,
@@ -155,7 +159,7 @@ public class CassandraMailboxSessionMapperFactory extends MailboxSessionMapperFa
indexTableHandler,
firstUnseenDAO,
deletedMessageDAO,
- cassandraConfiguration);
+ cassandraConfiguration, recomputeMailboxCountersService);
}
@Override
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapper.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapper.java
index 851db3d..b72d56d 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapper.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapper.java
@@ -19,6 +19,7 @@
package org.apache.james.mailbox.cassandra.mail;
+import java.security.SecureRandom;
import java.time.Duration;
import java.util.Comparator;
import java.util.Iterator;
@@ -39,6 +40,7 @@ import org.apache.james.mailbox.MessageUid;
import org.apache.james.mailbox.ModSeq;
import org.apache.james.mailbox.cassandra.ids.CassandraId;
import org.apache.james.mailbox.cassandra.ids.CassandraMessageId;
+import org.apache.james.mailbox.cassandra.mail.task.RecomputeMailboxCountersService;
import org.apache.james.mailbox.cassandra.mail.utils.FlagsUpdateStageResult;
import org.apache.james.mailbox.exception.MailboxException;
import org.apache.james.mailbox.model.ComposedMessageId;
@@ -52,6 +54,7 @@ import org.apache.james.mailbox.store.FlagsUpdateCalculator;
import org.apache.james.mailbox.store.mail.MessageMapper;
import org.apache.james.mailbox.store.mail.model.MailboxMessage;
import org.apache.james.mailbox.store.mail.model.impl.SimpleMailboxMessage;
+import org.apache.james.task.Task;
import org.apache.james.util.ReactorUtils;
import org.apache.james.util.streams.Limit;
import org.slf4j.Logger;
@@ -85,6 +88,8 @@ public class CassandraMessageMapper implements MessageMapper {
private final AttachmentLoader attachmentLoader;
private final CassandraDeletedMessageDAO deletedMessageDAO;
private final CassandraConfiguration cassandraConfiguration;
+ private final RecomputeMailboxCountersService recomputeMailboxCountersService;
+ private final SecureRandom secureRandom;
public CassandraMessageMapper(CassandraUidProvider uidProvider, CassandraModSeqProvider modSeqProvider,
CassandraAttachmentMapper attachmentMapper,
@@ -92,7 +97,8 @@ public class CassandraMessageMapper implements MessageMapper {
CassandraMessageIdToImapUidDAO imapUidDAO, CassandraMailboxCounterDAO mailboxCounterDAO,
CassandraMailboxRecentsDAO mailboxRecentDAO, CassandraApplicableFlagDAO applicableFlagDAO,
CassandraIndexTableHandler indexTableHandler, CassandraFirstUnseenDAO firstUnseenDAO,
- CassandraDeletedMessageDAO deletedMessageDAO, CassandraConfiguration cassandraConfiguration) {
+ CassandraDeletedMessageDAO deletedMessageDAO, CassandraConfiguration cassandraConfiguration,
+ RecomputeMailboxCountersService recomputeMailboxCountersService) {
this.uidProvider = uidProvider;
this.modSeqProvider = modSeqProvider;
this.messageDAO = messageDAO;
@@ -106,6 +112,8 @@ public class CassandraMessageMapper implements MessageMapper {
this.applicableFlagDAO = applicableFlagDAO;
this.deletedMessageDAO = deletedMessageDAO;
this.cassandraConfiguration = cassandraConfiguration;
+ this.recomputeMailboxCountersService = recomputeMailboxCountersService;
+ this.secureRandom = new SecureRandom();
}
@Override
@@ -128,6 +136,18 @@ public class CassandraMessageMapper implements MessageMapper {
@Override
public Mono<MailboxCounters> getMailboxCountersReactive(Mailbox mailbox) {
CassandraId mailboxId = (CassandraId) mailbox.getMailboxId();
+ return readMailboxCounters(mailboxId)
+ .flatMap(counters -> {
+ if (!counters.isValid()) {
+ return fixCounters(mailbox)
+ .then(readMailboxCounters(mailboxId));
+ }
+ return Mono.just(counters);
+ })
+ .doOnNext(counters -> readRepair(mailbox, counters));
+ }
+
+ public Mono<MailboxCounters> readMailboxCounters(CassandraId mailboxId) {
return mailboxCounterDAO.retrieveMailboxCounters(mailboxId)
.defaultIfEmpty(MailboxCounters.builder()
.mailboxId(mailboxId)
@@ -136,6 +156,27 @@ public class CassandraMessageMapper implements MessageMapper {
.build());
}
+ private void readRepair(Mailbox mailbox, MailboxCounters counters) {
+ if (shouldReadRepair(counters)) {
+ fixCounters(mailbox)
+ .subscribeOn(Schedulers.elastic())
+ .subscribe();
+ }
+ }
+
+ private Mono<Task.Result> fixCounters(Mailbox mailbox) {
+ return recomputeMailboxCountersService.recomputeMailboxCounter(
+ new RecomputeMailboxCountersService.Context(),
+ mailbox,
+ RecomputeMailboxCountersService.Options.trustMessageProjection());
+ }
+
+ private boolean shouldReadRepair(MailboxCounters counters) {
+ double readRepairChance = 0.1;
+ double ponderatedReadRepairChance = readRepairChance * (10.0 / counters.getUnseen());
+ return secureRandom.nextFloat() < Math.min(readRepairChance, ponderatedReadRepairChance);
+ }
+
@Override
public void delete(Mailbox mailbox, MailboxMessage message) {
ComposedMessageIdWithMetaData metaData = message.getComposedMessageIdWithMetaData();
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java
index be1fbec..18716c5 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java
@@ -122,7 +122,7 @@ public class RecomputeMailboxCountersService {
}
}
- static class Context {
+ public static class Context {
static class Snapshot {
private final long processedMailboxCount;
private final ImmutableList<CassandraId> failedMailboxes;
@@ -168,7 +168,7 @@ public class RecomputeMailboxCountersService {
private final AtomicLong processedMailboxCount;
private final ConcurrentLinkedDeque<CassandraId> failedMailboxes;
- Context() {
+ public Context() {
processedMailboxCount = new AtomicLong();
failedMailboxes = new ConcurrentLinkedDeque<>();
}
@@ -213,7 +213,7 @@ public class RecomputeMailboxCountersService {
});
}
- private Mono<Result> recomputeMailboxCounter(Context context, Mailbox mailbox, Options options) {
+ public Mono<Result> recomputeMailboxCounter(Context context, Mailbox mailbox, Options options) {
CassandraId mailboxId = (CassandraId) mailbox.getMailboxId();
Counter counter = new Counter(mailboxId);
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/CassandraSubscriptionManagerTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/CassandraSubscriptionManagerTest.java
index 781ef96..9919308 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/CassandraSubscriptionManagerTest.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/CassandraSubscriptionManagerTest.java
@@ -45,6 +45,7 @@ import org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO;
import org.apache.james.mailbox.cassandra.mail.CassandraModSeqProvider;
import org.apache.james.mailbox.cassandra.mail.CassandraUidProvider;
import org.apache.james.mailbox.cassandra.mail.CassandraUserMailboxRightsDAO;
+import org.apache.james.mailbox.cassandra.mail.task.RecomputeMailboxCountersService;
import org.apache.james.mailbox.cassandra.modules.CassandraSubscriptionModule;
import org.apache.james.mailbox.store.StoreSubscriptionManager;
import org.junit.jupiter.api.BeforeEach;
@@ -88,6 +89,7 @@ class CassandraSubscriptionManagerTest implements SubscriptionManagerContract {
CassandraUidProvider uidProvider = null;
CassandraModSeqProvider modSeqProvider = null;
CassandraSchemaVersionManager versionManager = null;
+ RecomputeMailboxCountersService recomputeMailboxCountersService = null;
subscriptionManager = new StoreSubscriptionManager(
new CassandraMailboxSessionMapperFactory(
@@ -113,6 +115,7 @@ class CassandraSubscriptionManagerTest implements SubscriptionManagerContract {
aclMapper,
userMailboxRightsDAO,
versionManager,
+ recomputeMailboxCountersService,
CassandraUtils.WITH_DEFAULT_CONFIGURATION,
CassandraConfiguration.DEFAULT_CONFIGURATION));
}
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapperTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapperTest.java
index aab3f51..20de2e5 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapperTest.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapperTest.java
@@ -19,6 +19,7 @@
package org.apache.james.mailbox.cassandra.mail;
+import static java.util.concurrent.TimeUnit.MILLISECONDS;
import static org.apache.james.backends.cassandra.Scenario.Builder.fail;
import static org.assertj.core.api.Assertions.assertThat;
@@ -43,6 +44,8 @@ import org.apache.james.mailbox.store.mail.model.MapperProvider;
import org.apache.james.mailbox.store.mail.model.MessageMapperTest;
import org.apache.james.util.streams.Limit;
import org.assertj.core.api.SoftAssertions;
+import org.awaitility.Awaitility;
+import org.awaitility.Duration;
import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
@@ -353,4 +356,63 @@ class CassandraMessageMapperTest extends MessageMapperTest {
.hasSize(1);
}
}
+
+ @Nested
+ class ReadRepairsTesting {
+ @Test
+ void readingShouldEventuallyFixCountersInconsistencies(CassandraCluster cassandra) throws MailboxException {
+ saveMessages();
+ FlagsUpdateCalculator newFlags = new FlagsUpdateCalculator(new Flags(Flags.Flag.SEEN), MessageManager.FlagsUpdateMode.REPLACE);
+ messageMapper.updateFlags(benwaInboxMailbox, message1.getUid(), newFlags);
+ // Expected count of unseen is 4 see MessageMapperTest::mailboxUnSeenCountShouldBeDecrementedAfterAMessageIsMarkedSeen
+
+ // Create an inconsistency
+ new CassandraMailboxCounterDAO(cassandra.getConf())
+ .incrementUnseenAndCount((CassandraId) benwaInboxMailbox.getMailboxId())
+ .block();
+
+ // 100 poll with a 0.1 probability to trigger read repair
+ Awaitility.await()
+ .pollInterval(new Duration(10, MILLISECONDS))
+ .atMost(Duration.ONE_SECOND)
+ .untilAsserted(() ->
+ assertThat(messageMapper.getMailboxCounters(benwaInboxMailbox).getUnseen()).isEqualTo(4));
+ }
+
+ @Test
+ void readingShouldEventuallyFixMissingCountersInconsistencies(CassandraCluster cassandra) throws MailboxException {
+ saveMessages();
+ FlagsUpdateCalculator newFlags = new FlagsUpdateCalculator(new Flags(Flags.Flag.SEEN), MessageManager.FlagsUpdateMode.REPLACE);
+ messageMapper.updateFlags(benwaInboxMailbox, message1.getUid(), newFlags);
+ // Expected count of unseen is 4 see MessageMapperTest::mailboxUnSeenCountShouldBeDecrementedAfterAMessageIsMarkedSeen
+
+ // Create an inconsistency
+ new CassandraMailboxCounterDAO(cassandra.getConf())
+ .delete((CassandraId) benwaInboxMailbox.getMailboxId())
+ .block();
+
+ // 100 poll with a 0.1 probability to trigger read repair
+ Awaitility.await()
+ .pollInterval(new Duration(10, MILLISECONDS))
+ .atMost(Duration.ONE_SECOND)
+ .untilAsserted(() ->
+ assertThat(messageMapper.getMailboxCounters(benwaInboxMailbox).getUnseen()).isEqualTo(4));
+ }
+
+ @Test
+ void readingShouldFixInvalidCounters(CassandraCluster cassandra) throws MailboxException {
+ saveMessages();
+ FlagsUpdateCalculator newFlags = new FlagsUpdateCalculator(new Flags(Flags.Flag.SEEN), MessageManager.FlagsUpdateMode.REPLACE);
+ messageMapper.updateFlags(benwaInboxMailbox, message1.getUid(), newFlags);
+ // Expected count of unseen is 4 see MessageMapperTest::mailboxUnSeenCountShouldBeDecrementedAfterAMessageIsMarkedSeen
+
+ // Create an inconsistency
+ new CassandraMailboxCounterDAO(cassandra.getConf())
+ .incrementUnseen((CassandraId) benwaInboxMailbox.getMailboxId())
+ .repeat(5)
+ .blockLast();
+
+ assertThat(messageMapper.getMailboxCounters(benwaInboxMailbox).getUnseen()).isEqualTo(4);
+ }
+ }
}
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 03/06: JAMES-3428 Document read-repairs for mailbox
counters
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 01d79e36f74eece1dba7e2446aa71daf7029dad3
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Wed Oct 14 14:25:45 2020 +0700
JAMES-3428 Document read-repairs for mailbox counters
---
CHANGELOG.md | 1 +
.../servers/pages/distributed/configure/cassandra.adoc | 14 ++++++++++++++
src/site/xdoc/server/config-cassandra.xml | 10 ++++++++++
3 files changed, 25 insertions(+)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 89b2e5e..790db3d 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -15,6 +15,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
- JAMES-3202 Reindex only outdated documents with the Mode option set to CORRECT in reindexing tasks
- JAMES-3405 Expose metrics of Guice servers over HTTP - enables easy Prometheus metrics collection
- JAMES-3407 Distributed server: Read-repairs for the mailbox entity
+- JAMES-3428 Distributed server: Read-repairs for the mailbox counters entity
### Changed
- Switch to Java 11 for build and run
diff --git a/docs/modules/servers/pages/distributed/configure/cassandra.adoc b/docs/modules/servers/pages/distributed/configure/cassandra.adoc
index d4bdafe..3439948 100644
--- a/docs/modules/servers/pages/distributed/configure/cassandra.adoc
+++ b/docs/modules/servers/pages/distributed/configure/cassandra.adoc
@@ -139,6 +139,20 @@ If you want more explanation about Cassandra configuration, you should visit the
| Optional. Defaults to 0.1 (10% chance).
Must be between 0 and 1 (inclusive). Controls the probability of doing a read-repair upon mailbox read.
+| mailbox.read.repair.chance
+| Optional. Defaults to 0.1 (10% chance).
+Must be between 0 and 1 (inclusive). Controls the probability of doing a read-repair upon mailbox read.
+
+| mailbox.counters.read.repair.chance.max
+| Optional. Defaults to 0.1 (10% chance).
+Must be between 0 and 1 (inclusive). Controls the probability of doing a read-repair upon mailbox counters read.
+Formula: read_repair_chance = min(mailbox.counters.read.repair.chance.max, (100/unseens)*mailbox.counters.read.repair.chance.one.hundred)
+
+| mailbox.counters.read.repair.chance.one.hundred
+| Optional. Defaults to 0.01 (1% chance).
+Must be between 0 and 1 (inclusive). Controls the probability of doing a read-repair upon mailbox counters read.
+Formula: read_repair_chance = min(mailbox.counters.read.repair.chance.max, (100/unseens)*mailbox.counters.read.repair.chance.one.hundred)
+
| mailbox.max.retry.acl
| Optional. Defaults to 1000.
Controls the number of retries upon Cassandra ACL updates.
diff --git a/src/site/xdoc/server/config-cassandra.xml b/src/site/xdoc/server/config-cassandra.xml
index 23280c4..a8e2470 100644
--- a/src/site/xdoc/server/config-cassandra.xml
+++ b/src/site/xdoc/server/config-cassandra.xml
@@ -120,6 +120,16 @@
<dt><strong>mailbox.read.repair.chance</strong></dt>
<dd>Optional. Defaults to 0.1 (10% chance).<br/> Must be between 0 and 1 (inclusive). Controls the probability of doing a read-repair upon mailbox read.</dd>
+ <dt><strong>mailbox.counters.read.repair.chance.max</strong></dt>
+ <dd>Optional. Defaults to 0.1 (10% chance).<br/>
+ Must be between 0 and 1 (inclusive). Controls the probability of doing a read-repair upon mailbox counters read.<br/>
+ Formula: read_repair_chance = min(mailbox.counters.read.repair.chance.max, (100/unseens)*mailbox.counters.read.repair.chance.one.hundred)
+ </dd>
+ <dt><strong>mailbox.counters.read.repair.chance.one.hundred</strong></dt>
+ <dd>Optional. Defaults to 0.01 (1% chance).<br/>
+ Must be between 0 and 1 (inclusive). Controls the probability of doing a read-repair upon mailbox counters read.<br/>
+ Formula: read_repair_chance = min(mailbox.counters.read.repair.chance.max, (100/unseens)*mailbox.counters.read.repair.chance.one.hundred)
+ </dd>
<dt><strong>mailbox.max.retry.acl</strong></dt>
<dd>Optional. Defaults to 1000.<br/> Controls the number of retries upon Cassandra ACL updates.</dd>
<dt><strong>mailbox.max.retry.modseq</strong></dt>
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 06/06: JAMES-3028 define implementation in blob
properties for cassandra docker images
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit f0d359a887c53b299c7a0b1071d93ec4291d1cd3
Author: Rémi Kowalski <rk...@linagora.com>
AuthorDate: Thu Oct 15 10:48:34 2020 +0200
JAMES-3028 define implementation in blob properties for cassandra docker images
---
.../destination/conf/blob.properties | 41 ++++++++++++++++++----
.../cassandra/destination/conf/blob.properties | 41 ++++++++++++++++++----
2 files changed, 70 insertions(+), 12 deletions(-)
diff --git a/dockerfiles/run/guice/cassandra-ldap/destination/conf/blob.properties b/dockerfiles/run/guice/cassandra-ldap/destination/conf/blob.properties
index 2ec4035..c999f80 100644
--- a/dockerfiles/run/guice/cassandra-ldap/destination/conf/blob.properties
+++ b/dockerfiles/run/guice/cassandra-ldap/destination/conf/blob.properties
@@ -1,10 +1,9 @@
-# ============================================ Blobs Exporting ==============================================
-# Read https://james.apache.org/server/config-blob-export.html for further details
+# ============================================= BlobStore Implementation ==================================
+# Read https://james.apache.org/server/config-blobstore.html for further details
-# Choosing blob exporting mechanism, allowed mechanism are: localFile, linshare
-# LinShare is a file sharing service, will be explained in the below section
-# Optional, default is localFile
-blob.export.implementation=localFile
+# Choose your BlobStore implementation
+# Mandatory, allowed values are: cassandra, s3
+implementation=cassandra
# ========================================= ObjectStorage deduplication ========================================
# If you choose to enable deduplication, the mails with the same content will be stored only once.
@@ -13,6 +12,36 @@ blob.export.implementation=localFile
# Mandatory, Allowed values are: true, false
deduplication.enable=false
+# ========================================= Cassandra BlobStore Cache ======================================
+# A cassandra cache can be enabled to reduce latency when reading small blobs frequently
+# A dedicated keyspace with a replication factor of one is then used
+# Cache eviction policy is TTL based
+# Only blobs below a given threshold will be stored.
+# To be noted that blobs are stored within a single Cassandra row, hence a low threshold should be used.
+
+# Enable the cache? Optional and default to false. Must be a boolean.
+cache.enable=false
+
+# Cache eviction policy is TTL based. Optional and defaults to 7 days. Must be a duration.
+# Valid units: ms, sec, min, hour, day, week, month, year
+# cache.cassandra.ttl=7days
+
+# Timeout after which this cache should be bypassed. Optional and defaults to 100ms. Can not exceed 1 hour.
+# Must be a duration Valid units: ms, sec, min, hour, day, week, month, year
+# cache.cassandra.timeout=100ms
+
+# Maximum size of stored objects expressed in bytes. Must be strictly positive. Defaults to 8192.
+# Units: bytes, Kib, MiB, GiB, TiB
+# cache.sizeThresholdInBytes=8 KiB
+
+# ============================================ Blobs Exporting ==============================================
+# Read https://james.apache.org/server/config-blob-export.html for further details
+
+# Choosing blob exporting mechanism, allowed mechanism are: localFile, linshare
+# LinShare is a file sharing service, will be explained in the below section
+# Optional, default is localFile
+blob.export.implementation=localFile
+
# ======================================= Local File Blobs Exporting ========================================
# Optional, directory to store exported blob, directory path follows James file system format
# default is file://var/blobExporting
diff --git a/dockerfiles/run/guice/cassandra/destination/conf/blob.properties b/dockerfiles/run/guice/cassandra/destination/conf/blob.properties
index 2ec4035..c999f80 100644
--- a/dockerfiles/run/guice/cassandra/destination/conf/blob.properties
+++ b/dockerfiles/run/guice/cassandra/destination/conf/blob.properties
@@ -1,10 +1,9 @@
-# ============================================ Blobs Exporting ==============================================
-# Read https://james.apache.org/server/config-blob-export.html for further details
+# ============================================= BlobStore Implementation ==================================
+# Read https://james.apache.org/server/config-blobstore.html for further details
-# Choosing blob exporting mechanism, allowed mechanism are: localFile, linshare
-# LinShare is a file sharing service, will be explained in the below section
-# Optional, default is localFile
-blob.export.implementation=localFile
+# Choose your BlobStore implementation
+# Mandatory, allowed values are: cassandra, s3
+implementation=cassandra
# ========================================= ObjectStorage deduplication ========================================
# If you choose to enable deduplication, the mails with the same content will be stored only once.
@@ -13,6 +12,36 @@ blob.export.implementation=localFile
# Mandatory, Allowed values are: true, false
deduplication.enable=false
+# ========================================= Cassandra BlobStore Cache ======================================
+# A cassandra cache can be enabled to reduce latency when reading small blobs frequently
+# A dedicated keyspace with a replication factor of one is then used
+# Cache eviction policy is TTL based
+# Only blobs below a given threshold will be stored.
+# To be noted that blobs are stored within a single Cassandra row, hence a low threshold should be used.
+
+# Enable the cache? Optional and default to false. Must be a boolean.
+cache.enable=false
+
+# Cache eviction policy is TTL based. Optional and defaults to 7 days. Must be a duration.
+# Valid units: ms, sec, min, hour, day, week, month, year
+# cache.cassandra.ttl=7days
+
+# Timeout after which this cache should be bypassed. Optional and defaults to 100ms. Can not exceed 1 hour.
+# Must be a duration Valid units: ms, sec, min, hour, day, week, month, year
+# cache.cassandra.timeout=100ms
+
+# Maximum size of stored objects expressed in bytes. Must be strictly positive. Defaults to 8192.
+# Units: bytes, Kib, MiB, GiB, TiB
+# cache.sizeThresholdInBytes=8 KiB
+
+# ============================================ Blobs Exporting ==============================================
+# Read https://james.apache.org/server/config-blob-export.html for further details
+
+# Choosing blob exporting mechanism, allowed mechanism are: localFile, linshare
+# LinShare is a file sharing service, will be explained in the below section
+# Optional, default is localFile
+blob.export.implementation=localFile
+
# ======================================= Local File Blobs Exporting ========================================
# Optional, directory to store exported blob, directory path follows James file system format
# default is file://var/blobExporting
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 02/06: JAMES-3428 Configure read-repairs for
mailbox counters
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 3146cded977c70bfbe6e82604729e79a3bfa9722
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Wed Oct 14 14:25:35 2020 +0700
JAMES-3428 Configure read-repairs for mailbox counters
---
.../init/configuration/CassandraConfiguration.java | 59 +++++++++++++++++++++-
.../destination/conf/cassandra.properties | 6 +++
.../destination/conf/cassandra.properties | 6 +++
.../destination/conf/cassandra.properties | 6 +++
.../destination/conf/cassandra.properties | 6 +++
.../cassandra/mail/CassandraMessageMapper.java | 6 +--
6 files changed, 84 insertions(+), 5 deletions(-)
diff --git a/backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/CassandraConfiguration.java b/backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/CassandraConfiguration.java
index 1acf52d..99ac1a7 100644
--- a/backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/CassandraConfiguration.java
+++ b/backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/CassandraConfiguration.java
@@ -40,6 +40,8 @@ public class CassandraConfiguration {
public static final int DEFAULT_MESSAGE_CHUNK_SIZE_ON_READ = 100;
public static final float DEFAULT_MAILBOX_READ_REPAIR = 0.1f;
+ public static final float DEFAULT_MAX_MAILBOX_COUNTERS_READ_REPAIR_CHANCE = 0.1f;
+ public static final float DEFAULT_ONE_HUNDRED_MAILBOX_COUNTERS_READ_REPAIR_CHANCE = 0.01f;
public static final int DEFAULT_EXPUNGE_BATCH_SIZE = 50;
public static final int DEFAULT_UPDATE_FLAGS_BATCH_SIZE = 20;
public static final int DEFAULT_FLAGS_UPDATE_MESSAGE_MAX_RETRY = 1000;
@@ -57,6 +59,8 @@ public class CassandraConfiguration {
public static final List<String> VALID_CONSISTENCY_LEVEL_LIGHTWEIGHT_TRANSACTION = ImmutableList.of("SERIAL", "LOCAL_SERIAL");
private static final String MAILBOX_READ_REPAIR = "mailbox.read.repair.chance";
+ private static final String MAILBOX_MAX_COUNTERS_READ_REPAIR = "mailbox.counters.read.repair.chance.max";
+ private static final String MAILBOX_ONE_HUNDRED_COUNTERS_READ_REPAIR = "mailbox.counters.read.repair.chance.one.hundred";
private static final String MAILBOX_MAX_RETRY_ACL = "mailbox.max.retry.acl";
private static final String MAILBOX_MAX_RETRY_MODSEQ = "mailbox.max.retry.modseq";
private static final String MAILBOX_MAX_RETRY_UID = "mailbox.max.retry.uid";
@@ -88,6 +92,8 @@ public class CassandraConfiguration {
private Optional<String> consistencyLevelRegular = Optional.empty();
private Optional<String> consistencyLevelLightweightTransaction = Optional.empty();
private Optional<Float> mailboxReadRepair = Optional.empty();
+ private Optional<Float> mailboxCountersReadRepairMax = Optional.empty();
+ private Optional<Float> mailboxCountersReadRepairChanceOneHundred = Optional.empty();
public Builder messageReadChunkSize(int value) {
Preconditions.checkArgument(value > 0, "messageReadChunkSize needs to be strictly positive");
@@ -162,6 +168,20 @@ public class CassandraConfiguration {
return this;
}
+ public Builder mailboxCountersReadRepairMax(float value) {
+ Preconditions.checkArgument(value >= 0, "mailboxCountersReadRepairMax needs to be positive");
+ Preconditions.checkArgument(value <= 1, "mailboxCountersReadRepairMax needs to be less or equal to 1");
+ this.mailboxCountersReadRepairMax = Optional.of(value);
+ return this;
+ }
+
+ public Builder mailboxCountersReadRepairChanceOneHundred(float value) {
+ Preconditions.checkArgument(value >= 0, "mailboxCountersReadRepairChanceOneHundred needs to be positive");
+ Preconditions.checkArgument(value <= 1, "mailboxCountersReadRepairChanceOneHundred needs to be less or equal to 1");
+ this.mailboxCountersReadRepairChanceOneHundred = Optional.of(value);
+ return this;
+ }
+
public Builder messageReadChunkSize(Optional<Integer> value) {
value.ifPresent(this::messageReadChunkSize);
return this;
@@ -222,6 +242,16 @@ public class CassandraConfiguration {
return this;
}
+ public Builder mailboxCountersReadRepairMax(Optional<Float> value) {
+ value.ifPresent(this::mailboxCountersReadRepairMax);
+ return this;
+ }
+
+ public Builder mailboxCountersReadRepairChanceOneHundred(Optional<Float> value) {
+ value.ifPresent(this::mailboxCountersReadRepairChanceOneHundred);
+ return this;
+ }
+
public Builder consistencyLevelRegular(String value) {
Preconditions.checkArgument(VALID_CONSISTENCY_LEVEL_REGULAR.contains(value),
"consistencyLevelRegular needs to be one of the following: " + String.join(", ", VALID_CONSISTENCY_LEVEL_REGULAR));
@@ -268,7 +298,9 @@ public class CassandraConfiguration {
messageAttachmentIdsReadTimeout.orElse(DEFAULT_MESSAGE_ATTACHMENT_ID_MIGRATION_READ_TIMEOUT),
consistencyLevelRegular,
consistencyLevelLightweightTransaction,
- mailboxReadRepair.orElse(DEFAULT_MAILBOX_READ_REPAIR));
+ mailboxReadRepair.orElse(DEFAULT_MAILBOX_READ_REPAIR),
+ mailboxCountersReadRepairMax.orElse(DEFAULT_MAX_MAILBOX_COUNTERS_READ_REPAIR_CHANCE),
+ mailboxCountersReadRepairChanceOneHundred.orElse(DEFAULT_ONE_HUNDRED_MAILBOX_COUNTERS_READ_REPAIR_CHANCE));
}
}
@@ -306,6 +338,10 @@ public class CassandraConfiguration {
propertiesConfiguration.getString(CONSISTENCY_LEVEL_LIGHTWEIGHT_TRANSACTION)))
.mailboxReadRepair(Optional.ofNullable(
propertiesConfiguration.getFloat(MAILBOX_READ_REPAIR, null)))
+ .mailboxCountersReadRepairMax(Optional.ofNullable(
+ propertiesConfiguration.getFloat(MAILBOX_MAX_COUNTERS_READ_REPAIR, null)))
+ .mailboxCountersReadRepairChanceOneHundred(Optional.ofNullable(
+ propertiesConfiguration.getFloat(MAILBOX_ONE_HUNDRED_COUNTERS_READ_REPAIR, null)))
.build();
}
@@ -323,13 +359,17 @@ public class CassandraConfiguration {
private final String consistencyLevelRegular;
private final String consistencyLevelLightweightTransaction;
private final float mailboxReadRepair;
+ private final float mailboxCountersReadRepairChanceMax;
+ private final float mailboxCountersReadRepairChanceOneHundred;
@VisibleForTesting
CassandraConfiguration(int aclMaxRetry, int messageReadChunkSize, int expungeChunkSize,
int flagsUpdateMessageIdMaxRetry, int flagsUpdateMessageMaxRetry,
int modSeqMaxRetry, int uidMaxRetry, int fetchNextPageInAdvanceRow,
int blobPartSize, final int attachmentV2MigrationReadTimeout, int messageAttachmentIdsReadTimeout,
- String consistencyLevelRegular, String consistencyLevelLightweightTransaction, float mailboxReadRepair) {
+ String consistencyLevelRegular, String consistencyLevelLightweightTransaction,
+ float mailboxReadRepair, float mailboxCountersReadRepairChanceMax,
+ float mailboxCountersReadRepairChanceOneHundred) {
this.aclMaxRetry = aclMaxRetry;
this.messageReadChunkSize = messageReadChunkSize;
this.expungeChunkSize = expungeChunkSize;
@@ -344,6 +384,8 @@ public class CassandraConfiguration {
this.consistencyLevelRegular = consistencyLevelRegular;
this.consistencyLevelLightweightTransaction = consistencyLevelLightweightTransaction;
this.mailboxReadRepair = mailboxReadRepair;
+ this.mailboxCountersReadRepairChanceMax = mailboxCountersReadRepairChanceMax;
+ this.mailboxCountersReadRepairChanceOneHundred = mailboxCountersReadRepairChanceOneHundred;
}
public float getMailboxReadRepair() {
@@ -402,6 +444,14 @@ public class CassandraConfiguration {
return consistencyLevelLightweightTransaction;
}
+ public float getMailboxCountersReadRepairChanceMax() {
+ return mailboxCountersReadRepairChanceMax;
+ }
+
+ public float getMailboxCountersReadRepairChanceOneHundred() {
+ return mailboxCountersReadRepairChanceOneHundred;
+ }
+
@Override
public final boolean equals(Object o) {
if (o instanceof CassandraConfiguration) {
@@ -415,6 +465,8 @@ public class CassandraConfiguration {
&& Objects.equals(this.modSeqMaxRetry, that.modSeqMaxRetry)
&& Objects.equals(this.uidMaxRetry, that.uidMaxRetry)
&& Objects.equals(this.mailboxReadRepair, that.mailboxReadRepair)
+ && Objects.equals(this.mailboxCountersReadRepairChanceMax, that.mailboxCountersReadRepairChanceMax)
+ && Objects.equals(this.mailboxCountersReadRepairChanceOneHundred, that.mailboxCountersReadRepairChanceOneHundred)
&& Objects.equals(this.fetchNextPageInAdvanceRow, that.fetchNextPageInAdvanceRow)
&& Objects.equals(this.blobPartSize, that.blobPartSize)
&& Objects.equals(this.attachmentV2MigrationReadTimeout, that.attachmentV2MigrationReadTimeout)
@@ -429,6 +481,7 @@ public class CassandraConfiguration {
public final int hashCode() {
return Objects.hash(aclMaxRetry, messageReadChunkSize, expungeChunkSize, flagsUpdateMessageIdMaxRetry,
flagsUpdateMessageMaxRetry, modSeqMaxRetry, uidMaxRetry, fetchNextPageInAdvanceRow,
+ mailboxCountersReadRepairChanceOneHundred, mailboxCountersReadRepairChanceMax,
blobPartSize, attachmentV2MigrationReadTimeout, messageAttachmentIdsReadTimeout,
consistencyLevelRegular, consistencyLevelLightweightTransaction, mailboxReadRepair);
}
@@ -444,6 +497,8 @@ public class CassandraConfiguration {
.add("modSeqMaxRetry", modSeqMaxRetry)
.add("fetchNextPageInAdvanceRow", fetchNextPageInAdvanceRow)
.add("mailboxReadRepair", mailboxReadRepair)
+ .add("mailboxCountersReadRepairChanceOneHundred", mailboxCountersReadRepairChanceOneHundred)
+ .add("mailboxCountersReadRepairChanceMax", mailboxCountersReadRepairChanceMax)
.add("uidMaxRetry", uidMaxRetry)
.add("blobPartSize", blobPartSize)
.add("attachmentV2MigrationReadTimeout", attachmentV2MigrationReadTimeout)
diff --git a/dockerfiles/run/guice/cassandra-ldap/destination/conf/cassandra.properties b/dockerfiles/run/guice/cassandra-ldap/destination/conf/cassandra.properties
index 57f1108..59421ee 100644
--- a/dockerfiles/run/guice/cassandra-ldap/destination/conf/cassandra.properties
+++ b/dockerfiles/run/guice/cassandra-ldap/destination/conf/cassandra.properties
@@ -15,6 +15,12 @@ cassandra.retryConnection.minDelay=5000
## These low level tuning options controls performance related options in Cassandra related code
## The default values are documented here. Understand the effects before editing.
# mailbox.read.repair.chance=0.1
+## Maximum probability of running a read repair for the mailbox counters
+# mailbox.counters.read.repair.chance.max=0.1
+## Probability of running a mailbox counters read repair for 100 messages.
+## Inversely proportional to the count of messages
+## Formula: read_repair_chance = min(mailbox.counters.read.repair.chance.max, (100/unseens)*mailbox.counters.read.repair.chance.one.hundred)
+# mailbox.counters.read.repair.chance.one.hundred=0.01
# mailbox.max.retry.acl=1000
# mailbox.max.retry.modseq=100000
# mailbox.max.retry.uid=100000
diff --git a/dockerfiles/run/guice/cassandra-rabbitmq-ldap/destination/conf/cassandra.properties b/dockerfiles/run/guice/cassandra-rabbitmq-ldap/destination/conf/cassandra.properties
index dfd3940..6b17425 100644
--- a/dockerfiles/run/guice/cassandra-rabbitmq-ldap/destination/conf/cassandra.properties
+++ b/dockerfiles/run/guice/cassandra-rabbitmq-ldap/destination/conf/cassandra.properties
@@ -25,6 +25,12 @@ cassandra.retryConnection.minDelay=5000
## These low level tuning options controls performance related options in Cassandra related code
## The default values are documented here. Understand the effects before editing.
# mailbox.read.repair.chance=0.1
+## Maximum probability of running a read repair for the mailbox counters
+# mailbox.counters.read.repair.chance.max=0.1
+## Probability of running a mailbox counters read repair for 100 messages.
+## Inversely proportional to the count of messages
+## Formula: read_repair_chance = min(mailbox.counters.read.repair.chance.max, (100/unseens)*mailbox.counters.read.repair.chance.one.hundred)
+# mailbox.counters.read.repair.chance.one.hundred=0.01
# mailbox.max.retry.acl=1000
# mailbox.max.retry.modseq=100000
# mailbox.max.retry.uid=100000
diff --git a/dockerfiles/run/guice/cassandra-rabbitmq/destination/conf/cassandra.properties b/dockerfiles/run/guice/cassandra-rabbitmq/destination/conf/cassandra.properties
index dfd3940..6b17425 100644
--- a/dockerfiles/run/guice/cassandra-rabbitmq/destination/conf/cassandra.properties
+++ b/dockerfiles/run/guice/cassandra-rabbitmq/destination/conf/cassandra.properties
@@ -25,6 +25,12 @@ cassandra.retryConnection.minDelay=5000
## These low level tuning options controls performance related options in Cassandra related code
## The default values are documented here. Understand the effects before editing.
# mailbox.read.repair.chance=0.1
+## Maximum probability of running a read repair for the mailbox counters
+# mailbox.counters.read.repair.chance.max=0.1
+## Probability of running a mailbox counters read repair for 100 messages.
+## Inversely proportional to the count of messages
+## Formula: read_repair_chance = min(mailbox.counters.read.repair.chance.max, (100/unseens)*mailbox.counters.read.repair.chance.one.hundred)
+# mailbox.counters.read.repair.chance.one.hundred=0.01
# mailbox.max.retry.acl=1000
# mailbox.max.retry.modseq=100000
# mailbox.max.retry.uid=100000
diff --git a/dockerfiles/run/guice/cassandra/destination/conf/cassandra.properties b/dockerfiles/run/guice/cassandra/destination/conf/cassandra.properties
index 49a8560..797d4a7 100644
--- a/dockerfiles/run/guice/cassandra/destination/conf/cassandra.properties
+++ b/dockerfiles/run/guice/cassandra/destination/conf/cassandra.properties
@@ -25,6 +25,12 @@ cassandra.retryConnection.minDelay=5000
## These low level tuning options controls performance related options in Cassandra related code
## The default values are documented here. Understand the effects before editing.
# mailbox.read.repair.chance=0.1
+## Maximum probability of running a read repair for the mailbox counters
+# mailbox.counters.read.repair.chance.max=0.1
+## Probability of running a mailbox counters read repair for 100 messages.
+## Inversely proportional to the count of messages
+## Formula: read_repair_chance = min(mailbox.counters.read.repair.chance.max, (100/unseens)*mailbox.counters.read.repair.chance.one.hundred)
+# mailbox.counters.read.repair.chance.one.hundred=0.01
# mailbox.max.retry.acl=1000
# mailbox.max.retry.modseq=100000
# mailbox.max.retry.uid=100000
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapper.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapper.java
index b72d56d..df29d11 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapper.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraMessageMapper.java
@@ -172,9 +172,9 @@ public class CassandraMessageMapper implements MessageMapper {
}
private boolean shouldReadRepair(MailboxCounters counters) {
- double readRepairChance = 0.1;
- double ponderatedReadRepairChance = readRepairChance * (10.0 / counters.getUnseen());
- return secureRandom.nextFloat() < Math.min(readRepairChance, ponderatedReadRepairChance);
+ double ponderedReadRepairChance = cassandraConfiguration.getMailboxCountersReadRepairChanceOneHundred() * (100.0 / counters.getUnseen());
+ return secureRandom.nextFloat() < Math.min(
+ cassandraConfiguration.getMailboxCountersReadRepairChanceMax(), ponderedReadRepairChance);
}
@Override
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[james-project] 05/06: JAMES-3415 Modified ADR and README.md for
James Webadmin CLI
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 1a3e49475c59a309c5e510cb4d474f5d1f81669f
Author: quanth <hq...@linagora.com>
AuthorDate: Fri Oct 16 10:22:46 2020 +0700
JAMES-3415 Modified ADR and README.md for James Webadmin CLI
---
server/protocols/webadmin-cli/README.md | 23 ++++++++++++-----------
src/adr/0042-james-cli-based-on-webadmin.md | 6 ++----
2 files changed, 14 insertions(+), 15 deletions(-)
diff --git a/server/protocols/webadmin-cli/README.md b/server/protocols/webadmin-cli/README.md
index 3b4e5d0..ab8185b 100644
--- a/server/protocols/webadmin-cli/README.md
+++ b/server/protocols/webadmin-cli/README.md
@@ -11,7 +11,7 @@ Webadmin command-line interface is an upcoming replacement for the outdated, sec
General syntax to run the command line
```
-$ james-cli [OPTION] ENTITY ACTION {ARGUMENT}
+$ java -jar james-cli.jar [OPTION] ENTITY ACTION {ARGUMENT}
```
where
@@ -26,13 +26,15 @@ where
Example:
```
-$ james-cli --url http://127.0.0.1 --port 9999 domain list
+$ java -jar james-cli.jar --url http://127.0.0.1 --port 9999 domain list
```
The above command lists all domain names available on domain route at address http://127.0.0.1:9999.
It does not require any argument to execute. Options --url and --port are optional. Without them, the default value is http://127.0.0.0:8000.
As for other commands, arguments might be required after the sub-command (ACTION such as list, add and remove).
+Note: the command line before ENTITY will be documented as {cli}.
+
## Navigation menu
- [Manage Domains](#manage-domains)
@@ -52,7 +54,7 @@ As for other commands, arguments might be required after the sub-command (ACTION
### Create a domain
Add a domain to the domain list.
```
-james-cli domain create <domainToBeCreated>
+{cli} domain create <domainToBeCreated>
```
Resource name **domainToBeCreated**:
@@ -65,20 +67,20 @@ Resource name **domainToBeCreated**:
Remove a domain from the domain list.
```
-james-cli domain delete <domainToBeDeleted>
+{cli} domain delete <domainToBeDeleted>
```
Note: Deletion of an auto-detected domain, default domain or of an auto-detected ip is not supported. We encourage you instead to review your [domain list configuration](https://james.apache.org/server/config-domainlist.html).
### Check if a domain exists
Check whether a domain exists on the domain list or not.
```
-james-cli domain exist <domainToBeChecked>
+{cli} domain exist <domainToBeChecked>
```
### Get the list of domains
Show all domains' name on the list.
```
-james-cli domain list
+{cli} domain list
```
@@ -89,7 +91,7 @@ james-cli domain list
Add an user to the user list.
```
-james-cli user create <username> <password>
+{cli} user create <username> <password>
```
Resource name <username> representing valid users, hence it should match the criteria at [User Repositories documentation](https://james.apache.org/server/config-users.html)
@@ -99,7 +101,7 @@ Note: if the user exists already, its password will be updated.
Check whether an user exists on the user list or not.
```
-james-cli user exist <username>
+{cli} user exist <username>
```
Resource name <username> representing valid users, hence it should match the criteria at [User Repositories documentation](https://james.apache.org/server/config-users.html)
@@ -107,7 +109,7 @@ Resource name <username> representing valid users, hence it should match the cri
Remove an user from the user list.
```
-james-cli user delete <username>
+{cli} user delete <username>
```
### Get users list
@@ -115,7 +117,7 @@ james-cli user delete <username>
Show all users' name on the list.
```
-james-cli user list
+{cli} user list
```
### Update an user password
@@ -123,4 +125,3 @@ Same as Create, but an user need to exist.
If the user do not exist, then it will be created.
-
diff --git a/src/adr/0042-james-cli-based-on-webadmin.md b/src/adr/0042-james-cli-based-on-webadmin.md
index 52b1c9d..8ce5562 100644
--- a/src/adr/0042-james-cli-based-on-webadmin.md
+++ b/src/adr/0042-james-cli-based-on-webadmin.md
@@ -34,8 +34,6 @@ We decided to write a new CLI client, running on top of the JVM, communicating w
* server/protocols/webadmin-cli
-* Distribute this CLI as a GraalVM native image.
-
* Write a man page.
* Picocli generates beautiful documentation for our CLI (HTML, PDF and Unix man pages).
@@ -43,7 +41,7 @@ We decided to write a new CLI client, running on top of the JVM, communicating w
* We decided to adopt a more modern, modular CLI syntax:
```
-$ james-cli [OPTION] ENTITY ACTION {ARGUMENT}
+$ java -jar james-cli.jar [OPTION] ENTITY ACTION {ARGUMENT}
```
where
@@ -59,7 +57,7 @@ where
Add a domain to the domain list.
```
-$ james-cli --url http://127.0.0.1 --port 9999 domain create domainNameToBeCreated
+$ java -jar james-cli.jar --url http://127.0.0.1 --port 9999 domain create domainNameToBeCreated
```
In above command-line
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org