You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by bt...@apache.org on 2019/11/01 10:51:21 UTC
[james-project] branch master updated (3d21b8b -> 70fb5d1)
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git.
from 3d21b8b JAMES-2936 reject path with empty names in the hierachy when renaming mailbox and sanitize new name
new 0fff261 JAMES-2917 Make To sort testing less context-dependent
new 5b6427a JAMES-2917 Rely on ElasticSearch routing key
new 6129d92 JAMES-2917 Require ElasticSearch routing upon index
new 0d72783 JAMES-2917 Solve some RequestOptions related deprecation warnings
new 4172365 JAMES-2917 Document breaking changes in changelog + upgrade instructions
new 70fb5d1 JAMES-2917 Factorize RoutingKey generation logic
The 6 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
CHANGELOG.md | 5 ++
.../james/backends/es/DeleteByQueryPerformer.java | 20 +++--
.../org/apache/james/backends/es/DocumentId.java | 29 +++----
.../james/backends/es/ElasticSearchIndexer.java | 39 ++++++----
.../james/backends/es/NodeMappingFactory.java | 2 +
.../org/apache/james/backends/es/RoutingKey.java | 38 ++++-----
.../james/backends/es/UpdatedRepresentation.java | 8 +-
.../james/backends/es/search/ScrolledSearch.java | 8 +-
.../apache/james/backends/es/DocumentIdTest.java | 24 ++----
.../backends/es/ElasticSearchIndexerTest.java | 90 ++++++++++++----------
.../apache/james/backends/es/RoutingKeyTest.java | 24 ++----
.../elasticsearch/MailboxIdRoutingKeyFactory.java} | 12 +--
.../elasticsearch/MailboxMappingFactory.java | 6 ++
.../ElasticSearchListeningMessageSearchIndex.java | 41 ++++++----
.../search/ElasticSearchSearcher.java | 19 ++++-
.../ElasticSearchIntegrationTest.java | 5 +-
.../MailboxIdRoutingKeyFactoryTest.java} | 15 ++--
...asticSearchListeningMessageSearchIndexTest.java | 8 +-
.../elasticsearch/QuotaRatioMappingFactory.java | 6 ++
.../elasticsearch/UserRoutingKeyFactory.java} | 13 ++--
.../events/ElasticSearchQuotaMailboxListener.java | 21 +++--
...lasticSearchQuotaSearchTestSystemExtension.java | 3 +-
.../elasticsearch/UserRoutingKeyFactoryTest.java} | 15 ++--
.../ElasticSearchQuotaMailboxListenerTest.java | 4 +-
.../search/AbstractMessageSearchIndexTest.java | 6 +-
mailbox/store/src/test/resources/eml/mail1.eml | 2 +-
mailbox/store/src/test/resources/eml/mail2.eml | 2 +-
mailbox/store/src/test/resources/eml/mail3.eml | 2 +-
mailbox/store/src/test/resources/eml/mail4.eml | 2 +-
.../host/ElasticSearchHostSystem.java | 6 +-
.../mailbox/ElasticSearchMailboxModule.java | 10 ++-
.../mailbox/ElasticSearchQuotaSearcherModule.java | 4 +-
.../routes/ElasticSearchQuotaSearchExtension.java | 4 +-
upgrade-instructions.md | 23 ++++++
34 files changed, 311 insertions(+), 205 deletions(-)
copy server/data/data-api/src/main/java/org/apache/james/sieverepository/api/ScriptName.java => backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/DocumentId.java (75%)
copy server/data/data-api/src/main/java/org/apache/james/sieverepository/api/ScriptContent.java => backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/RoutingKey.java (73%)
copy server/blob/blob-api/src/test/java/org/apache/james/blob/api/BucketNameTest.java => backends-common/elasticsearch/src/test/java/org/apache/james/backends/es/DocumentIdTest.java (74%)
copy server/blob/blob-api/src/test/java/org/apache/james/blob/api/BucketNameTest.java => backends-common/elasticsearch/src/test/java/org/apache/james/backends/es/RoutingKeyTest.java (74%)
copy mailbox/{store/src/main/java/org/apache/james/mailbox/store/mail/model/MailboxIdDeserializer.java => elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/MailboxIdRoutingKeyFactory.java} (80%)
copy mailbox/{backup/src/test/java/org/apache/james/mailbox/backup/DirectoryTest.java => elasticsearch/src/test/java/org/apache/james/mailbox/elasticsearch/MailboxIdRoutingKeyFactoryTest.java} (78%)
copy mailbox/plugin/{quota-search/src/main/java/org/apache/james/quota/search/QuotaSearcher.java => quota-search-elasticsearch/src/main/java/org/apache/james/quota/search/elasticsearch/UserRoutingKeyFactory.java} (81%)
copy mailbox/{backup/src/test/java/org/apache/james/mailbox/backup/DirectoryTest.java => plugin/quota-search-elasticsearch/src/test/java/org/apache/james/quota/search/elasticsearch/UserRoutingKeyFactoryTest.java} (79%)
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[james-project] 01/06: JAMES-2917 Make To sort testing less
context-dependent
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 0fff2612df3a2ee47e3f86938ea09ea4e6873e43
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Thu Oct 10 10:30:58 2019 +0700
JAMES-2917 Make To sort testing less context-dependent
---
.../james/mailbox/store/search/AbstractMessageSearchIndexTest.java | 6 +++---
mailbox/store/src/test/resources/eml/mail1.eml | 2 +-
mailbox/store/src/test/resources/eml/mail2.eml | 2 +-
mailbox/store/src/test/resources/eml/mail3.eml | 2 +-
mailbox/store/src/test/resources/eml/mail4.eml | 2 +-
5 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/mailbox/store/src/test/java/org/apache/james/mailbox/store/search/AbstractMessageSearchIndexTest.java b/mailbox/store/src/test/java/org/apache/james/mailbox/store/search/AbstractMessageSearchIndexTest.java
index 770ff29..6447070 100644
--- a/mailbox/store/src/test/java/org/apache/james/mailbox/store/search/AbstractMessageSearchIndexTest.java
+++ b/mailbox/store/src/test/java/org/apache/james/mailbox/store/search/AbstractMessageSearchIndexTest.java
@@ -1115,9 +1115,9 @@ public abstract class AbstractMessageSearchIndexTest {
assertThat(messageSearchIndex.search(session, mailbox, searchQuery))
.containsExactly(m5.getUid(), m3.getUid(), m2.getUid(), m4.getUid());
// 5 : "zzz" <ma...@james.apache.org>
- // 3 : "aaa" <se...@james.apache.org>
- // 2 : "abc" <se...@james.apache.org>
- // 4 : "server" <se...@james.apache.org>
+ // 3 : "aaa" <a-...@james.apache.org>
+ // 2 : "abc" <b-...@james.apache.org>
+ // 4 : "server" <c-...@james.apache.org>
}
@Test
diff --git a/mailbox/store/src/test/resources/eml/mail1.eml b/mailbox/store/src/test/resources/eml/mail1.eml
index 33df93b..1e984ae 100644
--- a/mailbox/store/src/test/resources/eml/mail1.eml
+++ b/mailbox/store/src/test/resources/eml/mail1.eml
@@ -34,7 +34,7 @@ Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28)
by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 04 Jun 2015 09:23:38 +0000
Date: Thu, 4 Jun 2015 09:23:37 +0000 (UTC)
From: "Tellier Benoit (JIRA)" <ji...@apache.org>
-To: "abc" <se...@james.apache.org>
+To: "abc" <b-...@james.apache.org>
Message-ID: <JI...@Atlassian.JIRA>
In-Reply-To: <JI...@Atlassian.JIRA>
References: <JI...@Atlassian.JIRA> <JI...@arcas>
diff --git a/mailbox/store/src/test/resources/eml/mail2.eml b/mailbox/store/src/test/resources/eml/mail2.eml
index 6539248..6cd6f4c 100644
--- a/mailbox/store/src/test/resources/eml/mail2.eml
+++ b/mailbox/store/src/test/resources/eml/mail2.eml
@@ -34,7 +34,7 @@ Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28)
by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 04 Jun 2015 09:27:38 +0000
Date: Thu, 4 Jun 2015 09:27:37 +0000 (UTC)
From: "efij" <ji...@apache.org>
-To: "aaa" <se...@james.apache.org>
+To: "aaa" <a-...@james.apache.org>
Cc: abc@abc.org
Message-ID: <JI...@Atlassian.JIRA>
In-Reply-To: <JI...@Atlassian.JIRA>
diff --git a/mailbox/store/src/test/resources/eml/mail3.eml b/mailbox/store/src/test/resources/eml/mail3.eml
index 99f2123..01f9600 100644
--- a/mailbox/store/src/test/resources/eml/mail3.eml
+++ b/mailbox/store/src/test/resources/eml/mail3.eml
@@ -37,7 +37,7 @@ Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28)
Date: Tue, 2 Jun 2015 08:16:19 +0000 (UTC)
From: "abcd" <ji...@apache.org>
Cc: zzz@bcd.org
-To: "server" <se...@james.apache.org>
+To: "server" <c-...@james.apache.org>
Message-ID: <JI...@Atlassian.JIRA>
In-Reply-To: <JI...@Atlassian.JIRA>
References: <JI...@Atlassian.JIRA> <JI...@arcas>
diff --git a/mailbox/store/src/test/resources/eml/mail4.eml b/mailbox/store/src/test/resources/eml/mail4.eml
index ad2a5f6..b4fe825 100644
--- a/mailbox/store/src/test/resources/eml/mail4.eml
+++ b/mailbox/store/src/test/resources/eml/mail4.eml
@@ -36,7 +36,7 @@ Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28)
by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 15 May 2015 06:36:00 +0000
Date: Fri, 15 May 2015 06:35:59 +0000 (UTC)
From: "Eric Charles (JIRA)" <ma...@james.apache.org>
-To: "zzz" <ma...@james.apache.org>
+To: "zzz" <a-...@james.apache.org>
Cc: monkey@any.com
Bcc: monkey@any.com
Message-ID: <JI...@Atlassian.JIRA>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[james-project] 06/06: JAMES-2917 Factorize RoutingKey generation
logic
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 70fb5d12f92024a07b73474028db92b25c29d129
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Sat Oct 12 10:45:58 2019 +0700
JAMES-2917 Factorize RoutingKey generation logic
---
.../org/apache/james/backends/es/RoutingKey.java | 4 ++
.../elasticsearch/MailboxIdRoutingKeyFactory.java | 41 +++----------------
.../ElasticSearchListeningMessageSearchIndex.java | 16 ++++----
.../search/ElasticSearchSearcher.java | 10 ++++-
.../ElasticSearchIntegrationTest.java | 5 ++-
.../MailboxIdRoutingKeyFactoryTest.java | 47 +++++-----------------
...asticSearchListeningMessageSearchIndexTest.java | 8 ++--
.../elasticsearch/UserRoutingKeyFactory.java | 41 +++----------------
.../events/ElasticSearchQuotaMailboxListener.java | 14 +++----
...lasticSearchQuotaSearchTestSystemExtension.java | 3 +-
.../elasticsearch/UserRoutingKeyFactoryTest.java | 47 +++++-----------------
.../ElasticSearchQuotaMailboxListenerTest.java | 4 +-
.../host/ElasticSearchHostSystem.java | 6 ++-
.../mailbox/ElasticSearchMailboxModule.java | 10 ++++-
.../mailbox/ElasticSearchQuotaSearcherModule.java | 4 +-
.../routes/ElasticSearchQuotaSearchExtension.java | 4 +-
16 files changed, 90 insertions(+), 174 deletions(-)
diff --git a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/RoutingKey.java b/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/RoutingKey.java
index 05021e5..6ce3598 100644
--- a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/RoutingKey.java
+++ b/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/RoutingKey.java
@@ -26,6 +26,10 @@ import org.elasticsearch.common.Strings;
import com.google.common.base.Preconditions;
public class RoutingKey {
+ public interface Factory<T> {
+ RoutingKey from(T t);
+ }
+
public static RoutingKey fromString(String value) {
return new RoutingKey(value);
}
diff --git a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/RoutingKey.java b/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/MailboxIdRoutingKeyFactory.java
similarity index 58%
copy from backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/RoutingKey.java
copy to mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/MailboxIdRoutingKeyFactory.java
index 05021e5..56fc834 100644
--- a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/RoutingKey.java
+++ b/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/MailboxIdRoutingKeyFactory.java
@@ -17,43 +17,14 @@
* under the License. *
****************************************************************/
-package org.apache.james.backends.es;
+package org.apache.james.mailbox.elasticsearch;
-import java.util.Objects;
-
-import org.elasticsearch.common.Strings;
-
-import com.google.common.base.Preconditions;
-
-public class RoutingKey {
- public static RoutingKey fromString(String value) {
- return new RoutingKey(value);
- }
-
-
- private final String value;
-
- private RoutingKey(String value) {
- Preconditions.checkArgument(!Strings.isNullOrEmpty(value), "RoutingKey must be specified");
- this.value = value;
- }
-
- public String asString() {
- return value;
- }
-
- @Override
- public final boolean equals(Object o) {
- if (o instanceof RoutingKey) {
- RoutingKey that = (RoutingKey) o;
-
- return Objects.equals(this.value, that.value);
- }
- return false;
- }
+import org.apache.james.backends.es.RoutingKey;
+import org.apache.james.mailbox.model.MailboxId;
+public class MailboxIdRoutingKeyFactory implements RoutingKey.Factory<MailboxId> {
@Override
- public final int hashCode() {
- return Objects.hash(value);
+ public RoutingKey from(MailboxId mailboxId) {
+ return RoutingKey.fromString(mailboxId.serialize());
}
}
diff --git a/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/events/ElasticSearchListeningMessageSearchIndex.java b/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/events/ElasticSearchListeningMessageSearchIndex.java
index c695b85..7cf681a 100644
--- a/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/events/ElasticSearchListeningMessageSearchIndex.java
+++ b/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/events/ElasticSearchListeningMessageSearchIndex.java
@@ -75,16 +75,18 @@ public class ElasticSearchListeningMessageSearchIndex extends ListeningMessageSe
private final ElasticSearchIndexer elasticSearchIndexer;
private final ElasticSearchSearcher searcher;
private final MessageToElasticSearchJson messageToElasticSearchJson;
+ private final RoutingKey.Factory<MailboxId> routingKeyFactory;
@Inject
public ElasticSearchListeningMessageSearchIndex(MailboxSessionMapperFactory factory,
@Named(MailboxElasticSearchConstants.InjectionNames.MAILBOX) ElasticSearchIndexer indexer,
ElasticSearchSearcher searcher, MessageToElasticSearchJson messageToElasticSearchJson,
- SessionProvider sessionProvider) {
+ SessionProvider sessionProvider, RoutingKey.Factory<MailboxId> routingKeyFactory) {
super(factory, sessionProvider);
this.elasticSearchIndexer = indexer;
this.messageToElasticSearchJson = messageToElasticSearchJson;
this.searcher = searcher;
+ this.routingKeyFactory = routingKeyFactory;
}
@Override
@@ -142,7 +144,7 @@ public class ElasticSearchListeningMessageSearchIndex extends ListeningMessageSe
String jsonContent = generateIndexedJson(mailbox, message, session);
- elasticSearchIndexer.index(indexIdFor(mailbox, message.getUid()), jsonContent, toRoutingKey(mailbox.getMailboxId()));
+ elasticSearchIndexer.index(indexIdFor(mailbox, message.getUid()), jsonContent, routingKeyFactory.from(mailbox.getMailboxId()));
}
private String generateIndexedJson(Mailbox mailbox, MailboxMessage message, MailboxSession session) throws JsonProcessingException {
@@ -165,7 +167,7 @@ public class ElasticSearchListeningMessageSearchIndex extends ListeningMessageSe
.delete(expungedUids.stream()
.map(uid -> indexIdFor(mailbox, uid))
.collect(Guavate.toImmutableList()),
- toRoutingKey(mailbox.getMailboxId()));
+ routingKeyFactory.from(mailbox.getMailboxId()));
}
@Override
@@ -175,7 +177,7 @@ public class ElasticSearchListeningMessageSearchIndex extends ListeningMessageSe
mailboxId.serialize());
elasticSearchIndexer
- .deleteAllMatchingQuery(queryBuilder, toRoutingKey(mailboxId));
+ .deleteAllMatchingQuery(queryBuilder, routingKeyFactory.from(mailboxId));
}
@Override
@@ -186,7 +188,7 @@ public class ElasticSearchListeningMessageSearchIndex extends ListeningMessageSe
.sneakyThrow())
.collect(Guavate.toImmutableList());
- elasticSearchIndexer.update(updates, toRoutingKey(mailbox.getMailboxId()));
+ elasticSearchIndexer.update(updates, routingKeyFactory.from(mailbox.getMailboxId()));
}
private UpdatedRepresentation createUpdatedDocumentPartFromUpdatedFlags(Mailbox mailbox, UpdatedFlags updatedFlags) throws JsonProcessingException {
@@ -205,8 +207,4 @@ public class ElasticSearchListeningMessageSearchIndex extends ListeningMessageSe
LOGGER.error("No messageUid for {} in mailbox {}", searchResult.getMessageUid(), searchResult.getMailboxId());
}
}
-
- private RoutingKey toRoutingKey(MailboxId mailboxId) {
- return RoutingKey.fromString(mailboxId.serialize());
- }
}
diff --git a/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/search/ElasticSearchSearcher.java b/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/search/ElasticSearchSearcher.java
index 48b67e4..bd3e629 100644
--- a/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/search/ElasticSearchSearcher.java
+++ b/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/search/ElasticSearchSearcher.java
@@ -26,6 +26,7 @@ import java.util.stream.Stream;
import org.apache.james.backends.es.AliasName;
import org.apache.james.backends.es.NodeMappingFactory;
import org.apache.james.backends.es.ReadAliasName;
+import org.apache.james.backends.es.RoutingKey;
import org.apache.james.backends.es.search.ScrolledSearch;
import org.apache.james.mailbox.MessageUid;
import org.apache.james.mailbox.elasticsearch.json.JsonMessageConstants;
@@ -59,16 +60,18 @@ public class ElasticSearchSearcher {
private final MailboxId.Factory mailboxIdFactory;
private final MessageId.Factory messageIdFactory;
private final AliasName aliasName;
+ private final RoutingKey.Factory<MailboxId> routingKeyFactory;
public ElasticSearchSearcher(RestHighLevelClient client, QueryConverter queryConverter, int size,
MailboxId.Factory mailboxIdFactory, MessageId.Factory messageIdFactory,
- ReadAliasName aliasName) {
+ ReadAliasName aliasName, RoutingKey.Factory<MailboxId> routingKeyFactory) {
this.client = client;
this.queryConverter = queryConverter;
this.size = size;
this.mailboxIdFactory = mailboxIdFactory;
this.messageIdFactory = messageIdFactory;
this.aliasName = aliasName;
+ this.routingKeyFactory = routingKeyFactory;
}
public Stream<MessageSearchIndex.SearchResult> search(Collection<MailboxId> mailboxIds, SearchQuery query,
@@ -101,7 +104,10 @@ public class ElasticSearchSearcher {
}
private String[] toRoutingKeys(Collection<MailboxId> mailboxIds) {
- return mailboxIds.stream().map(MailboxId::serialize).toArray(String[]::new);
+ return mailboxIds.stream()
+ .map(routingKeyFactory::from)
+ .map(RoutingKey::asString)
+ .toArray(String[]::new);
}
private int computeRequiredSize(Optional<Integer> limit) {
diff --git a/mailbox/elasticsearch/src/test/java/org/apache/james/mailbox/elasticsearch/ElasticSearchIntegrationTest.java b/mailbox/elasticsearch/src/test/java/org/apache/james/mailbox/elasticsearch/ElasticSearchIntegrationTest.java
index 55fca6a..8643f1b 100644
--- a/mailbox/elasticsearch/src/test/java/org/apache/james/mailbox/elasticsearch/ElasticSearchIntegrationTest.java
+++ b/mailbox/elasticsearch/src/test/java/org/apache/james/mailbox/elasticsearch/ElasticSearchIntegrationTest.java
@@ -97,6 +97,7 @@ public class ElasticSearchIntegrationTest extends AbstractMessageSearchIndexTest
elasticSearch.getDockerElasticSearch().configuration());
InMemoryMessageId.Factory messageIdFactory = new InMemoryMessageId.Factory();
+ MailboxIdRoutingKeyFactory routingKeyFactory = new MailboxIdRoutingKeyFactory();
InMemoryIntegrationResources resources = InMemoryIntegrationResources.builder()
.preProvisionnedFakeAuthenticator()
@@ -111,9 +112,9 @@ public class ElasticSearchIntegrationTest extends AbstractMessageSearchIndexTest
BATCH_SIZE),
new ElasticSearchSearcher(client, new QueryConverter(new CriterionConverter()), SEARCH_SIZE,
new InMemoryId.Factory(), messageIdFactory,
- MailboxElasticSearchConstants.DEFAULT_MAILBOX_READ_ALIAS),
+ MailboxElasticSearchConstants.DEFAULT_MAILBOX_READ_ALIAS, routingKeyFactory),
new MessageToElasticSearchJson(textExtractor, ZoneId.of("Europe/Paris"), IndexAttachments.YES),
- preInstanciationStage.getSessionProvider()))
+ preInstanciationStage.getSessionProvider(), routingKeyFactory))
.noPreDeletionHooks()
.storeQuotaManager()
.build();
diff --git a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/RoutingKey.java b/mailbox/elasticsearch/src/test/java/org/apache/james/mailbox/elasticsearch/MailboxIdRoutingKeyFactoryTest.java
similarity index 57%
copy from backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/RoutingKey.java
copy to mailbox/elasticsearch/src/test/java/org/apache/james/mailbox/elasticsearch/MailboxIdRoutingKeyFactoryTest.java
index 05021e5..b666ffa 100644
--- a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/RoutingKey.java
+++ b/mailbox/elasticsearch/src/test/java/org/apache/james/mailbox/elasticsearch/MailboxIdRoutingKeyFactoryTest.java
@@ -17,43 +17,18 @@
* under the License. *
****************************************************************/
-package org.apache.james.backends.es;
+package org.apache.james.mailbox.elasticsearch;
-import java.util.Objects;
+import static org.assertj.core.api.Assertions.assertThat;
-import org.elasticsearch.common.Strings;
+import org.apache.james.backends.es.RoutingKey;
+import org.apache.james.mailbox.model.TestId;
+import org.junit.jupiter.api.Test;
-import com.google.common.base.Preconditions;
-
-public class RoutingKey {
- public static RoutingKey fromString(String value) {
- return new RoutingKey(value);
- }
-
-
- private final String value;
-
- private RoutingKey(String value) {
- Preconditions.checkArgument(!Strings.isNullOrEmpty(value), "RoutingKey must be specified");
- this.value = value;
- }
-
- public String asString() {
- return value;
- }
-
- @Override
- public final boolean equals(Object o) {
- if (o instanceof RoutingKey) {
- RoutingKey that = (RoutingKey) o;
-
- return Objects.equals(this.value, that.value);
- }
- return false;
- }
-
- @Override
- public final int hashCode() {
- return Objects.hash(value);
+class MailboxIdRoutingKeyFactoryTest {
+ @Test
+ void fromShouldRelyOnSerializedMailboxId() {
+ assertThat(new MailboxIdRoutingKeyFactory().from(TestId.of(5)))
+ .isEqualTo(RoutingKey.fromString("5"));
}
-}
+}
\ No newline at end of file
diff --git a/mailbox/elasticsearch/src/test/java/org/apache/james/mailbox/elasticsearch/events/ElasticSearchListeningMessageSearchIndexTest.java b/mailbox/elasticsearch/src/test/java/org/apache/james/mailbox/elasticsearch/events/ElasticSearchListeningMessageSearchIndexTest.java
index e2df49a..a126eb9 100644
--- a/mailbox/elasticsearch/src/test/java/org/apache/james/mailbox/elasticsearch/events/ElasticSearchListeningMessageSearchIndexTest.java
+++ b/mailbox/elasticsearch/src/test/java/org/apache/james/mailbox/elasticsearch/events/ElasticSearchListeningMessageSearchIndexTest.java
@@ -38,6 +38,7 @@ import org.apache.james.mailbox.MailboxSession;
import org.apache.james.mailbox.MessageUid;
import org.apache.james.mailbox.elasticsearch.IndexAttachments;
import org.apache.james.mailbox.elasticsearch.MailboxElasticSearchConstants;
+import org.apache.james.mailbox.elasticsearch.MailboxIdRoutingKeyFactory;
import org.apache.james.mailbox.elasticsearch.MailboxIndexCreationUtil;
import org.apache.james.mailbox.elasticsearch.json.MessageToElasticSearchJson;
import org.apache.james.mailbox.elasticsearch.query.CriterionConverter;
@@ -160,7 +161,8 @@ public class ElasticSearchListeningMessageSearchIndexTest {
ElasticSearchSearcher.DEFAULT_SEARCH_SIZE,
new InMemoryId.Factory(),
messageIdFactory,
- MailboxElasticSearchConstants.DEFAULT_MAILBOX_READ_ALIAS);
+ MailboxElasticSearchConstants.DEFAULT_MAILBOX_READ_ALIAS,
+ new MailboxIdRoutingKeyFactory());
FakeAuthenticator fakeAuthenticator = new FakeAuthenticator();
fakeAuthenticator.addUser(ManagerTestProvisionner.USER, ManagerTestProvisionner.USER_PASS);
@@ -170,7 +172,7 @@ public class ElasticSearchListeningMessageSearchIndexTest {
elasticSearchIndexer = new ElasticSearchIndexer(client, MailboxElasticSearchConstants.DEFAULT_MAILBOX_WRITE_ALIAS);
testee = new ElasticSearchListeningMessageSearchIndex(mapperFactory, elasticSearchIndexer, elasticSearchSearcher,
- messageToElasticSearchJson, sessionProvider);
+ messageToElasticSearchJson, sessionProvider, new MailboxIdRoutingKeyFactory());
session = sessionProvider.createSystemSession(USERNAME);
mailbox = new Mailbox(MailboxPath.forUser(USERNAME, DefaultMailboxes.INBOX), MAILBOX_ID.id);
@@ -236,7 +238,7 @@ public class ElasticSearchListeningMessageSearchIndexTest {
IndexAttachments.YES);
testee = new ElasticSearchListeningMessageSearchIndex(mapperFactory, elasticSearchIndexer, elasticSearchSearcher,
- messageToElasticSearchJson, sessionProvider);
+ messageToElasticSearchJson, sessionProvider, new MailboxIdRoutingKeyFactory());
testee.add(session, mailbox, MESSAGE_WITH_ATTACHMENT);
elasticSearch.awaitForElasticSearch();
diff --git a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/RoutingKey.java b/mailbox/plugin/quota-search-elasticsearch/src/main/java/org/apache/james/quota/search/elasticsearch/UserRoutingKeyFactory.java
similarity index 58%
copy from backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/RoutingKey.java
copy to mailbox/plugin/quota-search-elasticsearch/src/main/java/org/apache/james/quota/search/elasticsearch/UserRoutingKeyFactory.java
index 05021e5..46abf86 100644
--- a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/RoutingKey.java
+++ b/mailbox/plugin/quota-search-elasticsearch/src/main/java/org/apache/james/quota/search/elasticsearch/UserRoutingKeyFactory.java
@@ -17,43 +17,14 @@
* under the License. *
****************************************************************/
-package org.apache.james.backends.es;
+package org.apache.james.quota.search.elasticsearch;
-import java.util.Objects;
-
-import org.elasticsearch.common.Strings;
-
-import com.google.common.base.Preconditions;
-
-public class RoutingKey {
- public static RoutingKey fromString(String value) {
- return new RoutingKey(value);
- }
-
-
- private final String value;
-
- private RoutingKey(String value) {
- Preconditions.checkArgument(!Strings.isNullOrEmpty(value), "RoutingKey must be specified");
- this.value = value;
- }
-
- public String asString() {
- return value;
- }
-
- @Override
- public final boolean equals(Object o) {
- if (o instanceof RoutingKey) {
- RoutingKey that = (RoutingKey) o;
-
- return Objects.equals(this.value, that.value);
- }
- return false;
- }
+import org.apache.james.backends.es.RoutingKey;
+import org.apache.james.core.User;
+public class UserRoutingKeyFactory implements RoutingKey.Factory<User> {
@Override
- public final int hashCode() {
- return Objects.hash(value);
+ public RoutingKey from(User user) {
+ return RoutingKey.fromString(user.asString());
}
}
diff --git a/mailbox/plugin/quota-search-elasticsearch/src/main/java/org/apache/james/quota/search/elasticsearch/events/ElasticSearchQuotaMailboxListener.java b/mailbox/plugin/quota-search-elasticsearch/src/main/java/org/apache/james/quota/search/elasticsearch/events/ElasticSearchQuotaMailboxListener.java
index 7d27e53..c03a4f8 100644
--- a/mailbox/plugin/quota-search-elasticsearch/src/main/java/org/apache/james/quota/search/elasticsearch/events/ElasticSearchQuotaMailboxListener.java
+++ b/mailbox/plugin/quota-search-elasticsearch/src/main/java/org/apache/james/quota/search/elasticsearch/events/ElasticSearchQuotaMailboxListener.java
@@ -42,13 +42,15 @@ public class ElasticSearchQuotaMailboxListener implements MailboxListener.GroupM
private final ElasticSearchIndexer indexer;
private final QuotaRatioToElasticSearchJson quotaRatioToElasticSearchJson;
+ private final RoutingKey.Factory<User> routingKeyFactory;
@Inject
- public ElasticSearchQuotaMailboxListener(
- @Named(QuotaRatioElasticSearchConstants.InjectionNames.QUOTA_RATIO) ElasticSearchIndexer indexer,
- QuotaRatioToElasticSearchJson quotaRatioToElasticSearchJson) {
+ public ElasticSearchQuotaMailboxListener(@Named(QuotaRatioElasticSearchConstants.InjectionNames.QUOTA_RATIO) ElasticSearchIndexer indexer,
+ QuotaRatioToElasticSearchJson quotaRatioToElasticSearchJson,
+ RoutingKey.Factory<User> routingKeyFactory) {
this.indexer = indexer;
this.quotaRatioToElasticSearchJson = quotaRatioToElasticSearchJson;
+ this.routingKeyFactory = routingKeyFactory;
}
@Override
@@ -70,11 +72,7 @@ public class ElasticSearchQuotaMailboxListener implements MailboxListener.GroupM
User user = event.getUser();
indexer.index(toDocumentId(user),
quotaRatioToElasticSearchJson.convertToJson(event),
- toRoutingKey(user));
- }
-
- private RoutingKey toRoutingKey(User user) {
- return RoutingKey.fromString(user.asString());
+ routingKeyFactory.from(user));
}
private DocumentId toDocumentId(User user) {
diff --git a/mailbox/plugin/quota-search-elasticsearch/src/test/java/org/apache/james/quota/search/elasticsearch/ElasticSearchQuotaSearchTestSystemExtension.java b/mailbox/plugin/quota-search-elasticsearch/src/test/java/org/apache/james/quota/search/elasticsearch/ElasticSearchQuotaSearchTestSystemExtension.java
index bdeaf8d..9af1dc6 100644
--- a/mailbox/plugin/quota-search-elasticsearch/src/test/java/org/apache/james/quota/search/elasticsearch/ElasticSearchQuotaSearchTestSystemExtension.java
+++ b/mailbox/plugin/quota-search-elasticsearch/src/test/java/org/apache/james/quota/search/elasticsearch/ElasticSearchQuotaSearchTestSystemExtension.java
@@ -70,7 +70,8 @@ public class ElasticSearchQuotaSearchTestSystemExtension implements ParameterRes
ElasticSearchQuotaMailboxListener listener = new ElasticSearchQuotaMailboxListener(
new ElasticSearchIndexer(client,
QuotaRatioElasticSearchConstants.DEFAULT_QUOTA_RATIO_WRITE_ALIAS),
- new QuotaRatioToElasticSearchJson());
+ new QuotaRatioToElasticSearchJson(),
+ new UserRoutingKeyFactory());
resources.getMailboxManager().getEventBus().register(listener);
diff --git a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/RoutingKey.java b/mailbox/plugin/quota-search-elasticsearch/src/test/java/org/apache/james/quota/search/elasticsearch/UserRoutingKeyFactoryTest.java
similarity index 57%
copy from backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/RoutingKey.java
copy to mailbox/plugin/quota-search-elasticsearch/src/test/java/org/apache/james/quota/search/elasticsearch/UserRoutingKeyFactoryTest.java
index 05021e5..c1a2102 100644
--- a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/RoutingKey.java
+++ b/mailbox/plugin/quota-search-elasticsearch/src/test/java/org/apache/james/quota/search/elasticsearch/UserRoutingKeyFactoryTest.java
@@ -17,43 +17,18 @@
* under the License. *
****************************************************************/
-package org.apache.james.backends.es;
+package org.apache.james.quota.search.elasticsearch;
-import java.util.Objects;
+import static org.assertj.core.api.Assertions.assertThat;
-import org.elasticsearch.common.Strings;
+import org.apache.james.backends.es.RoutingKey;
+import org.apache.james.core.User;
+import org.junit.jupiter.api.Test;
-import com.google.common.base.Preconditions;
-
-public class RoutingKey {
- public static RoutingKey fromString(String value) {
- return new RoutingKey(value);
- }
-
-
- private final String value;
-
- private RoutingKey(String value) {
- Preconditions.checkArgument(!Strings.isNullOrEmpty(value), "RoutingKey must be specified");
- this.value = value;
- }
-
- public String asString() {
- return value;
- }
-
- @Override
- public final boolean equals(Object o) {
- if (o instanceof RoutingKey) {
- RoutingKey that = (RoutingKey) o;
-
- return Objects.equals(this.value, that.value);
- }
- return false;
- }
-
- @Override
- public final int hashCode() {
- return Objects.hash(value);
+class UserRoutingKeyFactoryTest {
+ @Test
+ void fromShouldRelyOnUsername() {
+ assertThat(new UserRoutingKeyFactory().from(User.fromUsername("bob")))
+ .isEqualTo(RoutingKey.fromString("bob"));
}
-}
+}
\ No newline at end of file
diff --git a/mailbox/plugin/quota-search-elasticsearch/src/test/java/org/apache/james/quota/search/elasticsearch/events/ElasticSearchQuotaMailboxListenerTest.java b/mailbox/plugin/quota-search-elasticsearch/src/test/java/org/apache/james/quota/search/elasticsearch/events/ElasticSearchQuotaMailboxListenerTest.java
index 69e55e8..9cdd817 100644
--- a/mailbox/plugin/quota-search-elasticsearch/src/test/java/org/apache/james/quota/search/elasticsearch/events/ElasticSearchQuotaMailboxListenerTest.java
+++ b/mailbox/plugin/quota-search-elasticsearch/src/test/java/org/apache/james/quota/search/elasticsearch/events/ElasticSearchQuotaMailboxListenerTest.java
@@ -37,6 +37,7 @@ import org.apache.james.mailbox.quota.QuotaFixture.Sizes;
import org.apache.james.mailbox.store.event.EventFactory;
import org.apache.james.quota.search.elasticsearch.QuotaRatioElasticSearchConstants;
import org.apache.james.quota.search.elasticsearch.QuotaSearchIndexCreationUtil;
+import org.apache.james.quota.search.elasticsearch.UserRoutingKeyFactory;
import org.apache.james.quota.search.elasticsearch.json.QuotaRatioToElasticSearchJson;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
@@ -70,7 +71,8 @@ public class ElasticSearchQuotaMailboxListenerTest {
new ElasticSearchIndexer(client,
QuotaRatioElasticSearchConstants.DEFAULT_QUOTA_RATIO_WRITE_ALIAS,
BATCH_SIZE),
- new QuotaRatioToElasticSearchJson());
+ new QuotaRatioToElasticSearchJson(),
+ new UserRoutingKeyFactory());
}
@After
diff --git a/mpt/impl/imap-mailbox/elasticsearch/src/test/java/org/apache/james/mpt/imapmailbox/elasticsearch/host/ElasticSearchHostSystem.java b/mpt/impl/imap-mailbox/elasticsearch/src/test/java/org/apache/james/mpt/imapmailbox/elasticsearch/host/ElasticSearchHostSystem.java
index 87151a2..b84bc80 100644
--- a/mpt/impl/imap-mailbox/elasticsearch/src/test/java/org/apache/james/mpt/imapmailbox/elasticsearch/host/ElasticSearchHostSystem.java
+++ b/mpt/impl/imap-mailbox/elasticsearch/src/test/java/org/apache/james/mpt/imapmailbox/elasticsearch/host/ElasticSearchHostSystem.java
@@ -36,6 +36,7 @@ import org.apache.james.imap.processor.main.DefaultImapProcessorFactory;
import org.apache.james.mailbox.MailboxManager;
import org.apache.james.mailbox.elasticsearch.IndexAttachments;
import org.apache.james.mailbox.elasticsearch.MailboxElasticSearchConstants;
+import org.apache.james.mailbox.elasticsearch.MailboxIdRoutingKeyFactory;
import org.apache.james.mailbox.elasticsearch.MailboxIndexCreationUtil;
import org.apache.james.mailbox.elasticsearch.events.ElasticSearchListeningMessageSearchIndex;
import org.apache.james.mailbox.elasticsearch.json.MessageToElasticSearchJson;
@@ -86,6 +87,7 @@ public class ElasticSearchHostSystem extends JamesImapHostSystem {
.build());
InMemoryMessageId.Factory messageIdFactory = new InMemoryMessageId.Factory();
+ MailboxIdRoutingKeyFactory routingKeyFactory = new MailboxIdRoutingKeyFactory();
InMemoryIntegrationResources resources = InMemoryIntegrationResources.builder()
.authenticator(authenticator)
@@ -99,9 +101,9 @@ public class ElasticSearchHostSystem extends JamesImapHostSystem {
MailboxElasticSearchConstants.DEFAULT_MAILBOX_WRITE_ALIAS),
new ElasticSearchSearcher(client, new QueryConverter(new CriterionConverter()), ElasticSearchSearcher.DEFAULT_SEARCH_SIZE,
new InMemoryId.Factory(), messageIdFactory,
- MailboxElasticSearchConstants.DEFAULT_MAILBOX_READ_ALIAS),
+ MailboxElasticSearchConstants.DEFAULT_MAILBOX_READ_ALIAS, routingKeyFactory),
new MessageToElasticSearchJson(new DefaultTextExtractor(), ZoneId.of("Europe/Paris"), IndexAttachments.YES),
- preInstanciationStage.getSessionProvider()))
+ preInstanciationStage.getSessionProvider(), routingKeyFactory))
.noPreDeletionHooks()
.storeQuotaManager()
.build();
diff --git a/server/container/guice/cassandra-guice/src/main/java/org/apache/james/modules/mailbox/ElasticSearchMailboxModule.java b/server/container/guice/cassandra-guice/src/main/java/org/apache/james/modules/mailbox/ElasticSearchMailboxModule.java
index 6bd4f54..bcc8148 100644
--- a/server/container/guice/cassandra-guice/src/main/java/org/apache/james/modules/mailbox/ElasticSearchMailboxModule.java
+++ b/server/container/guice/cassandra-guice/src/main/java/org/apache/james/modules/mailbox/ElasticSearchMailboxModule.java
@@ -32,11 +32,13 @@ import org.apache.commons.configuration2.Configuration;
import org.apache.commons.configuration2.ex.ConfigurationException;
import org.apache.james.backends.es.ElasticSearchConfiguration;
import org.apache.james.backends.es.ElasticSearchIndexer;
+import org.apache.james.backends.es.RoutingKey;
import org.apache.james.lifecycle.api.StartUpCheck;
import org.apache.james.lifecycle.api.Startable;
import org.apache.james.mailbox.elasticsearch.ElasticSearchMailboxConfiguration;
import org.apache.james.mailbox.elasticsearch.IndexAttachments;
import org.apache.james.mailbox.elasticsearch.MailboxElasticSearchConstants;
+import org.apache.james.mailbox.elasticsearch.MailboxIdRoutingKeyFactory;
import org.apache.james.mailbox.elasticsearch.MailboxIndexCreationUtil;
import org.apache.james.mailbox.elasticsearch.events.ElasticSearchListeningMessageSearchIndex;
import org.apache.james.mailbox.elasticsearch.query.QueryConverter;
@@ -56,6 +58,7 @@ import org.slf4j.LoggerFactory;
import com.google.inject.AbstractModule;
import com.google.inject.Provides;
import com.google.inject.Scopes;
+import com.google.inject.TypeLiteral;
import com.google.inject.multibindings.Multibinder;
import com.google.inject.multibindings.ProvidesIntoSet;
@@ -97,6 +100,8 @@ public class ElasticSearchMailboxModule extends AbstractModule {
bind(MessageSearchIndex.class).to(ElasticSearchListeningMessageSearchIndex.class);
bind(ListeningMessageSearchIndex.class).to(ElasticSearchListeningMessageSearchIndex.class);
+ bind(new TypeLiteral<RoutingKey.Factory<MailboxId>>() {}).to(MailboxIdRoutingKeyFactory.class);
+
Multibinder.newSetBinder(binder(), MailboxListener.GroupMailboxListener.class)
.addBinding()
.to(ElasticSearchListeningMessageSearchIndex.class);
@@ -122,14 +127,15 @@ public class ElasticSearchMailboxModule extends AbstractModule {
QueryConverter queryConverter,
MailboxId.Factory mailboxIdFactory,
MessageId.Factory messageIdFactory,
- ElasticSearchMailboxConfiguration configuration) {
+ ElasticSearchMailboxConfiguration configuration,
+ RoutingKey.Factory<MailboxId> routingKeyFactory) {
return new ElasticSearchSearcher(
client,
queryConverter,
DEFAULT_SEARCH_SIZE,
mailboxIdFactory,
messageIdFactory,
- configuration.getReadAliasMailboxName());
+ configuration.getReadAliasMailboxName(), routingKeyFactory);
}
@Provides
diff --git a/server/container/guice/cassandra-guice/src/main/java/org/apache/james/modules/mailbox/ElasticSearchQuotaSearcherModule.java b/server/container/guice/cassandra-guice/src/main/java/org/apache/james/modules/mailbox/ElasticSearchQuotaSearcherModule.java
index fc98f6e..eb4f9b4 100644
--- a/server/container/guice/cassandra-guice/src/main/java/org/apache/james/modules/mailbox/ElasticSearchQuotaSearcherModule.java
+++ b/server/container/guice/cassandra-guice/src/main/java/org/apache/james/modules/mailbox/ElasticSearchQuotaSearcherModule.java
@@ -36,6 +36,7 @@ import org.apache.james.quota.search.QuotaSearcher;
import org.apache.james.quota.search.elasticsearch.ElasticSearchQuotaConfiguration;
import org.apache.james.quota.search.elasticsearch.ElasticSearchQuotaSearcher;
import org.apache.james.quota.search.elasticsearch.QuotaSearchIndexCreationUtil;
+import org.apache.james.quota.search.elasticsearch.UserRoutingKeyFactory;
import org.apache.james.quota.search.elasticsearch.events.ElasticSearchQuotaMailboxListener;
import org.apache.james.quota.search.elasticsearch.json.QuotaRatioToElasticSearchJson;
import org.apache.james.utils.InitializationOperation;
@@ -111,7 +112,8 @@ public class ElasticSearchQuotaSearcherModule extends AbstractModule {
return new ElasticSearchQuotaMailboxListener(
new ElasticSearchIndexer(client,
configuration.getWriteAliasQuotaRatioName()),
- new QuotaRatioToElasticSearchJson());
+ new QuotaRatioToElasticSearchJson(),
+ new UserRoutingKeyFactory());
}
@ProvidesIntoSet
diff --git a/server/protocols/webadmin/webadmin-mailbox/src/test/java/org/apache/james/webadmin/routes/ElasticSearchQuotaSearchExtension.java b/server/protocols/webadmin/webadmin-mailbox/src/test/java/org/apache/james/webadmin/routes/ElasticSearchQuotaSearchExtension.java
index a469985..5cbe210 100644
--- a/server/protocols/webadmin/webadmin-mailbox/src/test/java/org/apache/james/webadmin/routes/ElasticSearchQuotaSearchExtension.java
+++ b/server/protocols/webadmin/webadmin-mailbox/src/test/java/org/apache/james/webadmin/routes/ElasticSearchQuotaSearchExtension.java
@@ -35,6 +35,7 @@ import org.apache.james.quota.search.QuotaSearchTestSystem;
import org.apache.james.quota.search.elasticsearch.ElasticSearchQuotaSearcher;
import org.apache.james.quota.search.elasticsearch.QuotaRatioElasticSearchConstants;
import org.apache.james.quota.search.elasticsearch.QuotaSearchIndexCreationUtil;
+import org.apache.james.quota.search.elasticsearch.UserRoutingKeyFactory;
import org.apache.james.quota.search.elasticsearch.events.ElasticSearchQuotaMailboxListener;
import org.apache.james.quota.search.elasticsearch.json.QuotaRatioToElasticSearchJson;
import org.apache.james.user.memory.MemoryUsersRepository;
@@ -76,7 +77,8 @@ public class ElasticSearchQuotaSearchExtension implements ParameterResolver, Bef
ElasticSearchQuotaMailboxListener listener = new ElasticSearchQuotaMailboxListener(
new ElasticSearchIndexer(client, QuotaRatioElasticSearchConstants.DEFAULT_QUOTA_RATIO_WRITE_ALIAS),
- new QuotaRatioToElasticSearchJson());
+ new QuotaRatioToElasticSearchJson(),
+ new UserRoutingKeyFactory());
resources.getMailboxManager().getEventBus().register(listener);
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[james-project] 05/06: JAMES-2917 Document breaking changes in
changelog + upgrade instructions
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 41723651088d0fd68b67c2142919760fc7b40824
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Thu Oct 10 14:14:58 2019 +0700
JAMES-2917 Document breaking changes in changelog + upgrade instructions
---
CHANGELOG.md | 5 +++++
upgrade-instructions.md | 23 +++++++++++++++++++++++
2 files changed, 28 insertions(+)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 5558963..d2894c4 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,11 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
## [Unreleased]
+### Changed
+- Multiple changes have been made to enhance ElasticSearch performance:
+ - Use of routing keys to collocate documents per mailbox
+ - Read related [upgrade instructions](upgrade-instructions.md)
+
### Removed
- Classes marked as deprecated whose removal was planned after 3.4.0 release (See JAMES-2703). This includes:
- SieveDefaultRepository. Please use SieveFileRepository instead.
diff --git a/upgrade-instructions.md b/upgrade-instructions.md
index 3ee23cb..18b8d7f 100644
--- a/upgrade-instructions.md
+++ b/upgrade-instructions.md
@@ -16,8 +16,31 @@ Changes to apply between 3.4.x and 3.5.x will be reported here.
Change list:
+ - [ElasticSearch performance enhancements](#elasticsearch-performance-enhancements)
- [JAMES-2703 Post 3.4.0 release removals](#james-2703-post-340-release-removals)
+### ElasticSearch performance enhancements
+
+Date 10/10/2019
+
+SHA-1 0d72783ff4
+
+JIRAS:
+ - https://issues.apache.org/jira/browse/JAMES-2917
+
+Concerned product: Guice product relying on ElasticSearch
+
+We significantly improved our usage of ElasticSearch. Underlying changes include:
+
+ - The use of routing to collocate emails of a same mailbox within a same shard. This enables search queries to avoid cluster
+ level synchronisation, and thus enhance throughput, latencies and scalability.
+
+The downside of these changes is that a reindex is needed, implying a downtime on search:
+ - Delete the indexes used by James
+ - Start James in order to create the missing indexes
+ - Trigger a [Full ReIndexing](https://james.apache.org/server/manage-webadmin.html#ReIndexing_all_mails), which can take
+ time to complete.
+
#### JAMES-2703 Post 3.4.0 release removals
Date: 25/09/2019
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[james-project] 02/06: JAMES-2917 Rely on ElasticSearch routing key
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 5b6427a687f265f8f1e511462e2ce3f6b66d0136
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Thu Oct 10 10:44:44 2019 +0700
JAMES-2917 Rely on ElasticSearch routing key
Our queries are mostly bounded to a mailbox or an user. We can easily
limit the number of ElasticSearch nodes involved in a given query by
grouping the underlying documents on the same node using a routingKey.
Without routing key, each shard needs to execute the query. The coordinator
needs also to be waiting for the slowest shard.
This changeset unlocks significant throughput enhancement (proportional
to the number of shard) and also a possible high percentile latencies enhancement.
However a data reindex is needed.
---
.../james/backends/es/DeleteByQueryPerformer.java | 20 +++--
...{UpdatedRepresentation.java => DocumentId.java} | 44 ++++-------
.../james/backends/es/ElasticSearchIndexer.java | 39 ++++++----
...{UpdatedRepresentation.java => RoutingKey.java} | 46 +++++------
.../james/backends/es/UpdatedRepresentation.java | 8 +-
.../apache/james/backends/es/DocumentIdTest.java | 46 +++++++++++
.../backends/es/ElasticSearchIndexerTest.java | 90 ++++++++++++----------
.../apache/james/backends/es/RoutingKeyTest.java | 46 +++++++++++
.../ElasticSearchListeningMessageSearchIndex.java | 39 ++++++----
.../search/ElasticSearchSearcher.java | 11 ++-
.../events/ElasticSearchQuotaMailboxListener.java | 17 +++-
11 files changed, 264 insertions(+), 142 deletions(-)
diff --git a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/DeleteByQueryPerformer.java b/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/DeleteByQueryPerformer.java
index 26376c6..f9ba528 100644
--- a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/DeleteByQueryPerformer.java
+++ b/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/DeleteByQueryPerformer.java
@@ -44,23 +44,26 @@ public class DeleteByQueryPerformer {
private final WriteAliasName aliasName;
@VisibleForTesting
- public DeleteByQueryPerformer(RestHighLevelClient client, int batchSize, WriteAliasName aliasName) {
+ DeleteByQueryPerformer(RestHighLevelClient client, int batchSize, WriteAliasName aliasName) {
this.client = client;
this.batchSize = batchSize;
this.aliasName = aliasName;
}
- public Mono<Void> perform(QueryBuilder queryBuilder) {
- return Flux.fromStream(new ScrolledSearch(client, prepareSearch(queryBuilder)).searchResponses())
- .flatMap(searchResponse -> deleteRetrievedIds(client, searchResponse))
+ public Mono<Void> perform(QueryBuilder queryBuilder, RoutingKey routingKey) {
+ SearchRequest searchRequest = prepareSearch(queryBuilder, routingKey);
+
+ return Flux.fromStream(new ScrolledSearch(client, searchRequest).searchResponses())
+ .flatMap(searchResponse -> deleteRetrievedIds(client, searchResponse, routingKey))
.thenEmpty(Mono.empty());
}
- private SearchRequest prepareSearch(QueryBuilder queryBuilder) {
+ private SearchRequest prepareSearch(QueryBuilder queryBuilder, RoutingKey routingKey) {
return new SearchRequest(aliasName.getValue())
.types(NodeMappingFactory.DEFAULT_MAPPING_NAME)
.scroll(TIMEOUT)
- .source(searchSourceBuilder(queryBuilder));
+ .source(searchSourceBuilder(queryBuilder))
+ .routing(routingKey.asString());
}
private SearchSourceBuilder searchSourceBuilder(QueryBuilder queryBuilder) {
@@ -69,14 +72,15 @@ public class DeleteByQueryPerformer {
.size(batchSize);
}
- private Mono<BulkResponse> deleteRetrievedIds(RestHighLevelClient client, SearchResponse searchResponse) {
+ private Mono<BulkResponse> deleteRetrievedIds(RestHighLevelClient client, SearchResponse searchResponse, RoutingKey routingKey) {
BulkRequest request = new BulkRequest();
for (SearchHit hit : searchResponse.getHits()) {
request.add(
new DeleteRequest(aliasName.getValue())
.type(NodeMappingFactory.DEFAULT_MAPPING_NAME)
- .id(hit.getId()));
+ .id(hit.getId())
+ .routing(routingKey.asString()));
}
return Mono.fromCallable(() -> client.bulk(request));
diff --git a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/UpdatedRepresentation.java b/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/DocumentId.java
similarity index 56%
copy from backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/UpdatedRepresentation.java
copy to backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/DocumentId.java
index 92ce8c8..7633208 100644
--- a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/UpdatedRepresentation.java
+++ b/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/DocumentId.java
@@ -16,54 +16,44 @@
* specific language governing permissions and limitations *
* under the License. *
****************************************************************/
+
package org.apache.james.backends.es;
import java.util.Objects;
import org.elasticsearch.common.Strings;
-import com.google.common.base.MoreObjects;
import com.google.common.base.Preconditions;
-public class UpdatedRepresentation {
- private final String id;
- private final String updatedDocumentPart;
+public class DocumentId {
- public UpdatedRepresentation(String id, String updatedDocumentPart) {
- Preconditions.checkArgument(!Strings.isNullOrEmpty(id), "Updated id must be specified " + id);
- Preconditions.checkArgument(!Strings.isNullOrEmpty(updatedDocumentPart), "Updated document must be specified");
- this.id = id;
- this.updatedDocumentPart = updatedDocumentPart;
+ public static DocumentId fromString(String value) {
+ return new DocumentId(value);
}
- public String getId() {
- return id;
+ private final String value;
+
+ private DocumentId(String value) {
+ Preconditions.checkArgument(!Strings.isNullOrEmpty(value), "DocumentId must be specified");
+ this.value = value;
}
- public String getUpdatedDocumentPart() {
- return updatedDocumentPart;
+ public String asString() {
+ return value;
}
@Override
public final boolean equals(Object o) {
- if (o instanceof UpdatedRepresentation) {
- UpdatedRepresentation other = (UpdatedRepresentation) o;
- return Objects.equals(id, other.id)
- && Objects.equals(updatedDocumentPart, other.updatedDocumentPart);
+ if (o instanceof DocumentId) {
+ DocumentId that = (DocumentId) o;
+
+ return Objects.equals(this.value, that.value);
}
return false;
}
@Override
public final int hashCode() {
- return Objects.hash(id, updatedDocumentPart);
- }
-
- @Override
- public String toString() {
- return MoreObjects.toStringHelper(this)
- .add("id", id)
- .add("updatedDocumentPart", updatedDocumentPart)
- .toString();
+ return Objects.hash(value);
}
-}
\ No newline at end of file
+}
diff --git a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/ElasticSearchIndexer.java b/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/ElasticSearchIndexer.java
index bc586a4..99a9f80 100644
--- a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/ElasticSearchIndexer.java
+++ b/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/ElasticSearchIndexer.java
@@ -64,28 +64,34 @@ public class ElasticSearchIndexer {
this.aliasName = aliasName;
}
- public IndexResponse index(String id, String content) throws IOException {
+ public IndexResponse index(DocumentId id, String content, RoutingKey routingKey) throws IOException {
checkArgument(content);
- if (LOGGER.isDebugEnabled()) {
- LOGGER.debug("Indexing {}: {}", id, StringUtils.left(content, DEBUG_MAX_LENGTH_CONTENT));
- }
- return client.index(
- new IndexRequest(aliasName.getValue())
+ logContent(id, content);
+ return client.index(new IndexRequest(aliasName.getValue())
.type(NodeMappingFactory.DEFAULT_MAPPING_NAME)
- .id(id)
- .source(content, XContentType.JSON),
+ .id(id.asString())
+ .source(content, XContentType.JSON)
+ .routing(routingKey.asString()),
RequestOptions.DEFAULT);
}
- public Optional<BulkResponse> update(List<UpdatedRepresentation> updatedDocumentParts) throws IOException {
+ private void logContent(DocumentId id, String content) {
+ if (LOGGER.isDebugEnabled()) {
+ LOGGER.debug("Indexing {}: {}", id.asString(), StringUtils.left(content, DEBUG_MAX_LENGTH_CONTENT));
+ }
+ }
+
+ public Optional<BulkResponse> update(List<UpdatedRepresentation> updatedDocumentParts, RoutingKey routingKey) throws IOException {
try {
Preconditions.checkNotNull(updatedDocumentParts);
+ Preconditions.checkNotNull(routingKey);
BulkRequest request = new BulkRequest();
updatedDocumentParts.forEach(updatedDocumentPart -> request.add(
new UpdateRequest(aliasName.getValue(),
NodeMappingFactory.DEFAULT_MAPPING_NAME,
- updatedDocumentPart.getId())
- .doc(updatedDocumentPart.getUpdatedDocumentPart(), XContentType.JSON)));
+ updatedDocumentPart.getId().asString())
+ .doc(updatedDocumentPart.getUpdatedDocumentPart(), XContentType.JSON)
+ .routing(routingKey.asString())));
return Optional.of(client.bulk(request, RequestOptions.DEFAULT));
} catch (ValidationException e) {
LOGGER.warn("Error while updating index", e);
@@ -93,22 +99,23 @@ public class ElasticSearchIndexer {
}
}
- public Optional<BulkResponse> delete(List<String> ids) throws IOException {
+ public Optional<BulkResponse> delete(List<DocumentId> ids, RoutingKey routingKey) throws IOException {
try {
BulkRequest request = new BulkRequest();
ids.forEach(id -> request.add(
new DeleteRequest(aliasName.getValue())
.type(NodeMappingFactory.DEFAULT_MAPPING_NAME)
- .id(id)));
- return Optional.of(client.bulk(request));
+ .id(id.asString())
+ .routing(routingKey.asString())));
+ return Optional.of(client.bulk(request, RequestOptions.DEFAULT));
} catch (ValidationException e) {
LOGGER.warn("Error while deleting index", e);
return Optional.empty();
}
}
- public void deleteAllMatchingQuery(QueryBuilder queryBuilder) {
- deleteByQueryPerformer.perform(queryBuilder).block();
+ public void deleteAllMatchingQuery(QueryBuilder queryBuilder, RoutingKey routingKey) {
+ deleteByQueryPerformer.perform(queryBuilder, routingKey).block();
}
private void checkArgument(String content) {
diff --git a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/UpdatedRepresentation.java b/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/RoutingKey.java
similarity index 56%
copy from backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/UpdatedRepresentation.java
copy to backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/RoutingKey.java
index 92ce8c8..05021e5 100644
--- a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/UpdatedRepresentation.java
+++ b/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/RoutingKey.java
@@ -16,54 +16,44 @@
* specific language governing permissions and limitations *
* under the License. *
****************************************************************/
+
package org.apache.james.backends.es;
import java.util.Objects;
import org.elasticsearch.common.Strings;
-import com.google.common.base.MoreObjects;
import com.google.common.base.Preconditions;
-public class UpdatedRepresentation {
- private final String id;
- private final String updatedDocumentPart;
-
- public UpdatedRepresentation(String id, String updatedDocumentPart) {
- Preconditions.checkArgument(!Strings.isNullOrEmpty(id), "Updated id must be specified " + id);
- Preconditions.checkArgument(!Strings.isNullOrEmpty(updatedDocumentPart), "Updated document must be specified");
- this.id = id;
- this.updatedDocumentPart = updatedDocumentPart;
+public class RoutingKey {
+ public static RoutingKey fromString(String value) {
+ return new RoutingKey(value);
}
- public String getId() {
- return id;
+
+ private final String value;
+
+ private RoutingKey(String value) {
+ Preconditions.checkArgument(!Strings.isNullOrEmpty(value), "RoutingKey must be specified");
+ this.value = value;
}
- public String getUpdatedDocumentPart() {
- return updatedDocumentPart;
+ public String asString() {
+ return value;
}
@Override
public final boolean equals(Object o) {
- if (o instanceof UpdatedRepresentation) {
- UpdatedRepresentation other = (UpdatedRepresentation) o;
- return Objects.equals(id, other.id)
- && Objects.equals(updatedDocumentPart, other.updatedDocumentPart);
+ if (o instanceof RoutingKey) {
+ RoutingKey that = (RoutingKey) o;
+
+ return Objects.equals(this.value, that.value);
}
return false;
}
@Override
public final int hashCode() {
- return Objects.hash(id, updatedDocumentPart);
- }
-
- @Override
- public String toString() {
- return MoreObjects.toStringHelper(this)
- .add("id", id)
- .add("updatedDocumentPart", updatedDocumentPart)
- .toString();
+ return Objects.hash(value);
}
-}
\ No newline at end of file
+}
diff --git a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/UpdatedRepresentation.java b/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/UpdatedRepresentation.java
index 92ce8c8..a14605c 100644
--- a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/UpdatedRepresentation.java
+++ b/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/UpdatedRepresentation.java
@@ -26,17 +26,17 @@ import com.google.common.base.MoreObjects;
import com.google.common.base.Preconditions;
public class UpdatedRepresentation {
- private final String id;
+ private final DocumentId id;
private final String updatedDocumentPart;
- public UpdatedRepresentation(String id, String updatedDocumentPart) {
- Preconditions.checkArgument(!Strings.isNullOrEmpty(id), "Updated id must be specified " + id);
+ public UpdatedRepresentation(DocumentId id, String updatedDocumentPart) {
+ Preconditions.checkNotNull(id);
Preconditions.checkArgument(!Strings.isNullOrEmpty(updatedDocumentPart), "Updated document must be specified");
this.id = id;
this.updatedDocumentPart = updatedDocumentPart;
}
- public String getId() {
+ public DocumentId getId() {
return id;
}
diff --git a/backends-common/elasticsearch/src/test/java/org/apache/james/backends/es/DocumentIdTest.java b/backends-common/elasticsearch/src/test/java/org/apache/james/backends/es/DocumentIdTest.java
new file mode 100644
index 0000000..bc9306b
--- /dev/null
+++ b/backends-common/elasticsearch/src/test/java/org/apache/james/backends/es/DocumentIdTest.java
@@ -0,0 +1,46 @@
+/****************************************************************
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information *
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the *
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the *
+ * specific language governing permissions and limitations *
+ * under the License. *
+ ****************************************************************/
+
+package org.apache.james.backends.es;
+
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+import org.junit.jupiter.api.Test;
+
+import nl.jqno.equalsverifier.EqualsVerifier;
+
+class DocumentIdTest {
+ @Test
+ void documentIdShouldRespectBeanContract() {
+ EqualsVerifier.forClass(DocumentId.class)
+ .verify();
+ }
+
+ @Test
+ void fromStringShouldThrowWhenNull() {
+ assertThatThrownBy(() -> DocumentId.fromString(null))
+ .isInstanceOf(IllegalArgumentException.class);
+ }
+
+ @Test
+ void fromStringShouldThrowWhenEmpty() {
+ assertThatThrownBy(() -> DocumentId.fromString(""))
+ .isInstanceOf(IllegalArgumentException.class);
+ }
+}
\ No newline at end of file
diff --git a/backends-common/elasticsearch/src/test/java/org/apache/james/backends/es/ElasticSearchIndexerTest.java b/backends-common/elasticsearch/src/test/java/org/apache/james/backends/es/ElasticSearchIndexerTest.java
index ba4e4f7..e8fefbc 100644
--- a/backends-common/elasticsearch/src/test/java/org/apache/james/backends/es/ElasticSearchIndexerTest.java
+++ b/backends-common/elasticsearch/src/test/java/org/apache/james/backends/es/ElasticSearchIndexerTest.java
@@ -20,6 +20,7 @@
package org.apache.james.backends.es;
import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatCode;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.awaitility.Duration.ONE_HUNDRED_MILLISECONDS;
import static org.elasticsearch.index.query.QueryBuilders.termQuery;
@@ -43,6 +44,9 @@ import org.junit.Test;
import com.google.common.collect.ImmutableList;
public class ElasticSearchIndexerTest {
+ public static RoutingKey useDocumentId(DocumentId documentId) {
+ return RoutingKey.fromString(documentId.asString());
+ }
private static final int MINIMUM_BATCH_SIZE = 1;
private static final IndexName INDEX_NAME = new IndexName("index_name");
@@ -52,6 +56,8 @@ public class ElasticSearchIndexerTest {
.with().pollInterval(ONE_HUNDRED_MILLISECONDS)
.and().pollDelay(ONE_HUNDRED_MILLISECONDS)
.await();
+ private static final RoutingKey ROUTING = RoutingKey.fromString("routing");
+ private static final DocumentId DOCUMENT_ID = DocumentId.fromString("1");
@Rule
public DockerElasticSearchRule elasticSearch = new DockerElasticSearchRule();
@@ -75,10 +81,10 @@ public class ElasticSearchIndexerTest {
@Test
public void indexMessageShouldWork() throws Exception {
- String messageId = "1";
+ DocumentId documentId = DocumentId.fromString("1");
String content = "{\"message\": \"trying out Elasticsearch\"}";
- testee.index(messageId, content);
+ testee.index(documentId, content, useDocumentId(documentId));
elasticSearch.awaitForElasticSearch();
SearchResponse searchResponse = client.search(
@@ -90,19 +96,18 @@ public class ElasticSearchIndexerTest {
@Test
public void indexMessageShouldThrowWhenJsonIsNull() {
- assertThatThrownBy(() -> testee.index("1", null))
+ assertThatThrownBy(() -> testee.index(DOCUMENT_ID, null, ROUTING))
.isInstanceOf(IllegalArgumentException.class);
}
@Test
public void updateMessages() throws Exception {
- String messageId = "1";
String content = "{\"message\": \"trying out Elasticsearch\",\"field\":\"Should be unchanged\"}";
- testee.index(messageId, content);
+ testee.index(DOCUMENT_ID, content, useDocumentId(DOCUMENT_ID));
elasticSearch.awaitForElasticSearch();
- testee.update(ImmutableList.of(new UpdatedRepresentation(messageId, "{\"message\": \"mastering out Elasticsearch\"}")));
+ testee.update(ImmutableList.of(new UpdatedRepresentation(DOCUMENT_ID, "{\"message\": \"mastering out Elasticsearch\"}")), useDocumentId(DOCUMENT_ID));
elasticSearch.awaitForElasticSearch();
@@ -121,37 +126,42 @@ public class ElasticSearchIndexerTest {
@Test
public void updateMessageShouldThrowWhenJsonIsNull() {
- assertThatThrownBy(() -> testee.update(ImmutableList.of(new UpdatedRepresentation("1", null))))
+ assertThatThrownBy(() -> testee.update(ImmutableList.of(
+ new UpdatedRepresentation(DOCUMENT_ID, null)), ROUTING))
.isInstanceOf(IllegalArgumentException.class);
}
@Test
public void updateMessageShouldThrowWhenIdIsNull() {
- assertThatThrownBy(() -> testee.update(ImmutableList.of(new UpdatedRepresentation(null, "{\"message\": \"mastering out Elasticsearch\"}"))))
- .isInstanceOf(IllegalArgumentException.class);
+ assertThatThrownBy(() -> testee.update(ImmutableList.of(
+ new UpdatedRepresentation(null, "{\"message\": \"mastering out Elasticsearch\"}")), ROUTING))
+ .isInstanceOf(NullPointerException.class);
}
@Test
public void updateMessageShouldThrowWhenJsonIsEmpty() {
- assertThatThrownBy(() -> testee.update(ImmutableList.of(new UpdatedRepresentation("1", ""))))
+ assertThatThrownBy(() -> testee.update(ImmutableList.of(
+ new UpdatedRepresentation(DOCUMENT_ID, "")), ROUTING))
.isInstanceOf(IllegalArgumentException.class);
}
@Test
- public void updateMessageShouldThrowWhenIdIsEmpty() {
- assertThatThrownBy(() -> testee.update(ImmutableList.of(new UpdatedRepresentation("", "{\"message\": \"mastering out Elasticsearch\"}"))))
- .isInstanceOf(IllegalArgumentException.class);
+ public void updateMessageShouldThrowWhenRoutingKeyIsNull() {
+ assertThatThrownBy(() -> testee.update(ImmutableList.of(
+ new UpdatedRepresentation(DOCUMENT_ID, "{\"message\": \"mastering out Elasticsearch\"}")), null))
+ .isInstanceOf(NullPointerException.class);
}
@Test
public void deleteByQueryShouldWorkOnSingleMessage() throws Exception {
- String messageId = "1:2";
+ DocumentId documentId = DocumentId.fromString("1:2");
String content = "{\"message\": \"trying out Elasticsearch\", \"property\":\"1\"}";
+ RoutingKey routingKey = useDocumentId(documentId);
- testee.index(messageId, content);
+ testee.index(documentId, content, routingKey);
elasticSearch.awaitForElasticSearch();
- testee.deleteAllMatchingQuery(termQuery("property", "1"));
+ testee.deleteAllMatchingQuery(termQuery("property", "1"), routingKey);
elasticSearch.awaitForElasticSearch();
CALMLY_AWAIT.atMost(Duration.TEN_SECONDS)
@@ -164,23 +174,23 @@ public class ElasticSearchIndexerTest {
@Test
public void deleteByQueryShouldWorkWhenMultipleMessages() throws Exception {
- String messageId = "1:1";
+ DocumentId documentId = DocumentId.fromString("1:1");
String content = "{\"message\": \"trying out Elasticsearch\", \"property\":\"1\"}";
- testee.index(messageId, content);
-
- String messageId2 = "1:2";
+ testee.index(documentId, content, ROUTING);
+
+ DocumentId documentId2 = DocumentId.fromString("1:2");
String content2 = "{\"message\": \"trying out Elasticsearch 2\", \"property\":\"1\"}";
- testee.index(messageId2, content2);
-
- String messageId3 = "2:3";
+ testee.index(documentId2, content2, ROUTING);
+
+ DocumentId documentId3 = DocumentId.fromString("2:3");
String content3 = "{\"message\": \"trying out Elasticsearch 3\", \"property\":\"2\"}";
- testee.index(messageId3, content3);
+ testee.index(documentId3, content3, ROUTING);
elasticSearch.awaitForElasticSearch();
- testee.deleteAllMatchingQuery(termQuery("property", "1"));
+ testee.deleteAllMatchingQuery(termQuery("property", "1"), ROUTING);
elasticSearch.awaitForElasticSearch();
CALMLY_AWAIT.atMost(Duration.TEN_SECONDS)
@@ -193,13 +203,13 @@ public class ElasticSearchIndexerTest {
@Test
public void deleteMessage() throws Exception {
- String messageId = "1:2";
+ DocumentId documentId = DocumentId.fromString("1:2");
String content = "{\"message\": \"trying out Elasticsearch\"}";
- testee.index(messageId, content);
+ testee.index(documentId, content, useDocumentId(documentId));
elasticSearch.awaitForElasticSearch();
- testee.delete(ImmutableList.of(messageId));
+ testee.delete(ImmutableList.of(documentId), useDocumentId(documentId));
elasticSearch.awaitForElasticSearch();
SearchResponse searchResponse = client.search(
@@ -211,23 +221,23 @@ public class ElasticSearchIndexerTest {
@Test
public void deleteShouldWorkWhenMultipleMessages() throws Exception {
- String messageId = "1:1";
+ DocumentId documentId = DocumentId.fromString("1:1");
String content = "{\"message\": \"trying out Elasticsearch\", \"mailboxId\":\"1\"}";
- testee.index(messageId, content);
+ testee.index(documentId, content, ROUTING);
- String messageId2 = "1:2";
+ DocumentId documentId2 = DocumentId.fromString("1:2");
String content2 = "{\"message\": \"trying out Elasticsearch 2\", \"mailboxId\":\"1\"}";
- testee.index(messageId2, content2);
+ testee.index(documentId2, content2, ROUTING);
- String messageId3 = "2:3";
+ DocumentId documentId3 = DocumentId.fromString("2:3");
String content3 = "{\"message\": \"trying out Elasticsearch 3\", \"mailboxId\":\"2\"}";
- testee.index(messageId3, content3);
+ testee.index(documentId3, content3, ROUTING);
elasticSearch.awaitForElasticSearch();
- testee.delete(ImmutableList.of(messageId, messageId3));
+ testee.delete(ImmutableList.of(documentId, documentId3), ROUTING);
elasticSearch.awaitForElasticSearch();
SearchResponse searchResponse = client.search(
@@ -238,12 +248,14 @@ public class ElasticSearchIndexerTest {
}
@Test
- public void updateMessagesShouldNotThrowWhenEmptyList() throws Exception {
- testee.update(ImmutableList.of());
+ public void updateMessagesShouldNotThrowWhenEmptyList() {
+ assertThatCode(() -> testee.update(ImmutableList.of(), ROUTING))
+ .doesNotThrowAnyException();
}
@Test
- public void deleteMessagesShouldNotThrowWhenEmptyList() throws Exception {
- testee.delete(ImmutableList.of());
+ public void deleteMessagesShouldNotThrowWhenEmptyList() {
+ assertThatCode(() -> testee.delete(ImmutableList.of(), ROUTING))
+ .doesNotThrowAnyException();
}
}
diff --git a/backends-common/elasticsearch/src/test/java/org/apache/james/backends/es/RoutingKeyTest.java b/backends-common/elasticsearch/src/test/java/org/apache/james/backends/es/RoutingKeyTest.java
new file mode 100644
index 0000000..cfa4d4e
--- /dev/null
+++ b/backends-common/elasticsearch/src/test/java/org/apache/james/backends/es/RoutingKeyTest.java
@@ -0,0 +1,46 @@
+/****************************************************************
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information *
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the *
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the *
+ * specific language governing permissions and limitations *
+ * under the License. *
+ ****************************************************************/
+
+package org.apache.james.backends.es;
+
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+import org.junit.jupiter.api.Test;
+
+import nl.jqno.equalsverifier.EqualsVerifier;
+
+class RoutingKeyTest {
+ @Test
+ void routingKeyShouldRespectBeanContract() {
+ EqualsVerifier.forClass(RoutingKey.class)
+ .verify();
+ }
+
+ @Test
+ void fromStringShouldThrowWhenNull() {
+ assertThatThrownBy(() -> RoutingKey.fromString(null))
+ .isInstanceOf(IllegalArgumentException.class);
+ }
+
+ @Test
+ void fromStringShouldThrowWhenEmpty() {
+ assertThatThrownBy(() -> RoutingKey.fromString(""))
+ .isInstanceOf(IllegalArgumentException.class);
+ }
+}
\ No newline at end of file
diff --git a/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/events/ElasticSearchListeningMessageSearchIndex.java b/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/events/ElasticSearchListeningMessageSearchIndex.java
index 657f670..c695b85 100644
--- a/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/events/ElasticSearchListeningMessageSearchIndex.java
+++ b/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/events/ElasticSearchListeningMessageSearchIndex.java
@@ -30,7 +30,9 @@ import java.util.stream.Stream;
import javax.inject.Inject;
import javax.inject.Named;
+import org.apache.james.backends.es.DocumentId;
import org.apache.james.backends.es.ElasticSearchIndexer;
+import org.apache.james.backends.es.RoutingKey;
import org.apache.james.backends.es.UpdatedRepresentation;
import org.apache.james.mailbox.MailboxManager.MessageCapabilities;
import org.apache.james.mailbox.MailboxManager.SearchCapabilities;
@@ -51,6 +53,7 @@ import org.apache.james.mailbox.store.SessionProvider;
import org.apache.james.mailbox.store.mail.model.MailboxMessage;
import org.apache.james.mailbox.store.search.ListeningMessageSearchIndex;
import org.apache.james.util.OptionalUtils;
+import org.elasticsearch.index.query.TermQueryBuilder;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -139,7 +142,7 @@ public class ElasticSearchListeningMessageSearchIndex extends ListeningMessageSe
String jsonContent = generateIndexedJson(mailbox, message, session);
- elasticSearchIndexer.index(indexIdFor(mailbox, message.getUid()), jsonContent);
+ elasticSearchIndexer.index(indexIdFor(mailbox, message.getUid()), jsonContent, toRoutingKey(mailbox.getMailboxId()));
}
private String generateIndexedJson(Mailbox mailbox, MailboxMessage message, MailboxSession session) throws JsonProcessingException {
@@ -161,26 +164,29 @@ public class ElasticSearchListeningMessageSearchIndex extends ListeningMessageSe
elasticSearchIndexer
.delete(expungedUids.stream()
.map(uid -> indexIdFor(mailbox, uid))
- .collect(Guavate.toImmutableList()));
+ .collect(Guavate.toImmutableList()),
+ toRoutingKey(mailbox.getMailboxId()));
}
@Override
public void deleteAll(MailboxSession session, MailboxId mailboxId) {
- elasticSearchIndexer
- .deleteAllMatchingQuery(
- termQuery(
- JsonMessageConstants.MAILBOX_ID,
- mailboxId.serialize()));
+ TermQueryBuilder queryBuilder = termQuery(
+ JsonMessageConstants.MAILBOX_ID,
+ mailboxId.serialize());
+
+ elasticSearchIndexer
+ .deleteAllMatchingQuery(queryBuilder, toRoutingKey(mailboxId));
}
@Override
public void update(MailboxSession session, Mailbox mailbox, List<UpdatedFlags> updatedFlagsList) throws IOException {
- elasticSearchIndexer
- .update(updatedFlagsList.stream()
- .map(Throwing.<UpdatedFlags, UpdatedRepresentation>function(
- updatedFlags -> createUpdatedDocumentPartFromUpdatedFlags(mailbox, updatedFlags))
- .sneakyThrow())
- .collect(Guavate.toImmutableList()));
+ ImmutableList<UpdatedRepresentation> updates = updatedFlagsList.stream()
+ .map(Throwing.<UpdatedFlags, UpdatedRepresentation>function(
+ updatedFlags -> createUpdatedDocumentPartFromUpdatedFlags(mailbox, updatedFlags))
+ .sneakyThrow())
+ .collect(Guavate.toImmutableList());
+
+ elasticSearchIndexer.update(updates, toRoutingKey(mailbox.getMailboxId()));
}
private UpdatedRepresentation createUpdatedDocumentPartFromUpdatedFlags(Mailbox mailbox, UpdatedFlags updatedFlags) throws JsonProcessingException {
@@ -190,8 +196,8 @@ public class ElasticSearchListeningMessageSearchIndex extends ListeningMessageSe
.getUpdatedJsonMessagePart(updatedFlags.getNewFlags(), updatedFlags.getModSeq()));
}
- private String indexIdFor(Mailbox mailbox, MessageUid uid) {
- return String.join(ID_SEPARATOR, mailbox.getMailboxId().serialize(), String.valueOf(uid.asLong()));
+ private DocumentId indexIdFor(Mailbox mailbox, MessageUid uid) {
+ return DocumentId.fromString(String.join(ID_SEPARATOR, mailbox.getMailboxId().serialize(), String.valueOf(uid.asLong())));
}
private void logIfNoMessageId(SearchResult searchResult) {
@@ -200,4 +206,7 @@ public class ElasticSearchListeningMessageSearchIndex extends ListeningMessageSe
}
}
+ private RoutingKey toRoutingKey(MailboxId mailboxId) {
+ return RoutingKey.fromString(mailboxId.serialize());
+ }
}
diff --git a/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/search/ElasticSearchSearcher.java b/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/search/ElasticSearchSearcher.java
index 9406b68..48b67e4 100644
--- a/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/search/ElasticSearchSearcher.java
+++ b/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/search/ElasticSearchSearcher.java
@@ -82,9 +82,9 @@ public class ElasticSearchSearcher {
.orElse(pairStream);
}
- private SearchRequest prepareSearch(Collection<MailboxId> users, SearchQuery query, Optional<Integer> limit) {
+ private SearchRequest prepareSearch(Collection<MailboxId> mailboxIds, SearchQuery query, Optional<Integer> limit) {
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder()
- .query(queryConverter.from(users, query))
+ .query(queryConverter.from(mailboxIds, query))
.size(computeRequiredSize(limit))
.storedFields(STORED_FIELDS);
@@ -96,7 +96,12 @@ public class ElasticSearchSearcher {
return new SearchRequest(aliasName.getValue())
.types(NodeMappingFactory.DEFAULT_MAPPING_NAME)
.scroll(TIMEOUT)
- .source(searchSourceBuilder);
+ .source(searchSourceBuilder)
+ .routing(toRoutingKeys(mailboxIds));
+ }
+
+ private String[] toRoutingKeys(Collection<MailboxId> mailboxIds) {
+ return mailboxIds.stream().map(MailboxId::serialize).toArray(String[]::new);
}
private int computeRequiredSize(Optional<Integer> limit) {
diff --git a/mailbox/plugin/quota-search-elasticsearch/src/main/java/org/apache/james/quota/search/elasticsearch/events/ElasticSearchQuotaMailboxListener.java b/mailbox/plugin/quota-search-elasticsearch/src/main/java/org/apache/james/quota/search/elasticsearch/events/ElasticSearchQuotaMailboxListener.java
index 4fb96cb..7d27e53 100644
--- a/mailbox/plugin/quota-search-elasticsearch/src/main/java/org/apache/james/quota/search/elasticsearch/events/ElasticSearchQuotaMailboxListener.java
+++ b/mailbox/plugin/quota-search-elasticsearch/src/main/java/org/apache/james/quota/search/elasticsearch/events/ElasticSearchQuotaMailboxListener.java
@@ -23,7 +23,10 @@ import java.io.IOException;
import javax.inject.Inject;
import javax.inject.Named;
+import org.apache.james.backends.es.DocumentId;
import org.apache.james.backends.es.ElasticSearchIndexer;
+import org.apache.james.backends.es.RoutingKey;
+import org.apache.james.core.User;
import org.apache.james.mailbox.events.Event;
import org.apache.james.mailbox.events.Group;
import org.apache.james.mailbox.events.MailboxListener;
@@ -64,7 +67,17 @@ public class ElasticSearchQuotaMailboxListener implements MailboxListener.GroupM
}
private void handleEvent(QuotaUsageUpdatedEvent event) throws IOException {
- indexer.index(event.getUser().asString(),
- quotaRatioToElasticSearchJson.convertToJson(event));
+ User user = event.getUser();
+ indexer.index(toDocumentId(user),
+ quotaRatioToElasticSearchJson.convertToJson(event),
+ toRoutingKey(user));
+ }
+
+ private RoutingKey toRoutingKey(User user) {
+ return RoutingKey.fromString(user.asString());
+ }
+
+ private DocumentId toDocumentId(User user) {
+ return DocumentId.fromString(user.asString());
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[james-project] 03/06: JAMES-2917 Require ElasticSearch routing
upon index
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 6129d92de7b091b82db85a873be383f1e834853a
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Thu Oct 10 12:57:17 2019 +0700
JAMES-2917 Require ElasticSearch routing upon index
---
.../main/java/org/apache/james/backends/es/NodeMappingFactory.java | 2 ++
.../apache/james/mailbox/elasticsearch/MailboxMappingFactory.java | 6 ++++++
.../james/quota/search/elasticsearch/QuotaRatioMappingFactory.java | 6 ++++++
3 files changed, 14 insertions(+)
diff --git a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/NodeMappingFactory.java b/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/NodeMappingFactory.java
index 2dfa375..05e15d4 100644
--- a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/NodeMappingFactory.java
+++ b/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/NodeMappingFactory.java
@@ -41,6 +41,8 @@ public class NodeMappingFactory {
public static final String TEXT = "text";
public static final String KEYWORD = "keyword";
public static final String PROPERTIES = "properties";
+ public static final String ROUTING = "_routing";
+ public static final String REQUIRED = "required";
public static final String DATE = "date";
public static final String FORMAT = "format";
public static final String NESTED = "nested";
diff --git a/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/MailboxMappingFactory.java b/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/MailboxMappingFactory.java
index 9494894..22d915f 100644
--- a/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/MailboxMappingFactory.java
+++ b/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/MailboxMappingFactory.java
@@ -32,6 +32,8 @@ import static org.apache.james.backends.es.NodeMappingFactory.NESTED;
import static org.apache.james.backends.es.NodeMappingFactory.NORMALIZER;
import static org.apache.james.backends.es.NodeMappingFactory.PROPERTIES;
import static org.apache.james.backends.es.NodeMappingFactory.RAW;
+import static org.apache.james.backends.es.NodeMappingFactory.REQUIRED;
+import static org.apache.james.backends.es.NodeMappingFactory.ROUTING;
import static org.apache.james.backends.es.NodeMappingFactory.SEARCH_ANALYZER;
import static org.apache.james.backends.es.NodeMappingFactory.SNOWBALL;
import static org.apache.james.backends.es.NodeMappingFactory.SPLIT_EMAIL;
@@ -87,6 +89,10 @@ public class MailboxMappingFactory {
.field("dynamic", "strict")
+ .startObject(ROUTING)
+ .field(REQUIRED, true)
+ .endObject()
+
.startObject(PROPERTIES)
.startObject(MESSAGE_ID)
diff --git a/mailbox/plugin/quota-search-elasticsearch/src/main/java/org/apache/james/quota/search/elasticsearch/QuotaRatioMappingFactory.java b/mailbox/plugin/quota-search-elasticsearch/src/main/java/org/apache/james/quota/search/elasticsearch/QuotaRatioMappingFactory.java
index 41883f7..fc6ad52 100644
--- a/mailbox/plugin/quota-search-elasticsearch/src/main/java/org/apache/james/quota/search/elasticsearch/QuotaRatioMappingFactory.java
+++ b/mailbox/plugin/quota-search-elasticsearch/src/main/java/org/apache/james/quota/search/elasticsearch/QuotaRatioMappingFactory.java
@@ -22,6 +22,8 @@ package org.apache.james.quota.search.elasticsearch;
import static org.apache.james.backends.es.NodeMappingFactory.DOUBLE;
import static org.apache.james.backends.es.NodeMappingFactory.KEYWORD;
import static org.apache.james.backends.es.NodeMappingFactory.PROPERTIES;
+import static org.apache.james.backends.es.NodeMappingFactory.REQUIRED;
+import static org.apache.james.backends.es.NodeMappingFactory.ROUTING;
import static org.apache.james.backends.es.NodeMappingFactory.TYPE;
import static org.apache.james.quota.search.elasticsearch.json.JsonMessageConstants.DOMAIN;
import static org.apache.james.quota.search.elasticsearch.json.JsonMessageConstants.QUOTA_RATIO;
@@ -38,6 +40,10 @@ class QuotaRatioMappingFactory {
try {
return jsonBuilder()
.startObject()
+ .startObject(ROUTING)
+ .field(REQUIRED, true)
+ .endObject()
+
.startObject(PROPERTIES)
.startObject(USER)
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[james-project] 04/06: JAMES-2917 Solve some RequestOptions related
deprecation warnings
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 0d72783ff472b0a0e027bb2473465cc3c5985c73
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Thu Oct 10 13:08:24 2019 +0700
JAMES-2917 Solve some RequestOptions related deprecation warnings
---
.../java/org/apache/james/backends/es/search/ScrolledSearch.java | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/search/ScrolledSearch.java b/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/search/ScrolledSearch.java
index 3a083e0..834f801 100644
--- a/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/search/ScrolledSearch.java
+++ b/backends-common/elasticsearch/src/main/java/org/apache/james/backends/es/search/ScrolledSearch.java
@@ -32,6 +32,7 @@ import org.elasticsearch.action.search.ClearScrollRequest;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.search.SearchScrollRequest;
+import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.unit.TimeValue;
import org.elasticsearch.search.SearchHit;
@@ -46,7 +47,7 @@ public class ScrolledSearch {
ScrollIterator(RestHighLevelClient client, SearchRequest searchRequest) {
this.client = client;
ListenerToFuture<SearchResponse> listener = new ListenerToFuture<>();
- client.searchAsync(searchRequest, listener);
+ client.searchAsync(searchRequest, RequestOptions.DEFAULT, listener);
this.searchResponseFuture = listener.getFuture();
}
@@ -55,7 +56,7 @@ public class ScrolledSearch {
public void close() throws IOException {
ClearScrollRequest clearScrollRequest = new ClearScrollRequest();
clearScrollRequest.addScrollId(searchResponseFuture.join().getScrollId());
- client.clearScroll(clearScrollRequest);
+ client.clearScroll(clearScrollRequest, RequestOptions.DEFAULT);
}
@Override
@@ -68,10 +69,11 @@ public class ScrolledSearch {
public SearchResponse next() {
SearchResponse result = searchResponseFuture.join();
ListenerToFuture<SearchResponse> listener = new ListenerToFuture<>();
- client.searchScrollAsync(
+ client.scrollAsync(
new SearchScrollRequest()
.scrollId(result.getScrollId())
.scroll(TIMEOUT),
+ RequestOptions.DEFAULT,
listener);
searchResponseFuture = listener.getFuture();
return result;
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org