You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by bt...@apache.org on 2020/05/04 03:21:19 UTC
[james-project] branch master updated (22d8d93 -> ca5389a)
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git.
from 22d8d93 JAMES-3140: Hiding technical details into class for CachedBlobStore
new 0b47fd7 JAMES-2997 JMAP attachments should use java.util.Objects
new 9d6aca4 JAMES-2997 Strong type for ContentType Field
new e7de644 JAMES-2997 TextExtractor should rely on ContentType
new cb51ca8 JAMES-2997 JSoupTextExtractor need to handle correctly charset
new 1b4e0c8 JAMES-2997 DefaultTextExtractor need to handle correctly charset
new 8f74fcf JAMES-2997 Move ContentType JavaDoc to the class
new ad5b0b6 JAMES-2997 Strong type for ContentType Field
new 2af5c40 JAMES-2997 Specifiy charset for some emails missing them
new 4501b92 JAMES-3105 Add an option to trust message denormalization when recomputing counters
new 7e873eb JAMES-3105 Expose trustMessageDenormalization over webadmin
new b72ac08 JAMES-3105 s/trustMessageDenormalization/trustMessageProjection/
new ca5389a JAMES-3105 Expose trustMessageDenormalization over webadmin
The 12 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../apache/james/mailbox/AttachmentManager.java | 3 +-
.../james/mailbox/extractor/TextExtractor.java | 4 +-
.../james/mailbox/model/AttachmentMetadata.java | 17 +-
.../java/org/apache/james/mailbox/model/Blob.java | 13 +-
.../apache/james/mailbox/model/ContentType.java | 240 +++++++++++
.../james/mailbox/model/ParsedAttachment.java | 12 +-
.../mailbox/model/AttachmentMetadataTest.java | 2 +-
.../org/apache/james/mailbox/model/BlobTest.java | 2 +-
.../james/mailbox/model/ContentTypeTest.java | 179 +++++++++
.../cassandra/mail/CassandraAttachmentDAOV2.java | 11 +-
.../cassandra/mail/CassandraAttachmentMapper.java | 3 +-
.../mail/task/RecomputeMailboxCountersService.java | 54 ++-
.../mail/task/RecomputeMailboxCountersTask.java | 10 +-
.../mail/task/RecomputeMailboxCountersTaskDTO.java | 19 +-
.../task/RecomputeMailboxCountersServiceTest.java | 440 +++++++++++++--------
...omputeMailboxCountersTaskSerializationTest.java | 46 ++-
.../james/mailbox/elasticsearch/json/MimePart.java | 35 +-
.../json/MimePartContainerBuilder.java | 6 +-
.../mailbox/elasticsearch/json/MimePartParser.java | 12 +-
.../json/RootMimePartContainerBuilder.java | 6 +-
...asticSearchListeningMessageSearchIndexTest.java | 3 +-
.../mailbox/elasticsearch/json/MimePartTest.java | 11 +-
.../inmemory/mail/InMemoryAttachmentMapper.java | 3 +-
.../mailbox/store/search/PDFTextExtractor.java | 10 +-
.../mailbox/store/search/PDFTextExtractorTest.java | 7 +-
.../mailbox/store/StoreAttachmentManager.java | 3 +-
.../james/mailbox/store/StoreBlobManager.java | 3 +-
.../store/extractor/DefaultTextExtractor.java | 9 +-
.../store/extractor/JsoupTextExtractor.java | 24 +-
.../james/mailbox/store/mail/AttachmentMapper.java | 3 +-
.../store/mail/model/impl/MessageParser.java | 8 +-
.../james/mailbox/store/StoreBlobManagerTest.java | 3 +-
.../store/extractor/DefaultTextExtractorTest.java | 14 +-
.../store/extractor/JsoupTextExtractorTest.java | 33 +-
.../store/mail/model/AttachmentMapperTest.java | 19 +-
.../store/mail/model/impl/MessageParserTest.java | 24 +-
.../test/resources/documents/html-iso-8859-1.html | 21 +-
.../resources/documents/simple-text-iso-8859-1.txt | 1 +
.../eml/emailWithNonIndexableAttachment.eml | 2 +-
.../james/mailbox/tika/CachingTextExtractor.java | 5 +-
.../tika/ContentTypeFilteringTextExtractor.java | 12 +-
.../james/mailbox/tika/TikaConfiguration.java | 11 +-
.../apache/james/mailbox/tika/TikaHttpClient.java | 4 +-
.../james/mailbox/tika/TikaHttpClientImpl.java | 9 +-
.../james/mailbox/tika/TikaTextExtractor.java | 8 +-
.../mailbox/tika/CachingTextExtractorTest.java | 3 +-
.../ContentTypeFilteringTextExtractorTest.java | 22 +-
.../james/mailbox/tika/TikaTextExtractorTest.java | 38 +-
.../modules/mailbox/TikaConfigurationReader.java | 4 +-
.../mailbox/TikaConfigurationReaderTest.java | 5 +-
.../methods/integration/SetMessagesMethodTest.java | 42 +-
.../james/jmap/draft/json/ObjectMapperFactory.java | 21 +
.../jmap/draft/methods/MIMEMessageConverter.java | 3 +-
.../apache/james/jmap/draft/model/Attachment.java | 38 +-
.../org/apache/james/jmap/http/DownloadRoutes.java | 5 +-
.../org/apache/james/jmap/http/UploadRoutes.java | 11 +-
.../james/jmap/draft/model/AttachmentTest.java | 7 +-
.../RecomputeMailboxCountersRequestToTask.java | 33 +-
.../RecomputeMailboxCountersRequestToTaskTest.java | 50 ++-
src/site/markdown/server/manage-webadmin.md | 10 +
60 files changed, 1244 insertions(+), 412 deletions(-)
create mode 100644 mailbox/api/src/main/java/org/apache/james/mailbox/model/ContentType.java
create mode 100644 mailbox/api/src/test/java/org/apache/james/mailbox/model/ContentTypeTest.java
copy backends-common/cassandra/src/main/java/org/apache/james/backends/cassandra/init/configuration/InjectionNames.java => mailbox/store/src/test/resources/documents/html-iso-8859-1.html (69%)
create mode 100644 mailbox/store/src/test/resources/documents/simple-text-iso-8859-1.txt
copy mailbox/api/src/test/java/org/apache/james/mailbox/model/BlobIdTest.java => server/protocols/webadmin/webadmin-cassandra/src/test/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTaskTest.java (54%)
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[james-project] 11/12: JAMES-3105
s/trustMessageDenormalization/trustMessageProjection/
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit b72ac0831458a0d7df216361d721a95f01797738
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Fri Apr 24 17:23:40 2020 +0700
JAMES-3105 s/trustMessageDenormalization/trustMessageProjection/
---
.../mail/task/RecomputeMailboxCountersService.java | 20 ++++++++++----------
.../mail/task/RecomputeMailboxCountersTaskDTO.java | 16 ++++++++--------
.../task/RecomputeMailboxCountersServiceTest.java | 4 ++--
...ecomputeMailboxCountersTaskSerializationTest.java | 8 ++++----
.../RecomputeMailboxCountersRequestToTask.java | 4 ++--
.../RecomputeMailboxCountersRequestToTaskTest.java | 8 ++++----
src/site/markdown/server/manage-webadmin.md | 2 +-
7 files changed, 31 insertions(+), 31 deletions(-)
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java
index 7970a32..7bb4093 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java
@@ -56,11 +56,11 @@ public class RecomputeMailboxCountersService {
private static final int MESSAGE_CONCURRENCY = 8;
public static class Options {
- public static Options trustMessageDenormalization() {
+ public static Options trustMessageProjection() {
return of(true);
}
- public static Options recheckMessageDenormalization() {
+ public static Options recheckMessageProjection() {
return of(false);
}
@@ -68,14 +68,14 @@ public class RecomputeMailboxCountersService {
return new Options(value);
}
- private final boolean trustMessageDenormalization;
+ private final boolean trustMessageProjection;
- private Options(boolean trustMessageDenormalization) {
- this.trustMessageDenormalization = trustMessageDenormalization;
+ private Options(boolean trustMessageProjection) {
+ this.trustMessageProjection = trustMessageProjection;
}
- public boolean isMessageDenormalizationTrusted() {
- return trustMessageDenormalization;
+ public boolean isMessageProjectionTrusted() {
+ return trustMessageProjection;
}
@Override
@@ -83,14 +83,14 @@ public class RecomputeMailboxCountersService {
if (o instanceof Options) {
Options options = (Options) o;
- return Objects.equals(this.trustMessageDenormalization, options.trustMessageDenormalization);
+ return Objects.equals(this.trustMessageProjection, options.trustMessageProjection);
}
return false;
}
@Override
public final int hashCode() {
- return Objects.hash(trustMessageDenormalization);
+ return Objects.hash(trustMessageProjection);
}
}
@@ -235,7 +235,7 @@ public class RecomputeMailboxCountersService {
private Flux<ComposedMessageIdWithMetaData> latestMetadata(CassandraId mailboxId,
ComposedMessageIdWithMetaData message,
Options options) {
- if (options.isMessageDenormalizationTrusted()) {
+ if (options.isMessageProjectionTrusted()) {
return Flux.just(message);
}
CassandraMessageId messageId = (CassandraMessageId) message.getComposedMessageId().getMessageId();
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTaskDTO.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTaskDTO.java
index d855aa1..6dde6e4 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTaskDTO.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTaskDTO.java
@@ -29,7 +29,7 @@ import com.fasterxml.jackson.annotation.JsonProperty;
public class RecomputeMailboxCountersTaskDTO implements TaskDTO {
private static RecomputeMailboxCountersTaskDTO toDTO(RecomputeMailboxCountersTask domainObject, String typeName) {
- return new RecomputeMailboxCountersTaskDTO(typeName, Optional.of(domainObject.getOptions().isMessageDenormalizationTrusted()));
+ return new RecomputeMailboxCountersTaskDTO(typeName, Optional.of(domainObject.getOptions().isMessageProjectionTrusted()));
}
public static TaskDTOModule<RecomputeMailboxCountersTask, RecomputeMailboxCountersTaskDTO> module(RecomputeMailboxCountersService service) {
@@ -43,16 +43,16 @@ public class RecomputeMailboxCountersTaskDTO implements TaskDTO {
}
private final String type;
- private final Optional<Boolean> trustMessageDenormalization;
+ private final Optional<Boolean> trustMessageProjection;
public RecomputeMailboxCountersTaskDTO(@JsonProperty("type") String type,
- @JsonProperty("trustMessageDenormalization") Optional<Boolean> trustMessageDenormalization) {
+ @JsonProperty("trustMessageProjection") Optional<Boolean> trustMessageProjection) {
this.type = type;
- this.trustMessageDenormalization = trustMessageDenormalization;
+ this.trustMessageProjection = trustMessageProjection;
}
private RecomputeMailboxCountersTask toDomainObject(RecomputeMailboxCountersService service) {
- Options options = trustMessageDenormalization.map(Options::of).orElse(Options.recheckMessageDenormalization());
+ Options options = trustMessageProjection.map(Options::of).orElse(Options.recheckMessageProjection());
return new RecomputeMailboxCountersTask(service, options);
}
@@ -62,8 +62,8 @@ public class RecomputeMailboxCountersTaskDTO implements TaskDTO {
return type;
}
- @JsonProperty("trustMessageDenormalization")
- public Optional<Boolean> trustMessageDenormalization() {
- return trustMessageDenormalization;
+ @JsonProperty("trustMessageProjection")
+ public Optional<Boolean> trustMessageProjection() {
+ return trustMessageProjection;
}
}
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersServiceTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersServiceTest.java
index 4df80a4..fdf059a 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersServiceTest.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersServiceTest.java
@@ -113,7 +113,7 @@ class RecomputeMailboxCountersServiceTest {
@Override
public Options options() {
- return Options.trustMessageDenormalization();
+ return Options.trustMessageProjection();
}
@Override
@@ -164,7 +164,7 @@ class RecomputeMailboxCountersServiceTest {
@Override
public Options options() {
- return Options.recheckMessageDenormalization();
+ return Options.recheckMessageProjection();
}
@Override
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTaskSerializationTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTaskSerializationTest.java
index e46fa08..d5bd9d8 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTaskSerializationTest.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTaskSerializationTest.java
@@ -36,15 +36,15 @@ class RecomputeMailboxCountersTaskSerializationTest {
private static final String MAILBOX_ID_AS_STRING = "464765a0-e4e7-11e4-aba4-710c1de3782b";
private static final RecomputeMailboxCountersService SERVICE = mock(RecomputeMailboxCountersService.class);
- private static final RecomputeMailboxCountersTask TASK_TRUSTED = new RecomputeMailboxCountersTask(SERVICE, Options.trustMessageDenormalization());
- private static final RecomputeMailboxCountersTask TASK_UNTRUSTED = new RecomputeMailboxCountersTask(SERVICE, Options.recheckMessageDenormalization());
+ private static final RecomputeMailboxCountersTask TASK_TRUSTED = new RecomputeMailboxCountersTask(SERVICE, Options.trustMessageProjection());
+ private static final RecomputeMailboxCountersTask TASK_UNTRUSTED = new RecomputeMailboxCountersTask(SERVICE, Options.recheckMessageProjection());
private static final String SERIALIZED_TASK_TRUSTED = "{" +
" \"type\": \"recompute-mailbox-counters\"," +
- " \"trustMessageDenormalization\": true" +
+ " \"trustMessageProjection\": true" +
"}";
private static final String SERIALIZED_TASK_UNTRUSTED = "{" +
" \"type\": \"recompute-mailbox-counters\"," +
- " \"trustMessageDenormalization\": false" +
+ " \"trustMessageProjection\": false" +
"}";
private static final String SERIALIZED_TASK_OLD = "{\"type\": \"recompute-mailbox-counters\"}";
private static final RecomputeMailboxCountersTask.Details DETAILS = new RecomputeMailboxCountersTask.Details(TIMESTAMP, 12, ImmutableList.of(MAILBOX_ID_AS_STRING));
diff --git a/server/protocols/webadmin/webadmin-cassandra/src/main/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTask.java b/server/protocols/webadmin/webadmin-cassandra/src/main/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTask.java
index a090913..f1e230c 100644
--- a/server/protocols/webadmin/webadmin-cassandra/src/main/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTask.java
+++ b/server/protocols/webadmin/webadmin-cassandra/src/main/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTask.java
@@ -36,7 +36,7 @@ import spark.Request;
public class RecomputeMailboxCountersRequestToTask extends TaskFromRequestRegistry.TaskRegistration {
private static final TaskRegistrationKey REGISTRATION_KEY = TaskRegistrationKey.of("RecomputeMailboxCounters");
- public static final String TRUST_PARAM = "trustMessageDenormalization";
+ public static final String TRUST_PARAM = "trustMessageProjection";
@Inject
public RecomputeMailboxCountersRequestToTask(RecomputeMailboxCountersService service) {
@@ -53,7 +53,7 @@ public class RecomputeMailboxCountersRequestToTask extends TaskFromRequestRegist
static Options parseOptions(Optional<String> stringValue) {
return stringValue
.map(RecomputeMailboxCountersRequestToTask::parseOptions)
- .orElse(Options.recheckMessageDenormalization());
+ .orElse(Options.recheckMessageProjection());
}
private static Options parseOptions(String stringValue) {
diff --git a/server/protocols/webadmin/webadmin-cassandra/src/test/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTaskTest.java b/server/protocols/webadmin/webadmin-cassandra/src/test/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTaskTest.java
index 780fb71..76691ee 100644
--- a/server/protocols/webadmin/webadmin-cassandra/src/test/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTaskTest.java
+++ b/server/protocols/webadmin/webadmin-cassandra/src/test/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTaskTest.java
@@ -31,25 +31,25 @@ class RecomputeMailboxCountersRequestToTaskTest {
@Test
void parseOptionsShouldReturnRecheckWhenEmpty() {
assertThat(RecomputeMailboxCountersRequestToTask.parseOptions(Optional.empty()))
- .isEqualTo(Options.recheckMessageDenormalization());
+ .isEqualTo(Options.recheckMessageProjection());
}
@Test
void parseOptionsShouldReturnRecheckWhenFalse() {
assertThat(RecomputeMailboxCountersRequestToTask.parseOptions(Optional.of("false")))
- .isEqualTo(Options.recheckMessageDenormalization());
+ .isEqualTo(Options.recheckMessageProjection());
}
@Test
void parseOptionsShouldBeCaseIncentive() {
assertThat(RecomputeMailboxCountersRequestToTask.parseOptions(Optional.of("False")))
- .isEqualTo(Options.recheckMessageDenormalization());
+ .isEqualTo(Options.recheckMessageProjection());
}
@Test
void parseOptionsShouldReturnTrueWhenTrust() {
assertThat(RecomputeMailboxCountersRequestToTask.parseOptions(Optional.of("true")))
- .isEqualTo(Options.trustMessageDenormalization());
+ .isEqualTo(Options.trustMessageProjection());
}
@Test
diff --git a/src/site/markdown/server/manage-webadmin.md b/src/site/markdown/server/manage-webadmin.md
index e3823fa..c16e700 100644
--- a/src/site/markdown/server/manage-webadmin.md
+++ b/src/site/markdown/server/manage-webadmin.md
@@ -491,7 +491,7 @@ Rerunning the task will *eventually* provide the consistent result. As such we a
In order to ensure being offline, stop the traffic on SMTP, JMAP and IMAP ports, for example via re-configuration or
firewall rules.
-`trustMessageDenormalization` query parameter can be set to `true`. This will result in a better performance running the
+`trustMessageProjection` query parameter can be set to `true`. This will result in a better performance running the
task at the cost of safety in the face of message denormalization inconsistencies. Defaults to false, which generates
additional checks.
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[james-project] 12/12: JAMES-3105 Expose
trustMessageDenormalization over webadmin
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit ca5389a1cb140caeeadb7f43ebafc09351e57b60
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Mon Apr 27 10:06:18 2020 +0700
JAMES-3105 Expose trustMessageDenormalization over webadmin
---
.../routes/RecomputeMailboxCountersRequestToTask.java | 4 ++--
.../routes/RecomputeMailboxCountersRequestToTaskTest.java | 2 +-
src/site/markdown/server/manage-webadmin.md | 12 +++++++++---
3 files changed, 12 insertions(+), 6 deletions(-)
diff --git a/server/protocols/webadmin/webadmin-cassandra/src/main/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTask.java b/server/protocols/webadmin/webadmin-cassandra/src/main/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTask.java
index f1e230c..adf3c8f 100644
--- a/server/protocols/webadmin/webadmin-cassandra/src/main/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTask.java
+++ b/server/protocols/webadmin/webadmin-cassandra/src/main/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTask.java
@@ -36,7 +36,7 @@ import spark.Request;
public class RecomputeMailboxCountersRequestToTask extends TaskFromRequestRegistry.TaskRegistration {
private static final TaskRegistrationKey REGISTRATION_KEY = TaskRegistrationKey.of("RecomputeMailboxCounters");
- public static final String TRUST_PARAM = "trustMessageProjection";
+ private static final String TRUST_PARAM = "trustMessageProjection";
@Inject
public RecomputeMailboxCountersRequestToTask(RecomputeMailboxCountersService service) {
@@ -45,7 +45,7 @@ public class RecomputeMailboxCountersRequestToTask extends TaskFromRequestRegist
}
private static Options parseOptions(Request request) {
- var stringValue = Optional.ofNullable(request.queryParams(TRUST_PARAM));
+ Optional<String> stringValue = Optional.ofNullable(request.queryParams(TRUST_PARAM));
return parseOptions(stringValue);
}
diff --git a/server/protocols/webadmin/webadmin-cassandra/src/test/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTaskTest.java b/server/protocols/webadmin/webadmin-cassandra/src/test/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTaskTest.java
index 76691ee..c0772d9 100644
--- a/server/protocols/webadmin/webadmin-cassandra/src/test/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTaskTest.java
+++ b/server/protocols/webadmin/webadmin-cassandra/src/test/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTaskTest.java
@@ -41,7 +41,7 @@ class RecomputeMailboxCountersRequestToTaskTest {
}
@Test
- void parseOptionsShouldBeCaseIncentive() {
+ void parseOptionsShouldNotBeSensitiveToCase() {
assertThat(RecomputeMailboxCountersRequestToTask.parseOptions(Optional.of("False")))
.isEqualTo(Options.recheckMessageProjection());
}
diff --git a/src/site/markdown/server/manage-webadmin.md b/src/site/markdown/server/manage-webadmin.md
index c16e700..c480748 100644
--- a/src/site/markdown/server/manage-webadmin.md
+++ b/src/site/markdown/server/manage-webadmin.md
@@ -491,9 +491,15 @@ Rerunning the task will *eventually* provide the consistent result. As such we a
In order to ensure being offline, stop the traffic on SMTP, JMAP and IMAP ports, for example via re-configuration or
firewall rules.
-`trustMessageProjection` query parameter can be set to `true`. This will result in a better performance running the
-task at the cost of safety in the face of message denormalization inconsistencies. Defaults to false, which generates
-additional checks.
+`trustMessageProjection` query parameter can be set to `true`. Cpntent of `messageIdTable` (listing messages by their
+mailbox context) table will be trusted and not compared against content of `imapUidTable` table (listing messages by their
+messageId mailbox independent identifier). This will result in a better performance running the
+task at the cost of safety in the face of message denormalization inconsistencies.
+
+Defaults to false, which generates
+additional checks. You can read
+[this ADR](https://github.com/apache/james-project/blob/master/src/adr/0022-cassandra-message-inconsistency.md) to
+better understand the message projection and how it can become inconsistent.
#### Recomputing Global JMAP fast message view projection
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[james-project] 02/12: JAMES-2997 Strong type for ContentType Field
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 9d6aca49311252e6014e226fe7f9543996415147
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Mon Apr 27 18:01:05 2020 +0700
JAMES-2997 Strong type for ContentType Field
---
.../apache/james/mailbox/AttachmentManager.java | 3 +-
.../james/mailbox/model/AttachmentMetadata.java | 17 +++---
.../java/org/apache/james/mailbox/model/Blob.java | 13 +++--
.../apache/james/mailbox/model/ContentType.java | 60 ++++++++++++++++++++++
.../james/mailbox/model/ParsedAttachment.java | 12 +++--
.../mailbox/model/AttachmentMetadataTest.java | 2 +-
.../org/apache/james/mailbox/model/BlobTest.java | 2 +-
.../cassandra/mail/CassandraAttachmentDAOV2.java | 11 ++--
.../cassandra/mail/CassandraAttachmentMapper.java | 3 +-
.../inmemory/mail/InMemoryAttachmentMapper.java | 3 +-
.../mailbox/store/StoreAttachmentManager.java | 3 +-
.../james/mailbox/store/StoreBlobManager.java | 3 +-
.../james/mailbox/store/mail/AttachmentMapper.java | 3 +-
.../store/mail/model/impl/MessageParser.java | 8 +--
.../mailbox/store/search/MessageSearches.java | 3 +-
.../james/mailbox/store/StoreBlobManagerTest.java | 3 +-
.../store/mail/model/AttachmentMapperTest.java | 19 +++----
.../store/mail/model/impl/MessageParserTest.java | 24 +++++----
.../methods/integration/SetMessagesMethodTest.java | 42 +++++++--------
.../james/jmap/draft/json/ObjectMapperFactory.java | 21 ++++++++
.../jmap/draft/methods/MIMEMessageConverter.java | 3 +-
.../apache/james/jmap/draft/model/Attachment.java | 18 ++++---
.../org/apache/james/jmap/http/DownloadRoutes.java | 5 +-
.../org/apache/james/jmap/http/UploadRoutes.java | 11 ++--
.../james/jmap/draft/model/AttachmentTest.java | 7 +--
25 files changed, 210 insertions(+), 89 deletions(-)
diff --git a/mailbox/api/src/main/java/org/apache/james/mailbox/AttachmentManager.java b/mailbox/api/src/main/java/org/apache/james/mailbox/AttachmentManager.java
index 468e037..3a7424b 100644
--- a/mailbox/api/src/main/java/org/apache/james/mailbox/AttachmentManager.java
+++ b/mailbox/api/src/main/java/org/apache/james/mailbox/AttachmentManager.java
@@ -27,6 +27,7 @@ import org.apache.james.mailbox.exception.AttachmentNotFoundException;
import org.apache.james.mailbox.exception.MailboxException;
import org.apache.james.mailbox.model.AttachmentId;
import org.apache.james.mailbox.model.AttachmentMetadata;
+import org.apache.james.mailbox.model.ContentType;
import org.reactivestreams.Publisher;
public interface AttachmentManager extends AttachmentContentLoader {
@@ -37,7 +38,7 @@ public interface AttachmentManager extends AttachmentContentLoader {
List<AttachmentMetadata> getAttachments(List<AttachmentId> attachmentIds, MailboxSession mailboxSession) throws MailboxException;
- Publisher<AttachmentMetadata> storeAttachment(String contentType, InputStream attachmentContent, MailboxSession mailboxSession);
+ Publisher<AttachmentMetadata> storeAttachment(ContentType contentType, InputStream attachmentContent, MailboxSession mailboxSession);
InputStream loadAttachmentContent(AttachmentId attachmentId, MailboxSession mailboxSession) throws AttachmentNotFoundException, IOException;
diff --git a/mailbox/api/src/main/java/org/apache/james/mailbox/model/AttachmentMetadata.java b/mailbox/api/src/main/java/org/apache/james/mailbox/model/AttachmentMetadata.java
index 9676f76..0f4e9f7 100644
--- a/mailbox/api/src/main/java/org/apache/james/mailbox/model/AttachmentMetadata.java
+++ b/mailbox/api/src/main/java/org/apache/james/mailbox/model/AttachmentMetadata.java
@@ -22,13 +22,12 @@ package org.apache.james.mailbox.model;
import com.google.common.base.MoreObjects;
import com.google.common.base.Objects;
import com.google.common.base.Preconditions;
-import com.google.common.base.Strings;
public class AttachmentMetadata {
public static class Builder {
private AttachmentId attachmentId;
private Long size;
- private String type;
+ private ContentType type;
public Builder attachmentId(AttachmentId attachmentId) {
Preconditions.checkArgument(attachmentId != null);
@@ -36,12 +35,16 @@ public class AttachmentMetadata {
return this;
}
- public Builder type(String type) {
- Preconditions.checkArgument(!Strings.isNullOrEmpty(type));
+ public Builder type(ContentType type) {
this.type = type;
return this;
}
+ public Builder type(String type) {
+ this.type = ContentType.of(type);
+ return this;
+ }
+
public Builder size(long size) {
Preconditions.checkArgument(size >= 0, "'size' must be positive");
this.size = size;
@@ -62,10 +65,10 @@ public class AttachmentMetadata {
}
private final AttachmentId attachmentId;
- private final String type;
+ private final ContentType type;
private final long size;
- private AttachmentMetadata(AttachmentId attachmentId, String type, long size) {
+ private AttachmentMetadata(AttachmentId attachmentId, ContentType type, long size) {
this.attachmentId = attachmentId;
this.type = type;
this.size = size;
@@ -75,7 +78,7 @@ public class AttachmentMetadata {
return attachmentId;
}
- public String getType() {
+ public ContentType getType() {
return type;
}
diff --git a/mailbox/api/src/main/java/org/apache/james/mailbox/model/Blob.java b/mailbox/api/src/main/java/org/apache/james/mailbox/model/Blob.java
index 141f3a8..0103085 100644
--- a/mailbox/api/src/main/java/org/apache/james/mailbox/model/Blob.java
+++ b/mailbox/api/src/main/java/org/apache/james/mailbox/model/Blob.java
@@ -44,7 +44,7 @@ public class Blob {
public static class Builder {
private BlobId blobId;
private InputStreamSupplier payload;
- private String contentType;
+ private ContentType contentType;
private Long size;
private Builder() {
@@ -61,6 +61,11 @@ public class Blob {
}
public Builder contentType(String contentType) {
+ this.contentType = ContentType.of(contentType);
+ return this;
+ }
+
+ public Builder contentType(ContentType contentType) {
this.contentType = contentType;
return this;
}
@@ -86,11 +91,11 @@ public class Blob {
private final BlobId blobId;
private final InputStreamSupplier payload;
- private final String contentType;
+ private final ContentType contentType;
private final long size;
@VisibleForTesting
- Blob(BlobId blobId, InputStreamSupplier payload, String contentType, long size) {
+ Blob(BlobId blobId, InputStreamSupplier payload, ContentType contentType, long size) {
this.blobId = blobId;
this.payload = payload;
this.contentType = contentType;
@@ -109,7 +114,7 @@ public class Blob {
return size;
}
- public String getContentType() {
+ public ContentType getContentType() {
return contentType;
}
diff --git a/mailbox/api/src/main/java/org/apache/james/mailbox/model/ContentType.java b/mailbox/api/src/main/java/org/apache/james/mailbox/model/ContentType.java
new file mode 100644
index 0000000..ed8ecf5
--- /dev/null
+++ b/mailbox/api/src/main/java/org/apache/james/mailbox/model/ContentType.java
@@ -0,0 +1,60 @@
+/****************************************************************
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information *
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the *
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the *
+ * specific language governing permissions and limitations *
+ * under the License. *
+ ****************************************************************/
+
+package org.apache.james.mailbox.model;
+
+import com.google.common.base.Preconditions;
+import com.google.common.base.Strings;
+
+public class ContentType {
+ public static ContentType of(String value) {
+ Preconditions.checkState(!Strings.isNullOrEmpty(value), "'content type' is mandatory");
+ return new ContentType(value);
+ }
+
+ /**
+ * Follows syntax and usage as defined in https://tools.ietf.org/html/rfc2045#section-5
+ * Thus includes media type and parameters, including charset
+ * Example: text/plain; charset=utf-8
+ */
+ private final String value;
+
+ public ContentType(String value) {
+ this.value = value;
+ }
+
+ public String asString() {
+ return value;
+ }
+
+ @Override
+ public final boolean equals(Object o) {
+ if (o instanceof ContentType) {
+ ContentType that = (ContentType) o;
+
+ return java.util.Objects.equals(this.value, that.value);
+ }
+ return false;
+ }
+
+ @Override
+ public final int hashCode() {
+ return java.util.Objects.hash(value);
+ }
+}
diff --git a/mailbox/api/src/main/java/org/apache/james/mailbox/model/ParsedAttachment.java b/mailbox/api/src/main/java/org/apache/james/mailbox/model/ParsedAttachment.java
index c615473..429528e 100644
--- a/mailbox/api/src/main/java/org/apache/james/mailbox/model/ParsedAttachment.java
+++ b/mailbox/api/src/main/java/org/apache/james/mailbox/model/ParsedAttachment.java
@@ -26,7 +26,11 @@ public class ParsedAttachment {
interface Builder {
@FunctionalInterface
interface RequireContentType {
- RequireContent contentType(String contentType);
+ RequireContent contentType(ContentType contentType);
+
+ default RequireContent contentType(String contentType) {
+ return contentType(ContentType.of(contentType));
+ }
}
@FunctionalInterface
@@ -74,13 +78,13 @@ public class ParsedAttachment {
return contentType -> content -> name -> cid -> isInline -> new ParsedAttachment(contentType, content, name, cid, isInline);
}
- private final String contentType;
+ private final ContentType contentType;
private final byte[] content;
private final Optional<String> name;
private final Optional<Cid> cid;
private final boolean isInline;
- private ParsedAttachment(String contentType, byte[] content, Optional<String> name, Optional<Cid> cid, boolean isInline) {
+ private ParsedAttachment(ContentType contentType, byte[] content, Optional<String> name, Optional<Cid> cid, boolean isInline) {
this.contentType = contentType;
this.content = content;
this.name = name;
@@ -88,7 +92,7 @@ public class ParsedAttachment {
this.isInline = isInline;
}
- public String getContentType() {
+ public ContentType getContentType() {
return contentType;
}
diff --git a/mailbox/api/src/test/java/org/apache/james/mailbox/model/AttachmentMetadataTest.java b/mailbox/api/src/test/java/org/apache/james/mailbox/model/AttachmentMetadataTest.java
index 1c0eaba..f75bc1b 100644
--- a/mailbox/api/src/test/java/org/apache/james/mailbox/model/AttachmentMetadataTest.java
+++ b/mailbox/api/src/test/java/org/apache/james/mailbox/model/AttachmentMetadataTest.java
@@ -35,7 +35,7 @@ class AttachmentMetadataTest {
@Test
void builderShouldThrowWhenTypeIsNull() {
assertThatThrownBy(() -> AttachmentMetadata.builder()
- .type(null))
+ .type((String) null))
.isInstanceOf(IllegalArgumentException.class);
}
diff --git a/mailbox/api/src/test/java/org/apache/james/mailbox/model/BlobTest.java b/mailbox/api/src/test/java/org/apache/james/mailbox/model/BlobTest.java
index 55c3b3c..931cf72 100644
--- a/mailbox/api/src/test/java/org/apache/james/mailbox/model/BlobTest.java
+++ b/mailbox/api/src/test/java/org/apache/james/mailbox/model/BlobTest.java
@@ -32,7 +32,7 @@ import nl.jqno.equalsverifier.EqualsVerifier;
class BlobTest {
static final BlobId ID = BlobId.fromString("123");
- static final String CONTENT_TYPE = "text/plain";
+ static final ContentType CONTENT_TYPE = ContentType.of("text/plain");
static final InputStreamSupplier PAYLOAD = () -> new ByteArrayInputStream("abc".getBytes(StandardCharsets.UTF_8));
static final int LENGTH = 3;
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraAttachmentDAOV2.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraAttachmentDAOV2.java
index 5d1cb80..8627a73 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraAttachmentDAOV2.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraAttachmentDAOV2.java
@@ -40,6 +40,7 @@ import org.apache.james.backends.cassandra.utils.CassandraAsyncExecutor;
import org.apache.james.blob.api.BlobId;
import org.apache.james.mailbox.model.AttachmentId;
import org.apache.james.mailbox.model.AttachmentMetadata;
+import org.apache.james.mailbox.model.ContentType;
import com.datastax.driver.core.PreparedStatement;
import com.datastax.driver.core.Row;
@@ -52,10 +53,10 @@ public class CassandraAttachmentDAOV2 {
public static class DAOAttachment {
private final AttachmentId attachmentId;
private final BlobId blobId;
- private final String type;
+ private final ContentType type;
private final long size;
- DAOAttachment(AttachmentId attachmentId, BlobId blobId, String type, long size) {
+ DAOAttachment(AttachmentId attachmentId, BlobId blobId, ContentType type, long size) {
this.attachmentId = attachmentId;
this.blobId = blobId;
this.type = type;
@@ -70,7 +71,7 @@ public class CassandraAttachmentDAOV2 {
return blobId;
}
- public String getType() {
+ public ContentType getType() {
return type;
}
@@ -117,7 +118,7 @@ public class CassandraAttachmentDAOV2 {
return new DAOAttachment(
AttachmentId.from(row.getString(ID)),
blobIfFactory.from(row.getString(BLOB_ID)),
- row.getString(TYPE),
+ ContentType.of(row.getString(TYPE)),
row.getLong(SIZE));
}
@@ -166,7 +167,7 @@ public class CassandraAttachmentDAOV2 {
.setUUID(ID_AS_UUID, attachment.getAttachmentId().asUUID())
.setString(ID, attachment.getAttachmentId().getId())
.setLong(SIZE, attachment.getSize())
- .setString(TYPE, attachment.getType())
+ .setString(TYPE, attachment.getType().asString())
.setString(BLOB_ID, attachment.getBlobId().asString()));
}
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraAttachmentMapper.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraAttachmentMapper.java
index 8d84078..dd6edec 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraAttachmentMapper.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/CassandraAttachmentMapper.java
@@ -35,6 +35,7 @@ import org.apache.james.mailbox.exception.AttachmentNotFoundException;
import org.apache.james.mailbox.exception.MailboxException;
import org.apache.james.mailbox.model.AttachmentId;
import org.apache.james.mailbox.model.AttachmentMetadata;
+import org.apache.james.mailbox.model.ContentType;
import org.apache.james.mailbox.model.MessageAttachmentMetadata;
import org.apache.james.mailbox.model.MessageId;
import org.apache.james.mailbox.model.ParsedAttachment;
@@ -111,7 +112,7 @@ public class CassandraAttachmentMapper implements AttachmentMapper {
}
@Override
- public Mono<AttachmentMetadata> storeAttachmentForOwner(String contentType, InputStream inputStream, Username owner) {
+ public Mono<AttachmentMetadata> storeAttachmentForOwner(ContentType contentType, InputStream inputStream, Username owner) {
CurrentPositionInputStream currentPositionInputStream = new CurrentPositionInputStream(inputStream);
AttachmentId attachmentId = AttachmentId.random();
return ownerDAO.addOwner(attachmentId, owner)
diff --git a/mailbox/memory/src/main/java/org/apache/james/mailbox/inmemory/mail/InMemoryAttachmentMapper.java b/mailbox/memory/src/main/java/org/apache/james/mailbox/inmemory/mail/InMemoryAttachmentMapper.java
index d35f515..c5237b9 100644
--- a/mailbox/memory/src/main/java/org/apache/james/mailbox/inmemory/mail/InMemoryAttachmentMapper.java
+++ b/mailbox/memory/src/main/java/org/apache/james/mailbox/inmemory/mail/InMemoryAttachmentMapper.java
@@ -33,6 +33,7 @@ import org.apache.james.mailbox.exception.AttachmentNotFoundException;
import org.apache.james.mailbox.exception.MailboxException;
import org.apache.james.mailbox.model.AttachmentId;
import org.apache.james.mailbox.model.AttachmentMetadata;
+import org.apache.james.mailbox.model.ContentType;
import org.apache.james.mailbox.model.MessageAttachmentMetadata;
import org.apache.james.mailbox.model.MessageId;
import org.apache.james.mailbox.model.ParsedAttachment;
@@ -86,7 +87,7 @@ public class InMemoryAttachmentMapper implements AttachmentMapper {
}
@Override
- public Mono<AttachmentMetadata> storeAttachmentForOwner(String contentType, InputStream inputStream, Username owner) {
+ public Mono<AttachmentMetadata> storeAttachmentForOwner(ContentType contentType, InputStream inputStream, Username owner) {
return Mono.fromCallable(() -> {
byte[] bytes = toByteArray(inputStream);
AttachmentMetadata attachment = AttachmentMetadata.builder()
diff --git a/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreAttachmentManager.java b/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreAttachmentManager.java
index 6f230b9..7a4efe2 100644
--- a/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreAttachmentManager.java
+++ b/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreAttachmentManager.java
@@ -34,6 +34,7 @@ import org.apache.james.mailbox.exception.AttachmentNotFoundException;
import org.apache.james.mailbox.exception.MailboxException;
import org.apache.james.mailbox.model.AttachmentId;
import org.apache.james.mailbox.model.AttachmentMetadata;
+import org.apache.james.mailbox.model.ContentType;
import org.apache.james.mailbox.model.MessageId;
import org.apache.james.mailbox.store.mail.AttachmentMapperFactory;
import org.reactivestreams.Publisher;
@@ -77,7 +78,7 @@ public class StoreAttachmentManager implements AttachmentManager {
}
@Override
- public Publisher<AttachmentMetadata> storeAttachment(String contentType, InputStream attachmentContent, MailboxSession mailboxSession) {
+ public Publisher<AttachmentMetadata> storeAttachment(ContentType contentType, InputStream attachmentContent, MailboxSession mailboxSession) {
return attachmentMapperFactory.getAttachmentMapper(mailboxSession)
.storeAttachmentForOwner(contentType, attachmentContent, mailboxSession.getUser());
}
diff --git a/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreBlobManager.java b/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreBlobManager.java
index 90674b1..ea7ff91 100644
--- a/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreBlobManager.java
+++ b/mailbox/store/src/main/java/org/apache/james/mailbox/store/StoreBlobManager.java
@@ -35,6 +35,7 @@ import org.apache.james.mailbox.model.AttachmentMetadata;
import org.apache.james.mailbox.model.Blob;
import org.apache.james.mailbox.model.BlobId;
import org.apache.james.mailbox.model.Content;
+import org.apache.james.mailbox.model.ContentType;
import org.apache.james.mailbox.model.FetchGroup;
import org.apache.james.mailbox.model.MessageId;
import org.apache.james.mailbox.model.MessageResult;
@@ -42,7 +43,7 @@ import org.apache.james.mailbox.model.MessageResult;
import com.github.fge.lambdas.Throwing;
public class StoreBlobManager implements BlobManager {
- public static final String MESSAGE_RFC822_CONTENT_TYPE = "message/rfc822";
+ public static final ContentType MESSAGE_RFC822_CONTENT_TYPE = ContentType.of("message/rfc822");
private final AttachmentManager attachmentManager;
private final MessageIdManager messageIdManager;
private final MessageId.Factory messageIdFactory;
diff --git a/mailbox/store/src/main/java/org/apache/james/mailbox/store/mail/AttachmentMapper.java b/mailbox/store/src/main/java/org/apache/james/mailbox/store/mail/AttachmentMapper.java
index 377651b..8a90153 100644
--- a/mailbox/store/src/main/java/org/apache/james/mailbox/store/mail/AttachmentMapper.java
+++ b/mailbox/store/src/main/java/org/apache/james/mailbox/store/mail/AttachmentMapper.java
@@ -28,6 +28,7 @@ import org.apache.james.mailbox.exception.AttachmentNotFoundException;
import org.apache.james.mailbox.exception.MailboxException;
import org.apache.james.mailbox.model.AttachmentId;
import org.apache.james.mailbox.model.AttachmentMetadata;
+import org.apache.james.mailbox.model.ContentType;
import org.apache.james.mailbox.model.MessageAttachmentMetadata;
import org.apache.james.mailbox.model.MessageId;
import org.apache.james.mailbox.model.ParsedAttachment;
@@ -42,7 +43,7 @@ public interface AttachmentMapper extends Mapper {
List<AttachmentMetadata> getAttachments(Collection<AttachmentId> attachmentIds);
- Publisher<AttachmentMetadata> storeAttachmentForOwner(String contentType, InputStream attachmentContent, Username owner);
+ Publisher<AttachmentMetadata> storeAttachmentForOwner(ContentType contentType, InputStream attachmentContent, Username owner);
List<MessageAttachmentMetadata> storeAttachmentsForMessage(Collection<ParsedAttachment> attachments, MessageId ownerMessageId) throws MailboxException;
diff --git a/mailbox/store/src/main/java/org/apache/james/mailbox/store/mail/model/impl/MessageParser.java b/mailbox/store/src/main/java/org/apache/james/mailbox/store/mail/model/impl/MessageParser.java
index b8fc5ba..328a98d 100644
--- a/mailbox/store/src/main/java/org/apache/james/mailbox/store/mail/model/impl/MessageParser.java
+++ b/mailbox/store/src/main/java/org/apache/james/mailbox/store/mail/model/impl/MessageParser.java
@@ -28,6 +28,7 @@ import java.util.Optional;
import java.util.stream.Stream;
import org.apache.james.mailbox.model.Cid;
+import org.apache.james.mailbox.model.ContentType;
import org.apache.james.mailbox.model.ParsedAttachment;
import org.apache.james.mime4j.codec.DecodeMonitor;
import org.apache.james.mime4j.dom.Body;
@@ -55,7 +56,7 @@ public class MessageParser {
private static final String CONTENT_TYPE = "Content-Type";
private static final String CONTENT_ID = "Content-ID";
private static final String CONTENT_DISPOSITION = "Content-Disposition";
- private static final String DEFAULT_CONTENT_TYPE = "application/octet-stream";
+ private static final ContentType DEFAULT_CONTENT_TYPE = ContentType.of("application/octet-stream");
private static final List<String> ATTACHMENT_CONTENT_DISPOSITIONS = ImmutableList.of(
ContentDispositionField.DISPOSITION_TYPE_ATTACHMENT.toLowerCase(Locale.US),
ContentDispositionField.DISPOSITION_TYPE_INLINE.toLowerCase(Locale.US));
@@ -122,8 +123,9 @@ public class MessageParser {
private ParsedAttachment retrieveAttachment(Entity entity) throws IOException {
Optional<ContentTypeField> contentTypeField = getContentTypeField(entity);
Optional<ContentDispositionField> contentDispositionField = getContentDispositionField(entity);
- Optional<String> contentType = contentTypeField.map(ContentTypeField::getBody)
- .filter(string -> !string.isEmpty());
+ Optional<ContentType> contentType = contentTypeField.map(ContentTypeField::getBody)
+ .filter(string -> !string.isEmpty())
+ .map(ContentType::of);
Optional<String> name = name(contentTypeField, contentDispositionField);
Optional<Cid> cid = cid(readHeader(entity, CONTENT_ID, ContentIdField.class));
boolean isInline = isInline(readHeader(entity, CONTENT_DISPOSITION, ContentDispositionField.class)) && cid.isPresent();
diff --git a/mailbox/store/src/main/java/org/apache/james/mailbox/store/search/MessageSearches.java b/mailbox/store/src/main/java/org/apache/james/mailbox/store/search/MessageSearches.java
index 583ec25..690fcbf 100644
--- a/mailbox/store/src/main/java/org/apache/james/mailbox/store/search/MessageSearches.java
+++ b/mailbox/store/src/main/java/org/apache/james/mailbox/store/search/MessageSearches.java
@@ -267,7 +267,8 @@ public class MessageSearches implements Iterable<SimpleMessageSearchIndex.Search
return textExtractor
.extractContent(
rawData,
- attachment.getType())
+ // todo we likely want only the media type here
+ attachment.getType().asString())
.getTextualContent()
.stream();
} catch (Exception e) {
diff --git a/mailbox/store/src/test/java/org/apache/james/mailbox/store/StoreBlobManagerTest.java b/mailbox/store/src/test/java/org/apache/james/mailbox/store/StoreBlobManagerTest.java
index 3390947..49f929e 100644
--- a/mailbox/store/src/test/java/org/apache/james/mailbox/store/StoreBlobManagerTest.java
+++ b/mailbox/store/src/test/java/org/apache/james/mailbox/store/StoreBlobManagerTest.java
@@ -43,6 +43,7 @@ import org.apache.james.mailbox.model.AttachmentMetadata;
import org.apache.james.mailbox.model.Blob;
import org.apache.james.mailbox.model.BlobId;
import org.apache.james.mailbox.model.Content;
+import org.apache.james.mailbox.model.ContentType;
import org.apache.james.mailbox.model.FetchGroup;
import org.apache.james.mailbox.model.MessageResult;
import org.apache.james.mailbox.model.TestMessageId;
@@ -57,7 +58,7 @@ import com.google.common.collect.ImmutableList;
class StoreBlobManagerTest {
static final String ID = "abc";
static final AttachmentId ATTACHMENT_ID = AttachmentId.from(ID);
- static final String CONTENT_TYPE = "text/plain";
+ static final ContentType CONTENT_TYPE = ContentType.of("text/plain");
static final byte[] BYTES = "abc".getBytes(StandardCharsets.UTF_8);
static final TestMessageId MESSAGE_ID = TestMessageId.of(125);
static final BlobId BLOB_ID_ATTACHMENT = BlobId.fromString(ID);
diff --git a/mailbox/store/src/test/java/org/apache/james/mailbox/store/mail/model/AttachmentMapperTest.java b/mailbox/store/src/test/java/org/apache/james/mailbox/store/mail/model/AttachmentMapperTest.java
index 0bf9af7..89c9c9c 100644
--- a/mailbox/store/src/test/java/org/apache/james/mailbox/store/mail/model/AttachmentMapperTest.java
+++ b/mailbox/store/src/test/java/org/apache/james/mailbox/store/mail/model/AttachmentMapperTest.java
@@ -31,6 +31,7 @@ import org.apache.james.core.Username;
import org.apache.james.mailbox.exception.AttachmentNotFoundException;
import org.apache.james.mailbox.model.AttachmentId;
import org.apache.james.mailbox.model.AttachmentMetadata;
+import org.apache.james.mailbox.model.ContentType;
import org.apache.james.mailbox.model.MessageId;
import org.apache.james.mailbox.model.ParsedAttachment;
import org.apache.james.mailbox.store.mail.AttachmentMapper;
@@ -72,7 +73,7 @@ public abstract class AttachmentMapperTest {
@Test
void storeAttachmentForOwnerShouldReturnSuppliedInformation() throws Exception {
- String content = "content";
+ ContentType content = ContentType.of("content");
byte[] bytes = "payload".getBytes(StandardCharsets.UTF_8);
AttachmentMetadata stored = Mono.from(attachmentMapper.storeAttachmentForOwner(content, new ByteArrayInputStream(bytes), OWNER)).block();
@@ -85,7 +86,7 @@ public abstract class AttachmentMapperTest {
@Test
void getAttachmentShouldReturnTheAttachmentWhenReferenced() throws Exception {
- String content = "content";
+ ContentType content = ContentType.of("content");
byte[] bytes = "payload".getBytes(StandardCharsets.UTF_8);
AttachmentMetadata stored = Mono.from(attachmentMapper.storeAttachmentForOwner(content, new ByteArrayInputStream(bytes), OWNER)).block();
@@ -101,7 +102,7 @@ public abstract class AttachmentMapperTest {
@Test
void loadAttachmentContentShouldReturnStoredContent() throws Exception {
- String content = "content";
+ ContentType content = ContentType.of("content");
byte[] bytes = "payload".getBytes(StandardCharsets.UTF_8);
AttachmentMetadata stored = Mono.from(attachmentMapper.storeAttachmentForOwner(content, new ByteArrayInputStream(bytes), OWNER)).block();
@@ -126,10 +127,10 @@ public abstract class AttachmentMapperTest {
@Test
void getAttachmentsShouldReturnTheAttachmentsWhenSome() {
//Given
- String content1 = "content";
+ ContentType content1 = ContentType.of("content");
byte[] bytes1 = "payload".getBytes(StandardCharsets.UTF_8);
AttachmentMetadata stored1 = Mono.from(attachmentMapper.storeAttachmentForOwner(content1, new ByteArrayInputStream(bytes1), OWNER)).block();
- String content2 = "content";
+ ContentType content2 = ContentType.of("content");
byte[] bytes2 = "payload".getBytes(StandardCharsets.UTF_8);
AttachmentMetadata stored2 = Mono.from(attachmentMapper.storeAttachmentForOwner(content2, new ByteArrayInputStream(bytes2), OWNER)).block();
@@ -146,7 +147,7 @@ public abstract class AttachmentMapperTest {
@Test
void getOwnerMessageIdsShouldReturnEmptyWhenStoredWithoutMessageId() throws Exception {
- String content = "content";
+ ContentType content = ContentType.of("content");
byte[] bytes = "payload".getBytes(StandardCharsets.UTF_8);
AttachmentMetadata stored = Mono.from(attachmentMapper.storeAttachmentForOwner(content, new ByteArrayInputStream(bytes), OWNER)).block();
@@ -190,7 +191,7 @@ public abstract class AttachmentMapperTest {
@Test
void getOwnersShouldBeRetrievedWhenExplicitlySpecified() throws Exception {
- String content = "content";
+ ContentType content = ContentType.of("content");
byte[] bytes = "payload".getBytes(StandardCharsets.UTF_8);
AttachmentMetadata stored = Mono.from(attachmentMapper.storeAttachmentForOwner(content, new ByteArrayInputStream(bytes), OWNER)).block();
@@ -201,10 +202,10 @@ public abstract class AttachmentMapperTest {
@Test
void getOwnersShouldNotReturnUnrelatedOwners() throws Exception {
- String content = "content";
+ ContentType content = ContentType.of("content");
byte[] bytes = "payload".getBytes(StandardCharsets.UTF_8);
AttachmentMetadata stored = Mono.from(attachmentMapper.storeAttachmentForOwner(content, new ByteArrayInputStream(bytes), OWNER)).block();
- String content2 = "content";
+ ContentType content2 = ContentType.of("content");
byte[] bytes2 = "payload".getBytes(StandardCharsets.UTF_8);
AttachmentMetadata stored2 = Mono.from(attachmentMapper.storeAttachmentForOwner(content2, new ByteArrayInputStream(bytes2), ADDITIONAL_OWNER)).block();
diff --git a/mailbox/store/src/test/java/org/apache/james/mailbox/store/mail/model/impl/MessageParserTest.java b/mailbox/store/src/test/java/org/apache/james/mailbox/store/mail/model/impl/MessageParserTest.java
index 3f29cee..0bf210a 100644
--- a/mailbox/store/src/test/java/org/apache/james/mailbox/store/mail/model/impl/MessageParserTest.java
+++ b/mailbox/store/src/test/java/org/apache/james/mailbox/store/mail/model/impl/MessageParserTest.java
@@ -26,6 +26,7 @@ import java.util.List;
import java.util.Optional;
import org.apache.james.mailbox.model.Cid;
+import org.apache.james.mailbox.model.ContentType;
import org.apache.james.mailbox.model.ParsedAttachment;
import org.apache.james.mdn.MDN;
import org.apache.james.mdn.MDNReport;
@@ -90,7 +91,8 @@ class MessageParserTest {
List<ParsedAttachment> attachments = testee.retrieveAttachments(ClassLoader.getSystemResourceAsStream("eml/oneAttachmentWithoutContentType.eml"));
assertThat(attachments).hasSize(1);
- assertThat(attachments.get(0).getContentType()).isEqualTo("application/octet-stream");
+ assertThat(attachments.get(0).getContentType())
+ .isEqualTo(ContentType.of("application/octet-stream"));
}
@Test
@@ -98,7 +100,8 @@ class MessageParserTest {
List<ParsedAttachment> attachments = testee.retrieveAttachments(ClassLoader.getSystemResourceAsStream("eml/oneAttachmentWithEmptyContentType.eml"));
assertThat(attachments).hasSize(1);
- assertThat(attachments.get(0).getContentType()).isEqualTo("application/octet-stream");
+ assertThat(attachments.get(0).getContentType())
+ .isEqualTo(ContentType.of("application/octet-stream"));
}
@Test
@@ -106,7 +109,8 @@ class MessageParserTest {
List<ParsedAttachment> attachments = testee.retrieveAttachments(ClassLoader.getSystemResourceAsStream("eml/oneAttachmentAndSomeTextInlined.eml"));
assertThat(attachments).hasSize(1);
- assertThat(attachments.get(0).getContentType()).isEqualTo("application/octet-stream;\tname=\"exploits_of_a_mom.png\"");
+ assertThat(attachments.get(0).getContentType())
+ .isEqualTo(ContentType.of("application/octet-stream;\tname=\"exploits_of_a_mom.png\""));
}
@Test
@@ -128,7 +132,8 @@ class MessageParserTest {
List<ParsedAttachment> attachments = testee.retrieveAttachments(ClassLoader.getSystemResourceAsStream("eml/oneAttachmentWithSimpleContentType.eml"));
assertThat(attachments).hasSize(1);
- assertThat(attachments.get(0).getContentType()).isEqualTo("application/octet-stream");
+ assertThat(attachments.get(0).getContentType())
+ .isEqualTo(ContentType.of("application/octet-stream"));
}
@Test
@@ -270,7 +275,8 @@ class MessageParserTest {
assertThat(attachments).hasSize(1)
.first()
- .satisfies(attachment -> assertThat(attachment.getContentType()).isEqualTo("text/calendar; charset=\"iso-8859-1\"; method=COUNTER"));
+ .satisfies(attachment -> assertThat(attachment.getContentType())
+ .isEqualTo(ContentType.of("text/calendar; charset=\"iso-8859-1\"; method=COUNTER")));
}
@Test
@@ -280,8 +286,8 @@ class MessageParserTest {
assertThat(attachments).hasSize(2)
.extracting(ParsedAttachment::getContentType)
- .containsOnly("text/calendar; charset=\"iso-8859-1\"; method=COUNTER",
- "text/calendar; charset=\"iso-4444-5\"; method=COUNTER");
+ .containsOnly(ContentType.of("text/calendar; charset=\"iso-8859-1\"; method=COUNTER"),
+ ContentType.of("text/calendar; charset=\"iso-4444-5\"; method=COUNTER"));
}
@Test
@@ -292,7 +298,7 @@ class MessageParserTest {
assertThat(attachments)
.hasSize(1)
.extracting(ParsedAttachment::getContentType)
- .containsExactly("text/calendar; charset=\"utf-8\"; method=COUNTER");
+ .containsExactly(ContentType.of("text/calendar; charset=\"utf-8\"; method=COUNTER"));
}
@Test
@@ -328,6 +334,6 @@ class MessageParserTest {
List<ParsedAttachment> result = testee.retrieveAttachments(new ByteArrayInputStream(DefaultMessageWriter.asBytes(message)));
assertThat(result).hasSize(1)
- .allMatch(attachment -> attachment.getContentType().equals("message/disposition-notification; charset=UTF-8"));
+ .allMatch(attachment -> attachment.getContentType().equals(ContentType.of("message/disposition-notification; charset=UTF-8")));
}
}
diff --git a/server/protocols/jmap-draft-integration-testing/jmap-draft-integration-testing-common/src/test/java/org/apache/james/jmap/draft/methods/integration/SetMessagesMethodTest.java b/server/protocols/jmap-draft-integration-testing/jmap-draft-integration-testing-common/src/test/java/org/apache/james/jmap/draft/methods/integration/SetMessagesMethodTest.java
index f17efc6..b70eab1 100644
--- a/server/protocols/jmap-draft-integration-testing/jmap-draft-integration-testing-common/src/test/java/org/apache/james/jmap/draft/methods/integration/SetMessagesMethodTest.java
+++ b/server/protocols/jmap-draft-integration-testing/jmap-draft-integration-testing-common/src/test/java/org/apache/james/jmap/draft/methods/integration/SetMessagesMethodTest.java
@@ -2109,7 +2109,7 @@ public abstract class SetMessagesMethodTest {
" \"keywords\": {\"$Draft\": true}," +
" \"attachments\": [" +
" {\"blobId\" : \"" + uploadedAttachment.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment.getType() + "\"," +
+ " \"type\" : \"" + uploadedAttachment.getType().asString() + "\"," +
" \"size\" : " + uploadedAttachment.getSize() + "}" +
" ]," +
" \"mailboxIds\": [\"" + getDraftId(accessToken) + "\"]" +
@@ -3982,10 +3982,10 @@ public abstract class SetMessagesMethodTest {
" \"mailboxIds\": [\"" + outboxId + "\"], " +
" \"attachments\": [" +
" {\"blobId\" : \"" + uploadedAttachment1.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment1.getType() + "\", " +
+ " \"type\" : \"" + uploadedAttachment1.getType().asString() + "\", " +
" \"size\" : " + uploadedAttachment1.getSize() + "}," +
" {\"blobId\" : \"" + uploadedAttachment2.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment2.getType() + "\", " +
+ " \"type\" : \"" + uploadedAttachment2.getType().asString() + "\", " +
" \"size\" : " + uploadedAttachment2.getSize() + ", " +
" \"cid\" : \"123456789\", " +
" \"isInline\" : true }" +
@@ -4051,10 +4051,10 @@ public abstract class SetMessagesMethodTest {
" \"mailboxIds\": [\"" + outboxId + "\"], " +
" \"attachments\": [" +
" {\"blobId\" : \"" + uploadedAttachment1.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment1.getType() + "\", " +
+ " \"type\" : \"" + uploadedAttachment1.getType().asString() + "\", " +
" \"size\" : " + uploadedAttachment1.getSize() + "}," +
" {\"blobId\" : \"" + uploadedAttachment2.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment2.getType() + "\", " +
+ " \"type\" : \"" + uploadedAttachment2.getType().asString() + "\", " +
" \"size\" : " + uploadedAttachment2.getSize() + ", " +
" \"cid\" : \"123456789\", " +
" \"isInline\" : true }" +
@@ -4127,21 +4127,21 @@ public abstract class SetMessagesMethodTest {
" [" +
" {" +
" \"blobId\" : \"" + uploadedAttachment1.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment1.getType() + "\", " +
+ " \"type\" : \"" + uploadedAttachment1.getType().asString() + "\", " +
" \"size\" : " + uploadedAttachment1.getSize() + "," +
" \"name\" : \"ديناصور.png\", " +
" \"isInline\" : false" +
" }," +
" {" +
" \"blobId\" : \"" + uploadedAttachment2.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment2.getType() + "\", " +
+ " \"type\" : \"" + uploadedAttachment2.getType().asString() + "\", " +
" \"size\" : " + uploadedAttachment2.getSize() + "," +
" \"name\" : \"эволюционировать.png\", " +
" \"isInline\" : false" +
" }," +
" {" +
" \"blobId\" : \"" + uploadedAttachment3.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment3.getType() + "\", " +
+ " \"type\" : \"" + uploadedAttachment3.getType().asString() + "\", " +
" \"size\" : " + uploadedAttachment3.getSize() + "," +
" \"name\" : \"进化还是不.png\"," +
" \"isInline\" : false" +
@@ -4204,21 +4204,21 @@ public abstract class SetMessagesMethodTest {
" [" +
" {" +
" \"blobId\" : \"" + uploadedAttachment1.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment1.getType() + "\", " +
+ " \"type\" : \"" + uploadedAttachment1.getType().asString() + "\", " +
" \"size\" : " + uploadedAttachment1.getSize() + "," +
" \"name\" : \"ديناصور.png\", " +
" \"isInline\" : false" +
" }," +
" {" +
" \"blobId\" : \"" + uploadedAttachment2.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment2.getType() + "\", " +
+ " \"type\" : \"" + uploadedAttachment2.getType().asString() + "\", " +
" \"size\" : " + uploadedAttachment2.getSize() + "," +
" \"name\" : \"эволюционировать.png\", " +
" \"isInline\" : false" +
" }," +
" {" +
" \"blobId\" : \"" + uploadedAttachment3.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment3.getType() + "\", " +
+ " \"type\" : \"" + uploadedAttachment3.getType().asString() + "\", " +
" \"size\" : " + uploadedAttachment3.getSize() + "," +
" \"name\" : \"进化还是不.png\"," +
" \"isInline\" : false" +
@@ -4332,7 +4332,7 @@ public abstract class SetMessagesMethodTest {
" \"mailboxIds\": [\"" + outboxId + "\"], " +
" \"attachments\": [" +
" {\"blobId\" : \"" + uploadedAttachment.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment.getType() + "\", " +
+ " \"type\" : \"" + uploadedAttachment.getType().asString() + "\", " +
" \"size\" : " + uploadedAttachment.getSize() + ", " +
" \"cid\" : \"123456789\", " +
" \"isInline\" : true }" +
@@ -4404,7 +4404,7 @@ public abstract class SetMessagesMethodTest {
" \"mailboxIds\": [\"" + outboxId + "\"], " +
" \"attachments\": [" +
" {\"blobId\" : \"" + uploadedAttachment.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment.getType() + "\", " +
+ " \"type\" : \"" + uploadedAttachment.getType().asString() + "\", " +
" \"size\" : " + uploadedAttachment.getSize() + ", " +
" \"cid\" : \"123456789\", " +
" \"isInline\" : true }" +
@@ -4499,7 +4499,7 @@ public abstract class SetMessagesMethodTest {
" \"mailboxIds\": [\"" + outboxId + "\"], " +
" \"attachments\": [" +
" {\"blobId\" : \"" + uploadedAttachment.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment.getType() + "\", " +
+ " \"type\" : \"" + uploadedAttachment.getType().asString() + "\", " +
" \"size\" : " + uploadedAttachment.getSize() + ", " +
" \"isInline\" : false }" +
" ]" +
@@ -4574,7 +4574,7 @@ public abstract class SetMessagesMethodTest {
" \"mailboxIds\": [\"" + outboxId + "\"], " +
" \"attachments\": [" +
" {\"blobId\" : \"" + uploadedAttachment.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment.getType() + "\", " +
+ " \"type\" : \"" + uploadedAttachment.getType().asString() + "\", " +
" \"size\" : " + uploadedAttachment.getSize() + ", " +
" \"isInline\" : false }" +
" ]" +
@@ -4659,7 +4659,7 @@ public abstract class SetMessagesMethodTest {
" \"mailboxIds\": [\"" + outboxId + "\"], " +
" \"attachments\": [" +
" {\"blobId\" : \"" + uploadedAttachment.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment.getType() + "\", " +
+ " \"type\" : \"" + uploadedAttachment.getType().asString() + "\", " +
" \"size\" : " + uploadedAttachment.getSize() + ", " +
" \"isInline\" : false }" +
" ]" +
@@ -5420,7 +5420,7 @@ public abstract class SetMessagesMethodTest {
" \"mailboxIds\": [\"" + outboxId + "\"], " +
" \"attachments\": [" +
" {\"blobId\" : \"" + uploadedAttachment.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment.getType() + "\", " +
+ " \"type\" : \"" + uploadedAttachment.getType().asString() + "\", " +
" \"name\" : \"nonIndexableAttachment.html\", " +
" \"size\" : " + uploadedAttachment.getSize() + "}" +
" ]" +
@@ -5469,7 +5469,7 @@ public abstract class SetMessagesMethodTest {
" \"mailboxIds\": [\"" + outboxId + "\"], " +
" \"attachments\": [" +
" {\"blobId\" : \"" + uploadedAttachment.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment.getType() + "\", " +
+ " \"type\" : \"" + uploadedAttachment.getType().asString() + "\", " +
" \"name\" : \"nonIndexableAttachment.html\", " +
" \"size\" : " + uploadedAttachment.getSize() + "}" +
" ]" +
@@ -5528,7 +5528,7 @@ public abstract class SetMessagesMethodTest {
" \"mailboxIds\": [\"" + outboxId + "\"], " +
" \"attachments\": [" +
" {\"blobId\" : \"" + uploadedAttachment.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment.getType() + "\", " +
+ " \"type\" : \"" + uploadedAttachment.getType().asString() + "\", " +
" \"name\" : \"nonIndexableAttachment.html\", " +
" \"size\" : " + uploadedAttachment.getSize() + "}" +
" ]" +
@@ -5582,10 +5582,10 @@ public abstract class SetMessagesMethodTest {
" \"mailboxIds\": [\"" + outboxId + "\"], " +
" \"attachments\": [" +
" {\"blobId\" : \"" + uploadedAttachment1.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment1.getType() + "\", " +
+ " \"type\" : \"" + uploadedAttachment1.getType().asString() + "\", " +
" \"size\" : " + uploadedAttachment1.getSize() + "}," +
" {\"blobId\" : \"" + uploadedAttachment2.getAttachmentId().getId() + "\", " +
- " \"type\" : \"" + uploadedAttachment2.getType() + "\", " +
+ " \"type\" : \"" + uploadedAttachment2.getType().asString() + "\", " +
" \"size\" : " + uploadedAttachment2.getSize() + ", " +
" \"isInline\" : true }" +
" ]" +
diff --git a/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/draft/json/ObjectMapperFactory.java b/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/draft/json/ObjectMapperFactory.java
index 345a80b..cb923f5 100644
--- a/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/draft/json/ObjectMapperFactory.java
+++ b/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/draft/json/ObjectMapperFactory.java
@@ -28,6 +28,7 @@ import javax.inject.Inject;
import org.apache.james.core.Username;
import org.apache.james.jmap.draft.model.mailbox.Rights;
import org.apache.james.mailbox.Role;
+import org.apache.james.mailbox.model.ContentType;
import org.apache.james.mailbox.model.MailboxId;
import org.apache.james.mailbox.model.MessageId;
import org.apache.james.mdn.action.mode.DispositionActionMode;
@@ -87,10 +88,15 @@ public class ObjectMapperFactory {
mailboxIdModule.addDeserializer(DispositionSendingMode.class, new MDNSendingModeDeserializer());
mailboxIdModule.addDeserializer(DispositionType.class, new MDNTypeDeserializer());
+ SimpleModule contentTypeModule = new SimpleModule();
+ contentTypeModule.addDeserializer(ContentType.class, new ContentTypeDeserializer());
+ contentTypeModule.addSerializer(ContentType.class, new ContentTypeSerializer());
+
mailboxIdModule.setMixInAnnotation(Role.class, RoleMixIn.class);
jacksonModules = JACKSON_BASE_MODULES.add(mailboxIdModule)
.add(mdnModule)
+ .add(contentTypeModule)
.build();
}
@@ -134,6 +140,21 @@ public class ObjectMapperFactory {
}
}
+ public static class ContentTypeDeserializer extends JsonDeserializer<ContentType> {
+ @Override
+ public ContentType deserialize(JsonParser jsonParser, DeserializationContext deserializationContext) throws IOException, JsonProcessingException {
+ String value = jsonParser.getValueAsString();
+ return ContentType.of(value);
+ }
+ }
+
+ public static class ContentTypeSerializer extends JsonSerializer<ContentType> {
+ @Override
+ public void serialize(ContentType value, JsonGenerator gen, SerializerProvider serializers) throws IOException, JsonProcessingException {
+ gen.writeString(value.asString());
+ }
+ }
+
public static class MDNTypeDeserializer extends JsonDeserializer<DispositionType> {
private static final ImmutableList<String> ALLOWED_VALUES = Arrays.stream(DispositionType.values())
.map(DispositionType::getValue)
diff --git a/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/draft/methods/MIMEMessageConverter.java b/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/draft/methods/MIMEMessageConverter.java
index 4864f05..a7c9dc7 100644
--- a/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/draft/methods/MIMEMessageConverter.java
+++ b/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/draft/methods/MIMEMessageConverter.java
@@ -328,7 +328,8 @@ public class MIMEMessageConverter {
}
private ContentTypeField contentTypeField(MessageAttachmentMetadata att) {
- String type = att.getAttachment().getType();
+ // todo mailbox pojo should be able to expose itself as a mime4j object
+ String type = att.getAttachment().getType().asString();
ContentTypeField typeAsField = Fields.contentType(type);
if (att.getName().isPresent()) {
return Fields.contentType(typeAsField.getMimeType(),
diff --git a/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/draft/model/Attachment.java b/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/draft/model/Attachment.java
index 271e442..1ea228f 100644
--- a/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/draft/model/Attachment.java
+++ b/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/draft/model/Attachment.java
@@ -22,12 +22,13 @@ package org.apache.james.jmap.draft.model;
import java.util.Objects;
import java.util.Optional;
+import org.apache.james.mailbox.model.ContentType;
+
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.annotation.JsonPOJOBuilder;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.MoreObjects;
import com.google.common.base.Preconditions;
-import com.google.common.base.Strings;
@JsonDeserialize(builder = Attachment.Builder.class)
public class Attachment {
@@ -39,7 +40,7 @@ public class Attachment {
@JsonPOJOBuilder(withPrefix = "")
public static class Builder {
private BlobId blobId;
- private String type;
+ private ContentType type;
private String name;
private Number size;
private String cid;
@@ -52,7 +53,13 @@ public class Attachment {
return this;
}
+ @JsonDeserialize
public Builder type(String type) {
+ this.type = ContentType.of(type);
+ return this;
+ }
+
+ public Builder type(ContentType type) {
this.type = type;
return this;
}
@@ -104,14 +111,13 @@ public class Attachment {
public Attachment build() {
Preconditions.checkState(blobId != null, "'blobId' is mandatory");
- Preconditions.checkState(!Strings.isNullOrEmpty(type), "'type' is mandatory");
Preconditions.checkState(size != null, "'size' is mandatory");
return new Attachment(blobId, type, Optional.ofNullable(name), size, Optional.ofNullable(cid), isInline, Optional.ofNullable(width), Optional.ofNullable(height));
}
}
private final BlobId blobId;
- private final String type;
+ private final ContentType type;
private final Optional<String> name;
private final Number size;
private final Optional<String> cid;
@@ -119,7 +125,7 @@ public class Attachment {
private final Optional<Number> width;
private final Optional<Number> height;
- @VisibleForTesting Attachment(BlobId blobId, String type, Optional<String> name, Number size, Optional<String> cid, boolean isInline, Optional<Number> width, Optional<Number> height) {
+ @VisibleForTesting Attachment(BlobId blobId, ContentType type, Optional<String> name, Number size, Optional<String> cid, boolean isInline, Optional<Number> width, Optional<Number> height) {
this.blobId = blobId;
this.type = type;
this.name = name;
@@ -134,7 +140,7 @@ public class Attachment {
return blobId;
}
- public String getType() {
+ public ContentType getType() {
return type;
}
diff --git a/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/http/DownloadRoutes.java b/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/http/DownloadRoutes.java
index ddabcf7..7eb041a 100644
--- a/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/http/DownloadRoutes.java
+++ b/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/http/DownloadRoutes.java
@@ -52,6 +52,7 @@ import org.apache.james.mailbox.exception.BlobNotFoundException;
import org.apache.james.mailbox.exception.MailboxException;
import org.apache.james.mailbox.model.Blob;
import org.apache.james.mailbox.model.BlobId;
+import org.apache.james.mailbox.model.ContentType;
import org.apache.james.metrics.api.MetricFactory;
import org.apache.james.mime4j.codec.EncoderUtil;
import org.apache.james.mime4j.codec.EncoderUtil.Usage;
@@ -222,10 +223,10 @@ public class DownloadRoutes implements JMAPRoutes {
}
}
- private Mono<Void> downloadBlob(Optional<String> optionalName, HttpServerResponse response, long blobSize, String blobContentType, InputStream stream) {
+ private Mono<Void> downloadBlob(Optional<String> optionalName, HttpServerResponse response, long blobSize, ContentType blobContentType, InputStream stream) {
return addContentDispositionHeader(optionalName, response)
.header("Content-Length", String.valueOf(blobSize))
- .header(CONTENT_TYPE, blobContentType)
+ .header(CONTENT_TYPE, blobContentType.asString())
.status(OK)
.send(ReactorUtils.toChunks(stream, BUFFER_SIZE)
.map(Unpooled::wrappedBuffer)
diff --git a/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/http/UploadRoutes.java b/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/http/UploadRoutes.java
index 95e6d62..cc530b0 100644
--- a/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/http/UploadRoutes.java
+++ b/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/http/UploadRoutes.java
@@ -43,6 +43,7 @@ import org.apache.james.jmap.draft.model.UploadResponse;
import org.apache.james.jmap.exceptions.UnauthorizedException;
import org.apache.james.mailbox.AttachmentManager;
import org.apache.james.mailbox.MailboxSession;
+import org.apache.james.mailbox.model.ContentType;
import org.apache.james.metrics.api.MetricFactory;
import org.apache.james.util.ReactorUtils;
import org.slf4j.Logger;
@@ -98,7 +99,7 @@ public class UploadRoutes implements JMAPRoutes {
return response.status(BAD_REQUEST).send();
} else {
return authenticator.authenticate(request)
- .flatMap(session -> post(request, response, contentType, session)
+ .flatMap(session -> post(request, response, ContentType.of(contentType), session)
.subscriberContext(jmapAuthContext(session)))
.onErrorResume(CancelledUploadException.class, e -> handleCanceledUpload(response, e))
.onErrorResume(BadRequestException.class, e -> handleBadRequest(response, e))
@@ -111,13 +112,13 @@ public class UploadRoutes implements JMAPRoutes {
}
}
- private Mono<Void> post(HttpServerRequest request, HttpServerResponse response, String contentType, MailboxSession session) {
+ private Mono<Void> post(HttpServerRequest request, HttpServerResponse response, ContentType contentType, MailboxSession session) {
InputStream content = ReactorUtils.toInputStream(request.receive().asByteBuffer().subscribeOn(Schedulers.elastic()));
return Mono.from(metricFactory.runPublishingTimerMetric("JMAP-upload-post",
handle(contentType, content, session, response)));
}
- private Mono<Void> handle(String contentType, InputStream content, MailboxSession mailboxSession, HttpServerResponse response) {
+ private Mono<Void> handle(ContentType contentType, InputStream content, MailboxSession mailboxSession, HttpServerResponse response) {
return uploadContent(contentType, content, mailboxSession)
.flatMap(storedContent -> {
try {
@@ -131,11 +132,11 @@ public class UploadRoutes implements JMAPRoutes {
});
}
- private Mono<UploadResponse> uploadContent(String contentType, InputStream inputStream, MailboxSession session) {
+ private Mono<UploadResponse> uploadContent(ContentType contentType, InputStream inputStream, MailboxSession session) {
return Mono.from(attachmentManager.storeAttachment(contentType, inputStream, session))
.map(attachment -> UploadResponse.builder()
.blobId(attachment.getAttachmentId().getId())
- .type(attachment.getType())
+ .type(attachment.getType().asString())
.size(attachment.getSize())
.build())
.onErrorMap(e -> e.getCause() instanceof EOFException, any -> new CancelledUploadException())
diff --git a/server/protocols/jmap-draft/src/test/java/org/apache/james/jmap/draft/model/AttachmentTest.java b/server/protocols/jmap-draft/src/test/java/org/apache/james/jmap/draft/model/AttachmentTest.java
index 9e7e122..b81c986 100644
--- a/server/protocols/jmap-draft/src/test/java/org/apache/james/jmap/draft/model/AttachmentTest.java
+++ b/server/protocols/jmap-draft/src/test/java/org/apache/james/jmap/draft/model/AttachmentTest.java
@@ -22,6 +22,7 @@ import static org.assertj.core.api.Assertions.assertThat;
import java.util.Optional;
+import org.apache.james.mailbox.model.ContentType;
import org.junit.Test;
public class AttachmentTest {
@@ -46,7 +47,7 @@ public class AttachmentTest {
Attachment.builder().blobId(BlobId.of("blobId")).type("type").name("name").build();
}
- @Test(expected = IllegalStateException.class)
+ @Test(expected = IllegalArgumentException.class)
public void buildShouldThrowWhenTypeIsEmpty() {
Attachment.builder().blobId(BlobId.of("blobId")).type("").name("name").size(123).build();
}
@@ -54,7 +55,7 @@ public class AttachmentTest {
@Test
public void buildShouldWorkWhenMandatoryFieldsArePresent() {
Number attachmentSize = Number.fromLong(123);
- Attachment expected = new Attachment(BlobId.of("blobId"), "type", Optional.empty(), attachmentSize, Optional.empty(), false, Optional.empty(), Optional.empty());
+ Attachment expected = new Attachment(BlobId.of("blobId"), ContentType.of("type"), Optional.empty(), attachmentSize, Optional.empty(), false, Optional.empty(), Optional.empty());
Attachment tested = Attachment.builder()
.blobId(BlobId.of("blobId"))
.type("type")
@@ -68,7 +69,7 @@ public class AttachmentTest {
Number attachmentSize = Number.fromLong(123);
Optional<Number> attachmentWidth = Optional.of(Number.fromLong(456L));
Optional<Number> attachmentHeight = Optional.of(Number.fromLong(789L));
- Attachment expected = new Attachment(BlobId.of("blobId"), "type", Optional.of("name"), attachmentSize, Optional.of("cid"), true,
+ Attachment expected = new Attachment(BlobId.of("blobId"), ContentType.of("type"), Optional.of("name"), attachmentSize, Optional.of("cid"), true,
attachmentWidth, attachmentHeight);
Attachment tested = Attachment.builder()
.blobId(BlobId.of("blobId"))
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[james-project] 04/12: JAMES-2997 JSoupTextExtractor need to handle
correctly charset
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit cb51ca8b6781d35b65b834141ce9f8f6fef5ce08
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Mon Apr 27 19:37:22 2020 +0700
JAMES-2997 JSoupTextExtractor need to handle correctly charset
---
.../james/mailbox/store/extractor/JsoupTextExtractor.java | 14 ++++++++------
1 file changed, 8 insertions(+), 6 deletions(-)
diff --git a/mailbox/store/src/main/java/org/apache/james/mailbox/store/extractor/JsoupTextExtractor.java b/mailbox/store/src/main/java/org/apache/james/mailbox/store/extractor/JsoupTextExtractor.java
index 5404455..45378ae 100644
--- a/mailbox/store/src/main/java/org/apache/james/mailbox/store/extractor/JsoupTextExtractor.java
+++ b/mailbox/store/src/main/java/org/apache/james/mailbox/store/extractor/JsoupTextExtractor.java
@@ -21,6 +21,7 @@ package org.apache.james.mailbox.store.extractor;
import java.io.IOException;
import java.io.InputStream;
+import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import java.util.List;
import java.util.Map;
@@ -48,21 +49,22 @@ public class JsoupTextExtractor implements TextExtractor {
if (inputStream == null || contentType == null) {
return ParsedContent.empty();
}
+ Charset charset = contentType.charset().orElse(StandardCharsets.UTF_8);
if (contentType.mimeType().equals(TEXT_HTML)) {
- return parseHtmlContent(inputStream);
+ return parseHtmlContent(inputStream, charset);
}
if (contentType.mimeType().equals(TEXT_PLAIN)) {
- return parsePlainTextContent(inputStream);
+ return parsePlainTextContent(inputStream, charset);
}
return ParsedContent.empty();
}
- private ParsedContent parsePlainTextContent(InputStream inputStream) throws IOException {
- return new ParsedContent(Optional.ofNullable(IOUtils.toString(inputStream, StandardCharsets.UTF_8)), EMPTY_METADATA);
+ private ParsedContent parsePlainTextContent(InputStream inputStream, Charset charset) throws IOException {
+ return new ParsedContent(Optional.ofNullable(IOUtils.toString(inputStream, charset)), EMPTY_METADATA);
}
- private ParsedContent parseHtmlContent(InputStream inputStream) throws IOException {
- Document doc = Jsoup.parse(inputStream, StandardCharsets.UTF_8.name(), NO_BASE_URI);
+ private ParsedContent parseHtmlContent(InputStream inputStream, Charset charset) throws IOException {
+ Document doc = Jsoup.parse(inputStream, charset.name(), NO_BASE_URI);
doc.select(TITLE_HTML_TAG).remove();
return new ParsedContent(Optional.ofNullable(doc.text()), EMPTY_METADATA);
}
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[james-project] 01/12: JAMES-2997 JMAP attachments should use
java.util.Objects
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 0b47fd77990b3c27da3cbf88f9c016bb2ec6e7fb
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Mon Apr 27 15:10:38 2020 +0700
JAMES-2997 JMAP attachments should use java.util.Objects
---
.../apache/james/jmap/draft/model/Attachment.java | 20 ++++++++++----------
1 file changed, 10 insertions(+), 10 deletions(-)
diff --git a/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/draft/model/Attachment.java b/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/draft/model/Attachment.java
index 045b90e..271e442 100644
--- a/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/draft/model/Attachment.java
+++ b/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/draft/model/Attachment.java
@@ -19,13 +19,13 @@
package org.apache.james.jmap.draft.model;
+import java.util.Objects;
import java.util.Optional;
import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.fasterxml.jackson.databind.annotation.JsonPOJOBuilder;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.MoreObjects;
-import com.google.common.base.Objects;
import com.google.common.base.Preconditions;
import com.google.common.base.Strings;
@@ -170,21 +170,21 @@ public class Attachment {
public boolean equals(Object obj) {
if (obj instanceof Attachment) {
Attachment other = (Attachment) obj;
- return Objects.equal(blobId, other.blobId)
- && Objects.equal(type, other.type)
- && Objects.equal(name, other.name)
- && Objects.equal(size, other.size)
- && Objects.equal(cid, other.cid)
- && Objects.equal(isInline, other.isInline)
- && Objects.equal(width, other.width)
- && Objects.equal(height, other.height);
+ return Objects.equals(blobId, other.blobId)
+ && Objects.equals(type, other.type)
+ && Objects.equals(name, other.name)
+ && Objects.equals(size, other.size)
+ && Objects.equals(cid, other.cid)
+ && Objects.equals(isInline, other.isInline)
+ && Objects.equals(width, other.width)
+ && Objects.equals(height, other.height);
}
return false;
}
@Override
public int hashCode() {
- return Objects.hashCode(blobId, type, name, size, cid, isInline, width, height);
+ return Objects.hash(blobId, type, name, size, cid, isInline, width, height);
}
@Override
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[james-project] 09/12: JAMES-3105 Add an option to trust message
denormalization when recomputing counters
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 4501b921168c7f9b1048f8b1e33a2d64d059aa2e
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Thu Apr 23 17:16:28 2020 +0700
JAMES-3105 Add an option to trust message denormalization when recomputing counters
---
.../mail/task/RecomputeMailboxCountersService.java | 39 +-
.../mail/task/RecomputeMailboxCountersTask.java | 10 +-
.../mail/task/RecomputeMailboxCountersTaskDTO.java | 19 +-
.../task/RecomputeMailboxCountersServiceTest.java | 438 +++++++++++++--------
...omputeMailboxCountersTaskSerializationTest.java | 46 ++-
.../RecomputeMailboxCountersRequestToTask.java | 3 +-
6 files changed, 365 insertions(+), 190 deletions(-)
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java
index e14d16e..b39db27 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java
@@ -55,6 +55,30 @@ public class RecomputeMailboxCountersService {
private static final int MAILBOX_CONCURRENCY = 2;
private static final int MESSAGE_CONCURRENCY = 8;
+ public static class Options {
+ public static Options trustMessageDenormalization() {
+ return of(true);
+ }
+
+ public static Options recheckMessageDenormalization() {
+ return of(false);
+ }
+
+ public static Options of(boolean value) {
+ return new Options(value);
+ }
+
+ private final boolean trustMessageDenormalization;
+
+ private Options(boolean trustMessageDenormalization) {
+ this.trustMessageDenormalization = trustMessageDenormalization;
+ }
+
+ public boolean isMessageDenormalizationTrusted() {
+ return trustMessageDenormalization;
+ }
+ }
+
private static class Counter {
private final CassandraId mailboxId;
private final AtomicLong total;
@@ -163,9 +187,9 @@ public class RecomputeMailboxCountersService {
this.counterDAO = counterDAO;
}
- Mono<Result> recomputeMailboxCounters(Context context) {
+ Mono<Result> recomputeMailboxCounters(Context context, Options options) {
return mailboxDAO.retrieveAllMailboxes()
- .flatMap(mailbox -> recomputeMailboxCounter(context, mailbox), MAILBOX_CONCURRENCY)
+ .flatMap(mailbox -> recomputeMailboxCounter(context, mailbox, options), MAILBOX_CONCURRENCY)
.reduce(Result.COMPLETED, Task::combine)
.onErrorResume(e -> {
LOGGER.error("Error listing mailboxes", e);
@@ -173,12 +197,12 @@ public class RecomputeMailboxCountersService {
});
}
- private Mono<Result> recomputeMailboxCounter(Context context, Mailbox mailbox) {
+ private Mono<Result> recomputeMailboxCounter(Context context, Mailbox mailbox, Options options) {
CassandraId mailboxId = (CassandraId) mailbox.getMailboxId();
Counter counter = new Counter(mailboxId);
return imapUidToMessageIdDAO.retrieveMessages(mailboxId, MessageRange.all())
- .flatMap(message -> latestMetadata(mailboxId, message), MESSAGE_CONCURRENCY)
+ .flatMap(message -> latestMetadata(mailboxId, message, options), MESSAGE_CONCURRENCY)
.doOnNext(counter::process)
.then(Mono.defer(() -> counterDAO.resetCounters(counter.snapshot())))
.then(Mono.just(Result.COMPLETED))
@@ -193,7 +217,12 @@ public class RecomputeMailboxCountersService {
});
}
- private Flux<ComposedMessageIdWithMetaData> latestMetadata(CassandraId mailboxId, ComposedMessageIdWithMetaData message) {
+ private Flux<ComposedMessageIdWithMetaData> latestMetadata(CassandraId mailboxId,
+ ComposedMessageIdWithMetaData message,
+ Options options) {
+ if (options.isMessageDenormalizationTrusted()) {
+ return Flux.just(message);
+ }
CassandraMessageId messageId = (CassandraMessageId) message.getComposedMessageId().getMessageId();
return messageIdToImapUidDAO.retrieve(messageId, Optional.of(mailboxId))
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTask.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTask.java
index 74082f3..c93e1ea 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTask.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTask.java
@@ -70,16 +70,18 @@ public class RecomputeMailboxCountersTask implements Task {
}
private final RecomputeMailboxCountersService service;
+ private final RecomputeMailboxCountersService.Options options;
private RecomputeMailboxCountersService.Context context;
- public RecomputeMailboxCountersTask(RecomputeMailboxCountersService service) {
+ public RecomputeMailboxCountersTask(RecomputeMailboxCountersService service, RecomputeMailboxCountersService.Options options) {
this.service = service;
+ this.options = options;
this.context = new RecomputeMailboxCountersService.Context();
}
@Override
public Result run() {
- return service.recomputeMailboxCounters(context)
+ return service.recomputeMailboxCounters(context, options)
.subscribeOn(Schedulers.elastic())
.block();
}
@@ -89,6 +91,10 @@ public class RecomputeMailboxCountersTask implements Task {
return RECOMPUTE_MAILBOX_COUNTERS;
}
+ public RecomputeMailboxCountersService.Options getOptions() {
+ return options;
+ }
+
@Override
public Optional<TaskExecutionDetails.AdditionalInformation> details() {
Snapshot snapshot = context.snapshot();
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTaskDTO.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTaskDTO.java
index b8fa112..d855aa1 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTaskDTO.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTaskDTO.java
@@ -18,7 +18,10 @@
****************************************************************/
package org.apache.james.mailbox.cassandra.mail.task;
+import java.util.Optional;
+
import org.apache.james.json.DTOModule;
+import org.apache.james.mailbox.cassandra.mail.task.RecomputeMailboxCountersService.Options;
import org.apache.james.server.task.json.dto.TaskDTO;
import org.apache.james.server.task.json.dto.TaskDTOModule;
@@ -26,7 +29,7 @@ import com.fasterxml.jackson.annotation.JsonProperty;
public class RecomputeMailboxCountersTaskDTO implements TaskDTO {
private static RecomputeMailboxCountersTaskDTO toDTO(RecomputeMailboxCountersTask domainObject, String typeName) {
- return new RecomputeMailboxCountersTaskDTO(typeName);
+ return new RecomputeMailboxCountersTaskDTO(typeName, Optional.of(domainObject.getOptions().isMessageDenormalizationTrusted()));
}
public static TaskDTOModule<RecomputeMailboxCountersTask, RecomputeMailboxCountersTaskDTO> module(RecomputeMailboxCountersService service) {
@@ -40,17 +43,27 @@ public class RecomputeMailboxCountersTaskDTO implements TaskDTO {
}
private final String type;
+ private final Optional<Boolean> trustMessageDenormalization;
- public RecomputeMailboxCountersTaskDTO(@JsonProperty("type") String type) {
+ public RecomputeMailboxCountersTaskDTO(@JsonProperty("type") String type,
+ @JsonProperty("trustMessageDenormalization") Optional<Boolean> trustMessageDenormalization) {
this.type = type;
+ this.trustMessageDenormalization = trustMessageDenormalization;
}
private RecomputeMailboxCountersTask toDomainObject(RecomputeMailboxCountersService service) {
- return new RecomputeMailboxCountersTask(service);
+ Options options = trustMessageDenormalization.map(Options::of).orElse(Options.recheckMessageDenormalization());
+ return new RecomputeMailboxCountersTask(service, options);
}
@Override
+ @JsonProperty("type")
public String getType() {
return type;
}
+
+ @JsonProperty("trustMessageDenormalization")
+ public Optional<Boolean> trustMessageDenormalization() {
+ return trustMessageDenormalization;
+ }
}
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersServiceTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersServiceTest.java
index 683e1f7..10382a8 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersServiceTest.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersServiceTest.java
@@ -51,6 +51,8 @@ import org.apache.james.mailbox.model.MailboxPath;
import org.apache.james.mailbox.model.UidValidity;
import org.apache.james.task.Task.Result;
import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Disabled;
+import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
@@ -89,199 +91,297 @@ class RecomputeMailboxCountersServiceTest {
testee = new RecomputeMailboxCountersService(mailboxDAO, imapUidToMessageIdDAO, messageIdToImapUidDAO, counterDAO);
}
- @Test
- void recomputeMailboxCountersShouldReturnCompletedWhenNoMailboxes() {
- assertThat(testee.recomputeMailboxCounters(new Context()).block())
- .isEqualTo(Result.COMPLETED);
- }
-
- @Test
- void recomputeMailboxCountersShouldReturnCompletedWhenMailboxWithNoMessages() {
- mailboxDAO.save(MAILBOX).block();
-
- assertThat(testee.recomputeMailboxCounters(new Context()).block())
- .isEqualTo(Result.COMPLETED);
- }
-
- @Test
- void recomputeMailboxCountersShouldReturnCompletedWhenMailboxWithMessages() {
- mailboxDAO.save(MAILBOX).block();
- imapUidToMessageIdDAO.insert(METADATA_UNSEEN).block();
- messageIdToImapUidDAO.insert(METADATA_UNSEEN).block();
- counterDAO.incrementUnseenAndCount(CASSANDRA_ID_1).block();
- counterDAO.incrementCount(CASSANDRA_ID_1).block();
-
- testee.recomputeMailboxCounters(new Context()).block();
-
- assertThat(testee.recomputeMailboxCounters(new Context()).block())
- .isEqualTo(Result.COMPLETED);
- }
-
- @Test
- void recomputeMailboxCountersShouldReturnCompletedWhenMessageDenormalizationIssue() {
- mailboxDAO.save(MAILBOX).block();
- imapUidToMessageIdDAO.insert(METADATA_UNSEEN).block();
- messageIdToImapUidDAO.insert(METADATA_SEEN).block();
- counterDAO.incrementUnseenAndCount(CASSANDRA_ID_1).block();
- counterDAO.incrementCount(CASSANDRA_ID_1).block();
-
- testee.recomputeMailboxCounters(new Context()).block();
-
- assertThat(testee.recomputeMailboxCounters(new Context()).block())
- .isEqualTo(Result.COMPLETED);
- }
+ @Nested
+ class TrustMessageDenormalizationTest implements Contract {
+ @Override
+ public RecomputeMailboxCountersService testee() {
+ return testee;
+ }
- @Test
- void recomputeMailboxCountersShouldReturnCountersAreIncorrect() {
- mailboxDAO.save(MAILBOX).block();
- imapUidToMessageIdDAO.insert(METADATA_UNSEEN).block();
- messageIdToImapUidDAO.insert(METADATA_UNSEEN).block();
+ @Override
+ public CassandraMailboxDAO mailboxDAO() {
+ return mailboxDAO;
+ }
- testee.recomputeMailboxCounters(new Context()).block();
+ @Override
+ public RecomputeMailboxCountersService.Options options() {
+ return RecomputeMailboxCountersService.Options.trustMessageDenormalization();
+ }
- assertThat(testee.recomputeMailboxCounters(new Context()).block())
- .isEqualTo(Result.COMPLETED);
- }
-
- @Test
- void recomputeMailboxCountersShouldReturnCompletedWhenOrphanMailboxRegistration() {
- mailboxDAO.save(MAILBOX).block();
- imapUidToMessageIdDAO.insert(METADATA_UNSEEN).block();
- counterDAO.incrementUnseenAndCount(CASSANDRA_ID_1).block();
- counterDAO.incrementCount(CASSANDRA_ID_1).block();
-
- testee.recomputeMailboxCounters(new Context()).block();
+ @Override
+ public CassandraMessageIdDAO imapUidToMessageIdDAO() {
+ return imapUidToMessageIdDAO;
+ }
- assertThat(testee.recomputeMailboxCounters(new Context()).block())
- .isEqualTo(Result.COMPLETED);
- }
+ @Override
+ public CassandraMessageIdToImapUidDAO messageIdToImapUidDAO() {
+ return messageIdToImapUidDAO;
+ }
- @Test
- void recomputeMailboxCountersShouldReturnCompletedWhenMailboxListReferenceIsMissing() {
- mailboxDAO.save(MAILBOX).block();
- messageIdToImapUidDAO.insert(METADATA_UNSEEN).block();
- counterDAO.incrementUnseenAndCount(CASSANDRA_ID_1).block();
- counterDAO.incrementCount(CASSANDRA_ID_1).block();
+ @Override
+ public CassandraMailboxCounterDAO counterDAO() {
+ return counterDAO;
+ }
- testee.recomputeMailboxCounters(new Context()).block();
+ @Disabled("Inconsitencies can not be corrected on the fly as trust avoid their detection")
+ @Override
+ public void recomputeMailboxCountersShouldIgnoreMissingMailboxListReferences() {
- assertThat(testee.recomputeMailboxCounters(new Context()).block())
- .isEqualTo(Result.COMPLETED);
- }
+ }
- @Test
- void recomputeMailboxCountersShouldNoopWhenMailboxWithoutMessage() {
- mailboxDAO.save(MAILBOX).block();
+ @Disabled("Inconsitencies can not be corrected on the fly as trust avoid their detection")
+ @Override
+ public void recomputeMailboxCountersShouldUseSourceOfTruthForComputation() {
- testee.recomputeMailboxCounters(new Context()).block();
+ }
- assertThat(counterDAO.retrieveMailboxCounters(CASSANDRA_ID_1).blockOptional())
- .isEmpty();
- }
+ @Disabled("Inconsitencies can not be corrected on the fly as trust avoid their detection")
+ @Override
+ public void recomputeMailboxCountersShouldIgnoreOrphanMailboxListReference() {
- @Test
- void recomputeMailboxCountersShouldNoopWhenValidCounters() {
- mailboxDAO.save(MAILBOX).block();
- imapUidToMessageIdDAO.insert(METADATA_UNSEEN).block();
- messageIdToImapUidDAO.insert(METADATA_UNSEEN).block();
- counterDAO.incrementUnseenAndCount(CASSANDRA_ID_1).block();
- counterDAO.incrementCount(CASSANDRA_ID_1).block();
-
- testee.recomputeMailboxCounters(new Context()).block();
-
- assertThat(counterDAO.retrieveMailboxCounters(CASSANDRA_ID_1).blockOptional())
- .contains(MailboxCounters.builder()
- .mailboxId(CASSANDRA_ID_1)
- .count(1)
- .unseen(1)
- .build());
+ }
}
- @Test
- void recomputeMailboxCountersShouldRecreateMissingCounters() {
- mailboxDAO.save(MAILBOX).block();
- imapUidToMessageIdDAO.insert(METADATA_UNSEEN).block();
- messageIdToImapUidDAO.insert(METADATA_UNSEEN).block();
-
- testee.recomputeMailboxCounters(new Context()).block();
-
- assertThat(counterDAO.retrieveMailboxCounters(CASSANDRA_ID_1).blockOptional())
- .contains(MailboxCounters.builder()
- .mailboxId(CASSANDRA_ID_1)
- .count(1)
- .unseen(1)
- .build());
+ @Nested
+ class RecheckMessageDenormalizationTest implements Contract {
+ @Override
+ public RecomputeMailboxCountersService testee() {
+ return testee;
+ }
+
+ @Override
+ public CassandraMailboxDAO mailboxDAO() {
+ return mailboxDAO;
+ }
+
+ @Override
+ public RecomputeMailboxCountersService.Options options() {
+ return RecomputeMailboxCountersService.Options.recheckMessageDenormalization();
+ }
+
+ @Override
+ public CassandraMessageIdDAO imapUidToMessageIdDAO() {
+ return imapUidToMessageIdDAO;
+ }
+
+ @Override
+ public CassandraMessageIdToImapUidDAO messageIdToImapUidDAO() {
+ return messageIdToImapUidDAO;
+ }
+
+ @Override
+ public CassandraMailboxCounterDAO counterDAO() {
+ return counterDAO;
+ }
}
- @Test
- void recomputeMailboxCountersShouldResetIncorrectCounters() {
- mailboxDAO.save(MAILBOX).block();
- imapUidToMessageIdDAO.insert(METADATA_UNSEEN).block();
- messageIdToImapUidDAO.insert(METADATA_UNSEEN).block();
- counterDAO.incrementCount(CASSANDRA_ID_1).block();
-
- testee.recomputeMailboxCounters(new Context()).block();
-
- assertThat(counterDAO.retrieveMailboxCounters(CASSANDRA_ID_1).blockOptional())
- .contains(MailboxCounters.builder()
- .mailboxId(CASSANDRA_ID_1)
- .count(1)
- .unseen(1)
- .build());
- }
+ interface Contract {
+ RecomputeMailboxCountersService testee();
- @Test
- void recomputeMailboxCountersShouldTakeSeenIntoAccount() {
- mailboxDAO.save(MAILBOX).block();
- imapUidToMessageIdDAO.insert(METADATA_SEEN).block();
- messageIdToImapUidDAO.insert(METADATA_SEEN).block();
- counterDAO.incrementCount(CASSANDRA_ID_1).block();
-
- testee.recomputeMailboxCounters(new Context()).block();
-
- assertThat(counterDAO.retrieveMailboxCounters(CASSANDRA_ID_1).blockOptional())
- .contains(MailboxCounters.builder()
- .mailboxId(CASSANDRA_ID_1)
- .count(1)
- .unseen(0)
- .build());
- }
+ CassandraMailboxDAO mailboxDAO();
- @Test
- void recomputeMailboxCountersShouldUseSourceOfTruthForComputation() {
- mailboxDAO.save(MAILBOX).block();
- imapUidToMessageIdDAO.insert(METADATA_SEEN).block();
- messageIdToImapUidDAO.insert(METADATA_UNSEEN).block();
+ RecomputeMailboxCountersService.Options options();
- testee.recomputeMailboxCounters(new Context()).block();
+ CassandraMessageIdDAO imapUidToMessageIdDAO();
- assertThat(counterDAO.retrieveMailboxCounters(CASSANDRA_ID_1).blockOptional())
- .contains(MailboxCounters.builder()
- .mailboxId(CASSANDRA_ID_1)
- .count(1)
- .unseen(1)
- .build());
- }
+ CassandraMessageIdToImapUidDAO messageIdToImapUidDAO();
- @Test
- void recomputeMailboxCountersShouldIgnoreMissingMailboxListReferences() {
- mailboxDAO.save(MAILBOX).block();
- imapUidToMessageIdDAO.insert(METADATA_SEEN).block();
+ CassandraMailboxCounterDAO counterDAO();
- testee.recomputeMailboxCounters(new Context()).block();
+ @Test
+ default void recomputeMailboxCountersShouldReturnCompletedWhenNoMailboxes() {
+ assertThat(testee().recomputeMailboxCounters(new Context(), options()).block())
+ .isEqualTo(Result.COMPLETED);
+ }
- assertThat(counterDAO.retrieveMailboxCounters(CASSANDRA_ID_1).blockOptional())
- .isEmpty();
- }
+ @Test
+ default void recomputeMailboxCountersShouldReturnCompletedWhenMailboxWithNoMessages() {
+ mailboxDAO().save(MAILBOX).block();
- @Test
- void recomputeMailboxCountersShouldIgnoreOrphanMailboxListReference() {
- mailboxDAO.save(MAILBOX).block();
- imapUidToMessageIdDAO.insert(METADATA_UNSEEN).block();
+ assertThat(testee().recomputeMailboxCounters(new Context(), options()).block())
+ .isEqualTo(Result.COMPLETED);
+ }
- testee.recomputeMailboxCounters(new Context()).block();
+ @Test
+ default void recomputeMailboxCountersShouldReturnCompletedWhenMailboxWithMessages() {
+ mailboxDAO().save(MAILBOX).block();
+ imapUidToMessageIdDAO().insert(METADATA_UNSEEN).block();
+ messageIdToImapUidDAO().insert(METADATA_UNSEEN).block();
+ counterDAO().incrementUnseenAndCount(CASSANDRA_ID_1).block();
+ counterDAO().incrementCount(CASSANDRA_ID_1).block();
+
+ testee().recomputeMailboxCounters(new Context(), options()).block();
- assertThat(counterDAO.retrieveMailboxCounters(CASSANDRA_ID_1).blockOptional())
- .isEmpty();
+ assertThat(testee().recomputeMailboxCounters(new Context(), options()).block())
+ .isEqualTo(Result.COMPLETED);
+ }
+
+ @Test
+ default void recomputeMailboxCountersShouldReturnCompletedWhenMessageDenormalizationIssue() {
+ mailboxDAO().save(MAILBOX).block();
+ imapUidToMessageIdDAO().insert(METADATA_UNSEEN).block();
+ messageIdToImapUidDAO().insert(METADATA_SEEN).block();
+ counterDAO().incrementUnseenAndCount(CASSANDRA_ID_1).block();
+ counterDAO().incrementCount(CASSANDRA_ID_1).block();
+
+ testee().recomputeMailboxCounters(new Context(), options()).block();
+
+ assertThat(testee().recomputeMailboxCounters(new Context(), options()).block())
+ .isEqualTo(Result.COMPLETED);
+ }
+
+ @Test
+ default void recomputeMailboxCountersShouldReturnCountersAreIncorrect() {
+ mailboxDAO().save(MAILBOX).block();
+ imapUidToMessageIdDAO().insert(METADATA_UNSEEN).block();
+ messageIdToImapUidDAO().insert(METADATA_UNSEEN).block();
+
+ testee().recomputeMailboxCounters(new Context(), options()).block();
+
+ assertThat(testee().recomputeMailboxCounters(new Context(), options()).block())
+ .isEqualTo(Result.COMPLETED);
+ }
+
+ @Test
+ default void recomputeMailboxCountersShouldReturnCompletedWhenOrphanMailboxRegistration() {
+ mailboxDAO().save(MAILBOX).block();
+ imapUidToMessageIdDAO().insert(METADATA_UNSEEN).block();
+ counterDAO().incrementUnseenAndCount(CASSANDRA_ID_1).block();
+ counterDAO().incrementCount(CASSANDRA_ID_1).block();
+
+ testee().recomputeMailboxCounters(new Context(), options()).block();
+
+ assertThat(testee().recomputeMailboxCounters(new Context(), options()).block())
+ .isEqualTo(Result.COMPLETED);
+ }
+
+ @Test
+ default void recomputeMailboxCountersShouldReturnCompletedWhenMailboxListReferenceIsMissing() {
+ mailboxDAO().save(MAILBOX).block();
+ messageIdToImapUidDAO().insert(METADATA_UNSEEN).block();
+ counterDAO().incrementUnseenAndCount(CASSANDRA_ID_1).block();
+ counterDAO().incrementCount(CASSANDRA_ID_1).block();
+
+ testee().recomputeMailboxCounters(new Context(), options()).block();
+
+ assertThat(testee().recomputeMailboxCounters(new Context(), options()).block())
+ .isEqualTo(Result.COMPLETED);
+ }
+
+ @Test
+ default void recomputeMailboxCountersShouldNoopWhenMailboxWithoutMessage() {
+ mailboxDAO().save(MAILBOX).block();
+
+ testee().recomputeMailboxCounters(new Context(), options()).block();
+
+ assertThat(counterDAO().retrieveMailboxCounters(CASSANDRA_ID_1).blockOptional())
+ .isEmpty();
+ }
+
+ @Test
+ default void recomputeMailboxCountersShouldNoopWhenValidCounters() {
+ mailboxDAO().save(MAILBOX).block();
+ imapUidToMessageIdDAO().insert(METADATA_UNSEEN).block();
+ messageIdToImapUidDAO().insert(METADATA_UNSEEN).block();
+ counterDAO().incrementUnseenAndCount(CASSANDRA_ID_1).block();
+ counterDAO().incrementCount(CASSANDRA_ID_1).block();
+
+ testee().recomputeMailboxCounters(new Context(), options()).block();
+
+ assertThat(counterDAO().retrieveMailboxCounters(CASSANDRA_ID_1).blockOptional())
+ .contains(MailboxCounters.builder()
+ .mailboxId(CASSANDRA_ID_1)
+ .count(1)
+ .unseen(1)
+ .build());
+ }
+
+ @Test
+ default void recomputeMailboxCountersShouldRecreateMissingCounters() {
+ mailboxDAO().save(MAILBOX).block();
+ imapUidToMessageIdDAO().insert(METADATA_UNSEEN).block();
+ messageIdToImapUidDAO().insert(METADATA_UNSEEN).block();
+
+ testee().recomputeMailboxCounters(new Context(), options()).block();
+
+ assertThat(counterDAO().retrieveMailboxCounters(CASSANDRA_ID_1).blockOptional())
+ .contains(MailboxCounters.builder()
+ .mailboxId(CASSANDRA_ID_1)
+ .count(1)
+ .unseen(1)
+ .build());
+ }
+
+ @Test
+ default void recomputeMailboxCountersShouldResetIncorrectCounters() {
+ mailboxDAO().save(MAILBOX).block();
+ imapUidToMessageIdDAO().insert(METADATA_UNSEEN).block();
+ messageIdToImapUidDAO().insert(METADATA_UNSEEN).block();
+ counterDAO().incrementCount(CASSANDRA_ID_1).block();
+
+ testee().recomputeMailboxCounters(new Context(), options()).block();
+
+ assertThat(counterDAO().retrieveMailboxCounters(CASSANDRA_ID_1).blockOptional())
+ .contains(MailboxCounters.builder()
+ .mailboxId(CASSANDRA_ID_1)
+ .count(1)
+ .unseen(1)
+ .build());
+ }
+
+ @Test
+ default void recomputeMailboxCountersShouldTakeSeenIntoAccount() {
+ mailboxDAO().save(MAILBOX).block();
+ imapUidToMessageIdDAO().insert(METADATA_SEEN).block();
+ messageIdToImapUidDAO().insert(METADATA_SEEN).block();
+ counterDAO().incrementCount(CASSANDRA_ID_1).block();
+
+ testee().recomputeMailboxCounters(new Context(), options()).block();
+
+ assertThat(counterDAO().retrieveMailboxCounters(CASSANDRA_ID_1).blockOptional())
+ .contains(MailboxCounters.builder()
+ .mailboxId(CASSANDRA_ID_1)
+ .count(1)
+ .unseen(0)
+ .build());
+ }
+
+ @Test
+ default void recomputeMailboxCountersShouldUseSourceOfTruthForComputation() {
+ mailboxDAO().save(MAILBOX).block();
+ imapUidToMessageIdDAO().insert(METADATA_SEEN).block();
+ messageIdToImapUidDAO().insert(METADATA_UNSEEN).block();
+
+ testee().recomputeMailboxCounters(new Context(), options()).block();
+
+ assertThat(counterDAO().retrieveMailboxCounters(CASSANDRA_ID_1).blockOptional())
+ .contains(MailboxCounters.builder()
+ .mailboxId(CASSANDRA_ID_1)
+ .count(1)
+ .unseen(1)
+ .build());
+ }
+
+ @Test
+ default void recomputeMailboxCountersShouldIgnoreMissingMailboxListReferences() {
+ mailboxDAO().save(MAILBOX).block();
+ imapUidToMessageIdDAO().insert(METADATA_SEEN).block();
+
+ testee().recomputeMailboxCounters(new Context(), options()).block();
+
+ assertThat(counterDAO().retrieveMailboxCounters(CASSANDRA_ID_1).blockOptional())
+ .isEmpty();
+ }
+
+ @Test
+ default void recomputeMailboxCountersShouldIgnoreOrphanMailboxListReference() {
+ mailboxDAO().save(MAILBOX).block();
+ imapUidToMessageIdDAO().insert(METADATA_UNSEEN).block();
+
+ testee().recomputeMailboxCounters(new Context(), options()).block();
+
+ assertThat(counterDAO().retrieveMailboxCounters(CASSANDRA_ID_1).blockOptional())
+ .isEmpty();
+ }
}
}
\ No newline at end of file
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTaskSerializationTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTaskSerializationTest.java
index 15c787c..e46fa08 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTaskSerializationTest.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersTaskSerializationTest.java
@@ -19,16 +19,14 @@
package org.apache.james.mailbox.cassandra.mail.task;
+import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.Mockito.mock;
import java.time.Instant;
-import java.util.UUID;
import org.apache.james.JsonSerializationVerifier;
-import org.apache.james.core.Username;
-import org.apache.james.mailbox.cassandra.ids.CassandraId;
-import org.apache.james.mailbox.model.MailboxConstants;
-import org.apache.james.mailbox.model.MailboxPath;
+import org.apache.james.json.JsonGenericSerializer;
+import org.apache.james.mailbox.cassandra.mail.task.RecomputeMailboxCountersService.Options;
import org.junit.jupiter.api.Test;
import com.google.common.collect.ImmutableList;
@@ -38,8 +36,17 @@ class RecomputeMailboxCountersTaskSerializationTest {
private static final String MAILBOX_ID_AS_STRING = "464765a0-e4e7-11e4-aba4-710c1de3782b";
private static final RecomputeMailboxCountersService SERVICE = mock(RecomputeMailboxCountersService.class);
- private static final RecomputeMailboxCountersTask TASK = new RecomputeMailboxCountersTask(SERVICE);
- private static final String SERIALIZED_TASK = "{\"type\": \"recompute-mailbox-counters\"}";
+ private static final RecomputeMailboxCountersTask TASK_TRUSTED = new RecomputeMailboxCountersTask(SERVICE, Options.trustMessageDenormalization());
+ private static final RecomputeMailboxCountersTask TASK_UNTRUSTED = new RecomputeMailboxCountersTask(SERVICE, Options.recheckMessageDenormalization());
+ private static final String SERIALIZED_TASK_TRUSTED = "{" +
+ " \"type\": \"recompute-mailbox-counters\"," +
+ " \"trustMessageDenormalization\": true" +
+ "}";
+ private static final String SERIALIZED_TASK_UNTRUSTED = "{" +
+ " \"type\": \"recompute-mailbox-counters\"," +
+ " \"trustMessageDenormalization\": false" +
+ "}";
+ private static final String SERIALIZED_TASK_OLD = "{\"type\": \"recompute-mailbox-counters\"}";
private static final RecomputeMailboxCountersTask.Details DETAILS = new RecomputeMailboxCountersTask.Details(TIMESTAMP, 12, ImmutableList.of(MAILBOX_ID_AS_STRING));
private static final String SERIALIZED_ADDITIONAL_INFORMATION = "{" +
" \"type\":\"recompute-mailbox-counters\"," +
@@ -49,14 +56,33 @@ class RecomputeMailboxCountersTaskSerializationTest {
"}";
@Test
- void taskShouldBeSerializable() throws Exception {
+ void taskShouldBeSerializableWhenTrusted() throws Exception {
JsonSerializationVerifier.dtoModule(RecomputeMailboxCountersTaskDTO.module(SERVICE))
- .bean(TASK)
- .json(SERIALIZED_TASK)
+ .bean(TASK_TRUSTED)
+ .json(SERIALIZED_TASK_TRUSTED)
.verify();
}
@Test
+ void taskShouldBeSerializableWhenUnTrusted() throws Exception {
+ JsonSerializationVerifier.dtoModule(RecomputeMailboxCountersTaskDTO.module(SERVICE))
+ .bean(TASK_UNTRUSTED)
+ .json(SERIALIZED_TASK_UNTRUSTED)
+ .verify();
+ }
+
+ @Test
+ void taskWithoutTrustFieldShouldBeWellDeSerialized() throws Exception {
+ RecomputeMailboxCountersTask domainObject = JsonGenericSerializer
+ .forModules(RecomputeMailboxCountersTaskDTO.module(SERVICE))
+ .withoutNestedType()
+ .deserialize(SERIALIZED_TASK_OLD);
+
+ assertThat(domainObject)
+ .isEqualToComparingFieldByFieldRecursively(TASK_UNTRUSTED);
+ }
+
+ @Test
void additionalInformationShouldBeSerializable() throws Exception {
JsonSerializationVerifier.dtoModule(RecomputeMailboxCountersTaskAdditionalInformationDTO.MODULE)
.bean(DETAILS)
diff --git a/server/protocols/webadmin/webadmin-cassandra/src/main/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTask.java b/server/protocols/webadmin/webadmin-cassandra/src/main/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTask.java
index 2ec6679..2d60e61 100644
--- a/server/protocols/webadmin/webadmin-cassandra/src/main/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTask.java
+++ b/server/protocols/webadmin/webadmin-cassandra/src/main/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTask.java
@@ -22,6 +22,7 @@ package org.apache.james.webadmin.routes;
import javax.inject.Inject;
import org.apache.james.mailbox.cassandra.mail.task.RecomputeMailboxCountersService;
+import org.apache.james.mailbox.cassandra.mail.task.RecomputeMailboxCountersService.Options;
import org.apache.james.mailbox.cassandra.mail.task.RecomputeMailboxCountersTask;
import org.apache.james.webadmin.tasks.TaskFromRequestRegistry;
import org.apache.james.webadmin.tasks.TaskRegistrationKey;
@@ -32,6 +33,6 @@ public class RecomputeMailboxCountersRequestToTask extends TaskFromRequestRegist
@Inject
public RecomputeMailboxCountersRequestToTask(RecomputeMailboxCountersService service) {
super(REGISTRATION_KEY,
- request -> new RecomputeMailboxCountersTask(service));
+ request -> new RecomputeMailboxCountersTask(service, Options.recheckMessageDenormalization()));
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[james-project] 06/12: JAMES-2997 Move ContentType JavaDoc to the
class
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 8f74fcf24b8891a7689d1f3af51ec6f899454014
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Tue Apr 28 16:13:43 2020 +0700
JAMES-2997 Move ContentType JavaDoc to the class
---
.../java/org/apache/james/mailbox/model/ContentType.java | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/mailbox/api/src/main/java/org/apache/james/mailbox/model/ContentType.java b/mailbox/api/src/main/java/org/apache/james/mailbox/model/ContentType.java
index 5eeeb26..09f1002 100644
--- a/mailbox/api/src/main/java/org/apache/james/mailbox/model/ContentType.java
+++ b/mailbox/api/src/main/java/org/apache/james/mailbox/model/ContentType.java
@@ -33,6 +33,14 @@ import com.google.common.base.MoreObjects;
import com.google.common.base.Preconditions;
import com.google.common.base.Strings;
+
+/**
+ * Follows syntax and usage as defined in https://tools.ietf.org/html/rfc2045#section-5
+ * Thus includes mime type, defined by its media type and subtype as well as contentType fields parameters,
+ * including charset
+ *
+ * Example: text/plain; charset=utf-8
+ */
public class ContentType {
public static class MimeType {
public static MimeType of(String value) {
@@ -181,11 +189,6 @@ public class ContentType {
.orElse(mimeType.asString()));
}
- /**
- * Follows syntax and usage as defined in https://tools.ietf.org/html/rfc2045#section-5
- * Thus includes media type and parameters, including charset
- * Example: text/plain; charset=utf-8
- */
private final String value;
public ContentType(String value) {
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[james-project] 05/12: JAMES-2997 DefaultTextExtractor need to
handle correctly charset
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 1b4e0c81d62c0d6684da0d49351c9f94083e58e0
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Mon Apr 27 19:38:11 2020 +0700
JAMES-2997 DefaultTextExtractor need to handle correctly charset
---
.../apache/james/mailbox/store/extractor/DefaultTextExtractor.java | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/mailbox/store/src/main/java/org/apache/james/mailbox/store/extractor/DefaultTextExtractor.java b/mailbox/store/src/main/java/org/apache/james/mailbox/store/extractor/DefaultTextExtractor.java
index ea4d76b..65c03bb 100644
--- a/mailbox/store/src/main/java/org/apache/james/mailbox/store/extractor/DefaultTextExtractor.java
+++ b/mailbox/store/src/main/java/org/apache/james/mailbox/store/extractor/DefaultTextExtractor.java
@@ -20,6 +20,7 @@
package org.apache.james.mailbox.store.extractor;
import java.io.InputStream;
+import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import java.util.HashMap;
import java.util.Optional;
@@ -39,7 +40,8 @@ public class DefaultTextExtractor implements TextExtractor {
@Override
public ParsedContent extractContent(InputStream inputStream, ContentType contentType) throws Exception {
if (contentType != null && contentType.asString().startsWith("text/")) {
- return new ParsedContent(Optional.ofNullable(IOUtils.toString(inputStream, StandardCharsets.UTF_8)), new HashMap<>());
+ Charset charset = contentType.charset().orElse(StandardCharsets.UTF_8);
+ return new ParsedContent(Optional.ofNullable(IOUtils.toString(inputStream, charset)), new HashMap<>());
} else {
return new ParsedContent(Optional.empty(), new HashMap<>());
}
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[james-project] 03/12: JAMES-2997 TextExtractor should rely on
ContentType
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit e7de644d3e5ff95f69ccdd47a768e738e01a1a31
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Mon Apr 27 19:34:27 2020 +0700
JAMES-2997 TextExtractor should rely on ContentType
These classes were not handling the full content field but expecting the
MimeType alone.
Furthermore, we can take advantage og having the charset available, ie
when calling tika.
---
.../james/mailbox/extractor/TextExtractor.java | 4 +-
.../apache/james/mailbox/model/ContentType.java | 179 ++++++++++++++++++++-
.../james/mailbox/model/ContentTypeTest.java | 179 +++++++++++++++++++++
.../james/mailbox/elasticsearch/json/MimePart.java | 35 ++--
.../json/MimePartContainerBuilder.java | 6 +-
.../mailbox/elasticsearch/json/MimePartParser.java | 12 +-
.../json/RootMimePartContainerBuilder.java | 6 +-
...asticSearchListeningMessageSearchIndexTest.java | 3 +-
.../mailbox/elasticsearch/json/MimePartTest.java | 11 +-
.../mailbox/store/search/PDFTextExtractor.java | 10 +-
.../mailbox/store/search/PDFTextExtractorTest.java | 7 +-
.../store/extractor/DefaultTextExtractor.java | 5 +-
.../store/extractor/JsoupTextExtractor.java | 10 +-
.../mailbox/store/search/MessageSearches.java | 3 +-
.../store/extractor/DefaultTextExtractorTest.java | 14 +-
.../store/extractor/JsoupTextExtractorTest.java | 33 +++-
.../test/resources/documents/html-iso-8859-1.html} | 25 +--
.../resources/documents/simple-text-iso-8859-1.txt | 1 +
.../james/mailbox/tika/CachingTextExtractor.java | 5 +-
.../tika/ContentTypeFilteringTextExtractor.java | 12 +-
.../james/mailbox/tika/TikaConfiguration.java | 11 +-
.../apache/james/mailbox/tika/TikaHttpClient.java | 4 +-
.../james/mailbox/tika/TikaHttpClientImpl.java | 9 +-
.../james/mailbox/tika/TikaTextExtractor.java | 8 +-
.../mailbox/tika/CachingTextExtractorTest.java | 3 +-
.../ContentTypeFilteringTextExtractorTest.java | 22 ++-
.../james/mailbox/tika/TikaTextExtractorTest.java | 38 +++--
.../modules/mailbox/TikaConfigurationReader.java | 4 +-
.../mailbox/TikaConfigurationReaderTest.java | 5 +-
.../jmap/draft/methods/MIMEMessageConverter.java | 4 +-
30 files changed, 567 insertions(+), 101 deletions(-)
diff --git a/mailbox/api/src/main/java/org/apache/james/mailbox/extractor/TextExtractor.java b/mailbox/api/src/main/java/org/apache/james/mailbox/extractor/TextExtractor.java
index b33d853..6b91122 100644
--- a/mailbox/api/src/main/java/org/apache/james/mailbox/extractor/TextExtractor.java
+++ b/mailbox/api/src/main/java/org/apache/james/mailbox/extractor/TextExtractor.java
@@ -21,8 +21,10 @@ package org.apache.james.mailbox.extractor;
import java.io.InputStream;
+import org.apache.james.mailbox.model.ContentType;
+
public interface TextExtractor {
- ParsedContent extractContent(InputStream inputStream, String contentType) throws Exception;
+ ParsedContent extractContent(InputStream inputStream, ContentType contentType) throws Exception;
}
diff --git a/mailbox/api/src/main/java/org/apache/james/mailbox/model/ContentType.java b/mailbox/api/src/main/java/org/apache/james/mailbox/model/ContentType.java
index ed8ecf5..5eeeb26 100644
--- a/mailbox/api/src/main/java/org/apache/james/mailbox/model/ContentType.java
+++ b/mailbox/api/src/main/java/org/apache/james/mailbox/model/ContentType.java
@@ -19,15 +19,168 @@
package org.apache.james.mailbox.model;
+import java.io.StringReader;
+import java.nio.charset.Charset;
+import java.util.Objects;
+import java.util.Optional;
+
+import org.apache.james.mime4j.dom.field.ContentTypeField;
+import org.apache.james.mime4j.field.Fields;
+import org.apache.james.mime4j.field.contenttype.parser.ContentTypeParser;
+import org.apache.james.mime4j.field.contenttype.parser.ParseException;
+
+import com.google.common.base.MoreObjects;
import com.google.common.base.Preconditions;
import com.google.common.base.Strings;
public class ContentType {
+ public static class MimeType {
+ public static MimeType of(String value) {
+ ContentTypeParser parser = new ContentTypeParser(new StringReader(value));
+ try {
+ parser.parseAll();
+ } catch (ParseException e) {
+ throw new IllegalArgumentException("Invalid mimeType", e);
+ }
+ return new MimeType(
+ new MediaType(parser.getType()),
+ new SubType(parser.getSubType()));
+ }
+
+ public static MimeType of(MediaType mediaType, SubType subType) {
+ return new MimeType(mediaType, subType);
+ }
+
+ private final MediaType mediaType;
+ private final SubType subType;
+
+ private MimeType(MediaType mediaType, SubType subType) {
+ this.mediaType = mediaType;
+ this.subType = subType;
+ }
+
+ public String asString() {
+ return mediaType.asString() + "/" + subType.asString();
+ }
+
+ @Override
+ public final boolean equals(Object o) {
+ if (o instanceof MimeType) {
+ MimeType mimeType = (MimeType) o;
+
+ return Objects.equals(this.mediaType, mimeType.mediaType)
+ && Objects.equals(this.subType, mimeType.subType);
+ }
+ return false;
+ }
+
+ @Override
+ public final int hashCode() {
+ return Objects.hash(mediaType, subType);
+ }
+
+ @Override
+ public String toString() {
+ return MoreObjects.toStringHelper(this)
+ .add("mediaType", mediaType)
+ .add("subType", subType)
+ .toString();
+ }
+ }
+
+ public static class MediaType {
+ public static MediaType of(String value) {
+ Preconditions.checkState(!Strings.isNullOrEmpty(value), "'media type' is mandatory");
+ return new MediaType(value);
+ }
+
+ private final String value;
+
+ private MediaType(String value) {
+ this.value = value;
+ }
+
+ public String asString() {
+ return value;
+ }
+
+ @Override
+ public final boolean equals(Object o) {
+ if (o instanceof MediaType) {
+ MediaType mediaType = (MediaType) o;
+
+ return Objects.equals(this.value, mediaType.value);
+ }
+ return false;
+ }
+
+ @Override
+ public final int hashCode() {
+ return Objects.hash(value);
+ }
+
+ @Override
+ public String toString() {
+ return MoreObjects.toStringHelper(this)
+ .add("value", value)
+ .toString();
+ }
+ }
+
+ public static class SubType {
+ public static SubType of(String value) {
+ Preconditions.checkState(!Strings.isNullOrEmpty(value), "'sub type' is mandatory");
+ return new SubType(value);
+ }
+
+ private final String value;
+
+ private SubType(String value) {
+ this.value = value;
+ }
+
+ public String asString() {
+ return value;
+ }
+
+ @Override
+ public final boolean equals(Object o) {
+ if (o instanceof SubType) {
+ SubType subType = (SubType) o;
+
+ return Objects.equals(this.value, subType.value);
+ }
+ return false;
+ }
+
+ @Override
+ public final int hashCode() {
+ return Objects.hash(value);
+ }
+
+ @Override
+ public String toString() {
+ return MoreObjects.toStringHelper(this)
+ .add("value", value)
+ .toString();
+ }
+ }
+
public static ContentType of(String value) {
- Preconditions.checkState(!Strings.isNullOrEmpty(value), "'content type' is mandatory");
+ Preconditions.checkArgument(!Strings.isNullOrEmpty(value), "'content type' is mandatory");
return new ContentType(value);
}
+ public static ContentType of(MimeType mimeType) {
+ return new ContentType(mimeType.asString());
+ }
+
+ public static ContentType of(MimeType mimeType, Optional<Charset> charset) {
+ return ContentType.of(
+ charset.map(value -> mimeType.asString() + "; charset=" + value.name())
+ .orElse(mimeType.asString()));
+ }
+
/**
* Follows syntax and usage as defined in https://tools.ietf.org/html/rfc2045#section-5
* Thus includes media type and parameters, including charset
@@ -39,6 +192,30 @@ public class ContentType {
this.value = value;
}
+ public ContentTypeField asMime4J() {
+ return Fields.contentType(value);
+ }
+
+ public MimeType mimeType() {
+ ContentTypeField contentTypeField = asMime4J();
+ return MimeType.of(
+ MediaType.of(contentTypeField.getMediaType()),
+ SubType.of(contentTypeField.getSubType()));
+ }
+
+ public MediaType mediaType() {
+ return MediaType.of(asMime4J().getMediaType());
+ }
+
+ public SubType subType() {
+ return SubType.of(asMime4J().getSubType());
+ }
+
+ public Optional<Charset> charset() {
+ return Optional.ofNullable(asMime4J().getCharset())
+ .map(Charset::forName);
+ }
+
public String asString() {
return value;
}
diff --git a/mailbox/api/src/test/java/org/apache/james/mailbox/model/ContentTypeTest.java b/mailbox/api/src/test/java/org/apache/james/mailbox/model/ContentTypeTest.java
new file mode 100644
index 0000000..d8568bb
--- /dev/null
+++ b/mailbox/api/src/test/java/org/apache/james/mailbox/model/ContentTypeTest.java
@@ -0,0 +1,179 @@
+/****************************************************************
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information *
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the *
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the *
+ * specific language governing permissions and limitations *
+ * under the License. *
+ ****************************************************************/
+
+package org.apache.james.mailbox.model;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatCode;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+import java.nio.charset.StandardCharsets;
+
+import org.junit.jupiter.api.Test;
+
+import nl.jqno.equalsverifier.EqualsVerifier;
+
+class ContentTypeTest {
+ @Test
+ void contentTypeShouldRespectBeanContract() {
+ EqualsVerifier.forClass(ContentType.class)
+ .verify();
+ }
+
+ @Test
+ void subTypeShouldRespectBeanContract() {
+ EqualsVerifier.forClass(ContentType.SubType.class)
+ .verify();
+ }
+
+ @Test
+ void mediaTypeShouldRespectBeanContract() {
+ EqualsVerifier.forClass(ContentType.MediaType.class)
+ .verify();
+ }
+
+ @Test
+ void mimeTypeShouldRespectBeanContract() {
+ EqualsVerifier.forClass(ContentType.MimeType.class)
+ .verify();
+ }
+
+ @Test
+ void mimeTypeOfShouldThrowWhenInvalid() {
+ assertThatThrownBy(() -> ContentType.MimeType.of("aaa"))
+ .isInstanceOf(IllegalArgumentException.class);
+ }
+
+ @Test
+ void mimeTypeOfShouldThrowWhenEmpty() {
+ assertThatThrownBy(() -> ContentType.of(""))
+ .isInstanceOf(IllegalArgumentException.class);
+ }
+
+ @Test
+ void mimeTypeOfShouldThrowWhenNull() {
+ assertThatThrownBy(() -> ContentType.of((String) null))
+ .isInstanceOf(IllegalArgumentException.class);
+ }
+
+ @Test
+ void mimeTypeOfShouldReturnExpectedValue() {
+ assertThat(ContentType.MimeType.of("text/html"))
+ .isEqualTo(ContentType.MimeType.of(
+ ContentType.MediaType.of("text"),
+ ContentType.SubType.of("html")));
+ }
+
+ @Test
+ void mediaTypeOfShouldThrowWhenEmpty() {
+ assertThatThrownBy(() -> ContentType.MediaType.of(""))
+ .isInstanceOf(IllegalStateException.class);
+ }
+
+ @Test
+ void subTypeOfShouldThrowWhenEmpty() {
+ assertThatThrownBy(() -> ContentType.SubType.of(""))
+ .isInstanceOf(IllegalStateException.class);
+ }
+
+ @Test
+ void asMime4JShouldNotThrowWhenInvalidContentType() {
+ ContentType invalid = ContentType.of("/invalid");
+ assertThatCode(invalid::asMime4J)
+ .doesNotThrowAnyException();
+ }
+
+ @Test
+ void mimeTypeShouldThrowWhenInvalidContentType() {
+ ContentType invalid = ContentType.of("invalid");
+ assertThatThrownBy(invalid::mimeType)
+ .isInstanceOf(IllegalStateException.class);
+ }
+
+ @Test
+ void mediaTypeShouldThrowWhenInvalidContentType() {
+ ContentType invalid = ContentType.of("invalid");
+ assertThatThrownBy(invalid::mediaType)
+ .isInstanceOf(IllegalStateException.class);
+ }
+
+ @Test
+ void subTypeShouldThrowWhenInvalidContentType() {
+ ContentType invalid = ContentType.of("invalid");
+ assertThatThrownBy(invalid::subType)
+ .isInstanceOf(IllegalStateException.class);
+ }
+
+ @Test
+ void charsetShouldThrowWhenInvalidCharset() {
+ ContentType invalid = ContentType.of("text/plain; charset=invalid");
+ assertThatThrownBy(invalid::charset)
+ .isInstanceOf(IllegalArgumentException.class);
+ }
+
+ @Test
+ void charsetShouldThrowWhenEmptyCharset() {
+ ContentType contentType = ContentType.of("text/plain; charset=");
+
+ assertThat(contentType.charset())
+ .isEmpty();
+ }
+
+ @Test
+ void asStringShouldReturnWhenInvalid() {
+ String value = "invalid";
+ ContentType invalid = ContentType.of(value);
+ assertThat(invalid.asString()).isEqualTo(value);
+ }
+
+ @Test
+ void charsetShouldReturnEmptyWhenNone() {
+ ContentType contentType = ContentType.of("text/html");
+
+ assertThat(contentType.charset()).isEmpty();
+ }
+
+ @Test
+ void charsetShouldReturnSuppliedCharset() {
+ ContentType contentType = ContentType.of("text/html; charset=UTF-8");
+
+ assertThat(contentType.charset()).contains(StandardCharsets.UTF_8);
+ }
+
+ @Test
+ void mimeTypeShouldReturnSuppliedValue() {
+ ContentType contentType = ContentType.of("text/html");
+
+ assertThat(contentType.mimeType()).isEqualTo(ContentType.MimeType.of("text/html"));
+ }
+
+ @Test
+ void subTypeShouldReturnSuppliedValue() {
+ ContentType contentType = ContentType.of("text/html");
+
+ assertThat(contentType.subType()).isEqualTo(ContentType.SubType.of("html"));
+ }
+
+ @Test
+ void mediaTypeShouldReturnSuppliedValue() {
+ ContentType contentType = ContentType.of("text/html");
+
+ assertThat(contentType.mediaType()).isEqualTo(ContentType.MediaType.of("text"));
+ }
+}
\ No newline at end of file
diff --git a/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/json/MimePart.java b/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/json/MimePart.java
index 55fe7b6..d818cfd 100644
--- a/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/json/MimePart.java
+++ b/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/json/MimePart.java
@@ -30,6 +30,9 @@ import org.apache.commons.io.FilenameUtils;
import org.apache.commons.io.IOUtils;
import org.apache.james.mailbox.extractor.ParsedContent;
import org.apache.james.mailbox.extractor.TextExtractor;
+import org.apache.james.mailbox.model.ContentType;
+import org.apache.james.mailbox.model.ContentType.MediaType;
+import org.apache.james.mailbox.model.ContentType.SubType;
import org.apache.james.mailbox.store.extractor.DefaultTextExtractor;
import org.apache.james.mime4j.stream.Field;
import org.slf4j.Logger;
@@ -48,8 +51,8 @@ public class MimePart {
private final HeaderCollection.Builder headerCollectionBuilder;
private Optional<InputStream> bodyContent;
private final List<MimePart> children;
- private Optional<String> mediaType;
- private Optional<String> subType;
+ private Optional<MediaType> mediaType;
+ private Optional<SubType> subType;
private Optional<String> fileName;
private Optional<String> fileExtension;
private Optional<String> contentDisposition;
@@ -95,14 +98,14 @@ public class MimePart {
}
@Override
- public Builder addMediaType(String mediaType) {
+ public Builder addMediaType(MediaType mediaType) {
this.mediaType = Optional.ofNullable(mediaType);
return this;
}
@Override
- public Builder addSubType(String subType) {
- this.subType = Optional.ofNullable(subType);
+ public Builder addSubType(SubType subType) {
+ this.subType = Optional.of(subType);
return this;
}
@@ -166,16 +169,18 @@ public class MimePart {
}
private Boolean isTextBody() {
- return mediaType.map("text"::equals).orElse(false);
+ return mediaType.map(MediaType.of("text")::equals).orElse(false);
}
private Boolean isHtml() {
- return isTextBody() && subType.map("html"::equals).orElse(false);
+ return isTextBody() && subType.map(SubType.of("html")::equals).orElse(false);
}
- private Optional<String> computeContentType() {
+ private Optional<ContentType> computeContentType() {
if (mediaType.isPresent() && subType.isPresent()) {
- return Optional.of(mediaType.get() + "/" + subType.get());
+ return Optional.of(ContentType.of(
+ ContentType.MimeType.of(mediaType.get(), subType.get()),
+ charset));
} else {
return Optional.empty();
}
@@ -191,15 +196,15 @@ public class MimePart {
private final HeaderCollection headerCollection;
private final Optional<String> bodyTextContent;
- private final Optional<String> mediaType;
- private final Optional<String> subType;
+ private final Optional<MediaType> mediaType;
+ private final Optional<SubType> subType;
private final Optional<String> fileName;
private final Optional<String> fileExtension;
private final Optional<String> contentDisposition;
private final List<MimePart> attachments;
- private MimePart(HeaderCollection headerCollection, Optional<String> bodyTextContent, Optional<String> mediaType,
- Optional<String> subType, Optional<String> fileName, Optional<String> fileExtension,
+ private MimePart(HeaderCollection headerCollection, Optional<String> bodyTextContent, Optional<MediaType> mediaType,
+ Optional<SubType> subType, Optional<String> fileName, Optional<String> fileExtension,
Optional<String> contentDisposition, List<MimePart> attachments) {
this.headerCollection = headerCollection;
this.mediaType = mediaType;
@@ -233,12 +238,12 @@ public class MimePart {
@JsonProperty(JsonMessageConstants.Attachment.MEDIA_TYPE)
public Optional<String> getMediaType() {
- return mediaType;
+ return mediaType.map(MediaType::asString);
}
@JsonProperty(JsonMessageConstants.Attachment.SUBTYPE)
public Optional<String> getSubType() {
- return subType;
+ return subType.map(SubType::asString);
}
@JsonProperty(JsonMessageConstants.Attachment.CONTENT_DISPOSITION)
diff --git a/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/json/MimePartContainerBuilder.java b/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/json/MimePartContainerBuilder.java
index 4a71d21..88439fb 100644
--- a/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/json/MimePartContainerBuilder.java
+++ b/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/json/MimePartContainerBuilder.java
@@ -23,6 +23,8 @@ import java.io.InputStream;
import java.nio.charset.Charset;
import org.apache.james.mailbox.extractor.TextExtractor;
+import org.apache.james.mailbox.model.ContentType.MediaType;
+import org.apache.james.mailbox.model.ContentType.SubType;
import org.apache.james.mime4j.stream.Field;
public interface MimePartContainerBuilder {
@@ -41,9 +43,9 @@ public interface MimePartContainerBuilder {
MimePartContainerBuilder charset(Charset charset);
- MimePartContainerBuilder addMediaType(String mediaType);
+ MimePartContainerBuilder addMediaType(MediaType mediaType);
- MimePartContainerBuilder addSubType(String subType);
+ MimePartContainerBuilder addSubType(SubType subType);
MimePartContainerBuilder addContentDisposition(String contentDisposition);
diff --git a/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/json/MimePartParser.java b/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/json/MimePartParser.java
index a87ae44..d12748f 100644
--- a/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/json/MimePartParser.java
+++ b/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/json/MimePartParser.java
@@ -26,6 +26,8 @@ import java.util.LinkedList;
import java.util.Optional;
import org.apache.james.mailbox.extractor.TextExtractor;
+import org.apache.james.mailbox.model.ContentType.MediaType;
+import org.apache.james.mailbox.model.ContentType.SubType;
import org.apache.james.mailbox.store.mail.model.Message;
import org.apache.james.mime4j.MimeException;
import org.apache.james.mime4j.message.DefaultBodyDescriptorBuilder;
@@ -119,9 +121,13 @@ public class MimePartParser {
private void extractMimePartBodyDescription(MimeTokenStream stream) {
MaximalBodyDescriptor descriptor = (MaximalBodyDescriptor) stream.getBodyDescriptor();
- currentlyBuildMimePart.addMediaType(descriptor.getMediaType())
- .addSubType(descriptor.getSubType())
- .addContentDisposition(descriptor.getContentDispositionType())
+ Optional.ofNullable(descriptor.getMediaType())
+ .map(MediaType::of)
+ .ifPresent(currentlyBuildMimePart::addMediaType);
+ Optional.ofNullable(descriptor.getSubType())
+ .map(SubType::of)
+ .ifPresent(currentlyBuildMimePart::addSubType);
+ currentlyBuildMimePart.addContentDisposition(descriptor.getContentDispositionType())
.addFileName(descriptor.getContentDispositionFilename());
extractCharset(descriptor);
}
diff --git a/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/json/RootMimePartContainerBuilder.java b/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/json/RootMimePartContainerBuilder.java
index 81e58be..71a88ce 100644
--- a/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/json/RootMimePartContainerBuilder.java
+++ b/mailbox/elasticsearch/src/main/java/org/apache/james/mailbox/elasticsearch/json/RootMimePartContainerBuilder.java
@@ -23,6 +23,8 @@ import java.io.InputStream;
import java.nio.charset.Charset;
import org.apache.james.mailbox.extractor.TextExtractor;
+import org.apache.james.mailbox.model.ContentType.MediaType;
+import org.apache.james.mailbox.model.ContentType.SubType;
import org.apache.james.mime4j.stream.Field;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -71,13 +73,13 @@ public class RootMimePartContainerBuilder implements MimePartContainerBuilder {
}
@Override
- public MimePartContainerBuilder addMediaType(String mediaType) {
+ public MimePartContainerBuilder addMediaType(MediaType mediaType) {
LOGGER.warn("Trying to add media type to the Root MimePart container");
return this;
}
@Override
- public MimePartContainerBuilder addSubType(String subType) {
+ public MimePartContainerBuilder addSubType(SubType subType) {
LOGGER.warn("Trying to add sub type to the Root MimePart container");
return this;
}
diff --git a/mailbox/elasticsearch/src/test/java/org/apache/james/mailbox/elasticsearch/events/ElasticSearchListeningMessageSearchIndexTest.java b/mailbox/elasticsearch/src/test/java/org/apache/james/mailbox/elasticsearch/events/ElasticSearchListeningMessageSearchIndexTest.java
index dcbeb0f..bd4fc78 100644
--- a/mailbox/elasticsearch/src/test/java/org/apache/james/mailbox/elasticsearch/events/ElasticSearchListeningMessageSearchIndexTest.java
+++ b/mailbox/elasticsearch/src/test/java/org/apache/james/mailbox/elasticsearch/events/ElasticSearchListeningMessageSearchIndexTest.java
@@ -58,6 +58,7 @@ import org.apache.james.mailbox.inmemory.InMemoryMessageId;
import org.apache.james.mailbox.manager.ManagerTestProvisionner;
import org.apache.james.mailbox.model.AttachmentId;
import org.apache.james.mailbox.model.AttachmentMetadata;
+import org.apache.james.mailbox.model.ContentType;
import org.apache.james.mailbox.model.Mailbox;
import org.apache.james.mailbox.model.MailboxPath;
import org.apache.james.mailbox.model.MessageAttachmentMetadata;
@@ -130,7 +131,7 @@ class ElasticSearchListeningMessageSearchIndexTest {
static class FailingTextExtractor implements TextExtractor {
@Override
- public ParsedContent extractContent(InputStream inputStream, String contentType) {
+ public ParsedContent extractContent(InputStream inputStream, ContentType contentType) {
throw new RuntimeException();
}
}
diff --git a/mailbox/elasticsearch/src/test/java/org/apache/james/mailbox/elasticsearch/json/MimePartTest.java b/mailbox/elasticsearch/src/test/java/org/apache/james/mailbox/elasticsearch/json/MimePartTest.java
index 79a03c8..19fde86 100644
--- a/mailbox/elasticsearch/src/test/java/org/apache/james/mailbox/elasticsearch/json/MimePartTest.java
+++ b/mailbox/elasticsearch/src/test/java/org/apache/james/mailbox/elasticsearch/json/MimePartTest.java
@@ -23,6 +23,9 @@ import static org.assertj.core.api.Assertions.assertThat;
import java.io.ByteArrayInputStream;
import java.nio.charset.StandardCharsets;
+import org.apache.james.mailbox.model.ContentType;
+import org.apache.james.mailbox.model.ContentType.MediaType;
+import org.apache.james.mailbox.model.ContentType.SubType;
import org.junit.jupiter.api.Test;
class MimePartTest {
@@ -31,8 +34,8 @@ class MimePartTest {
void buildShouldWorkWhenTextualContentFromParserIsEmpty() {
MimePart.builder()
.addBodyContent(new ByteArrayInputStream(new byte[] {}))
- .addMediaType("text")
- .addSubType("plain")
+ .addMediaType(MediaType.of("text"))
+ .addSubType(SubType.of("plain"))
.build();
}
@@ -41,8 +44,8 @@ class MimePartTest {
String body = "text";
MimePart mimePart = MimePart.builder()
.addBodyContent(new ByteArrayInputStream(body.getBytes(StandardCharsets.UTF_8)))
- .addMediaType("text")
- .addSubType("plain")
+ .addMediaType(MediaType.of("text"))
+ .addSubType(SubType.of("plain"))
.build();
assertThat(mimePart.getTextualBody()).contains(body);
diff --git a/mailbox/scanning-search/src/test/java/org/apache/james/mailbox/store/search/PDFTextExtractor.java b/mailbox/scanning-search/src/test/java/org/apache/james/mailbox/store/search/PDFTextExtractor.java
index 8a6f583..8f82685 100644
--- a/mailbox/scanning-search/src/test/java/org/apache/james/mailbox/store/search/PDFTextExtractor.java
+++ b/mailbox/scanning-search/src/test/java/org/apache/james/mailbox/store/search/PDFTextExtractor.java
@@ -26,6 +26,8 @@ import java.util.Optional;
import org.apache.commons.io.IOUtils;
import org.apache.james.mailbox.extractor.ParsedContent;
import org.apache.james.mailbox.extractor.TextExtractor;
+import org.apache.james.mailbox.model.ContentType;
+import org.apache.james.mailbox.model.ContentType.MimeType;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.text.PDFTextStripper;
@@ -34,10 +36,10 @@ import com.google.common.collect.ImmutableMap;
public class PDFTextExtractor implements TextExtractor {
- static final String PDF_TYPE = "application/pdf";
+ static final MimeType PDF_TYPE = MimeType.of("application/pdf");
@Override
- public ParsedContent extractContent(InputStream inputStream, String contentType) throws Exception {
+ public ParsedContent extractContent(InputStream inputStream, ContentType contentType) throws Exception {
Preconditions.checkNotNull(inputStream);
Preconditions.checkNotNull(contentType);
@@ -47,8 +49,8 @@ public class PDFTextExtractor implements TextExtractor {
return new ParsedContent(Optional.ofNullable(IOUtils.toString(inputStream, StandardCharsets.UTF_8)), ImmutableMap.of());
}
- private boolean isPDF(String contentType) {
- return contentType.equals(PDF_TYPE);
+ private boolean isPDF(ContentType contentType) {
+ return contentType.mimeType().equals(PDF_TYPE);
}
private ParsedContent extractTextFromPDF(InputStream inputStream) throws IOException {
diff --git a/mailbox/scanning-search/src/test/java/org/apache/james/mailbox/store/search/PDFTextExtractorTest.java b/mailbox/scanning-search/src/test/java/org/apache/james/mailbox/store/search/PDFTextExtractorTest.java
index 65c28fd..de4bac9 100644
--- a/mailbox/scanning-search/src/test/java/org/apache/james/mailbox/store/search/PDFTextExtractorTest.java
+++ b/mailbox/scanning-search/src/test/java/org/apache/james/mailbox/store/search/PDFTextExtractorTest.java
@@ -26,6 +26,7 @@ import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
+import org.apache.james.mailbox.model.ContentType;
import org.junit.Before;
import org.junit.Test;
@@ -41,7 +42,7 @@ public class PDFTextExtractorTest {
@Test
public void extractContentShouldThrowWhenNullInputStream() throws Exception {
assertThatThrownBy(() ->
- testee.extractContent(null, "any/any"))
+ testee.extractContent(null, ContentType.of("any/any")))
.isInstanceOf(NullPointerException.class);
}
@@ -57,7 +58,7 @@ public class PDFTextExtractorTest {
String content = "content";
InputStream inputStream = new ByteArrayInputStream(content.getBytes(StandardCharsets.UTF_8));
- assertThat(testee.extractContent(inputStream, "text/plain")
+ assertThat(testee.extractContent(inputStream, ContentType.of("text/plain"))
.getTextualContent())
.contains(content);
}
@@ -67,7 +68,7 @@ public class PDFTextExtractorTest {
String content = "Little PDF\n";
InputStream inputStream = ClassLoader.getSystemResourceAsStream("pdf.pdf");
- assertThat(testee.extractContent(inputStream, PDFTextExtractor.PDF_TYPE)
+ assertThat(testee.extractContent(inputStream, ContentType.of(PDFTextExtractor.PDF_TYPE))
.getTextualContent())
.contains(content);
}
diff --git a/mailbox/store/src/main/java/org/apache/james/mailbox/store/extractor/DefaultTextExtractor.java b/mailbox/store/src/main/java/org/apache/james/mailbox/store/extractor/DefaultTextExtractor.java
index 23e574e..ea4d76b 100644
--- a/mailbox/store/src/main/java/org/apache/james/mailbox/store/extractor/DefaultTextExtractor.java
+++ b/mailbox/store/src/main/java/org/apache/james/mailbox/store/extractor/DefaultTextExtractor.java
@@ -27,6 +27,7 @@ import java.util.Optional;
import org.apache.commons.io.IOUtils;
import org.apache.james.mailbox.extractor.ParsedContent;
import org.apache.james.mailbox.extractor.TextExtractor;
+import org.apache.james.mailbox.model.ContentType;
/**
* A default text extractor that is directly based on the input file provided.
@@ -36,8 +37,8 @@ import org.apache.james.mailbox.extractor.TextExtractor;
public class DefaultTextExtractor implements TextExtractor {
@Override
- public ParsedContent extractContent(InputStream inputStream, String contentType) throws Exception {
- if (contentType != null && contentType.startsWith("text/")) {
+ public ParsedContent extractContent(InputStream inputStream, ContentType contentType) throws Exception {
+ if (contentType != null && contentType.asString().startsWith("text/")) {
return new ParsedContent(Optional.ofNullable(IOUtils.toString(inputStream, StandardCharsets.UTF_8)), new HashMap<>());
} else {
return new ParsedContent(Optional.empty(), new HashMap<>());
diff --git a/mailbox/store/src/main/java/org/apache/james/mailbox/store/extractor/JsoupTextExtractor.java b/mailbox/store/src/main/java/org/apache/james/mailbox/store/extractor/JsoupTextExtractor.java
index 06b5d80..5404455 100644
--- a/mailbox/store/src/main/java/org/apache/james/mailbox/store/extractor/JsoupTextExtractor.java
+++ b/mailbox/store/src/main/java/org/apache/james/mailbox/store/extractor/JsoupTextExtractor.java
@@ -29,6 +29,8 @@ import java.util.Optional;
import org.apache.commons.io.IOUtils;
import org.apache.james.mailbox.extractor.ParsedContent;
import org.apache.james.mailbox.extractor.TextExtractor;
+import org.apache.james.mailbox.model.ContentType;
+import org.apache.james.mailbox.model.ContentType.MimeType;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
@@ -38,16 +40,18 @@ public class JsoupTextExtractor implements TextExtractor {
private static final String TITLE_HTML_TAG = "title";
private static final String NO_BASE_URI = "";
private static final Map<String, List<String>> EMPTY_METADATA = ImmutableMap.of();
+ private static final MimeType TEXT_HTML = MimeType.of("text/html");
+ private static final MimeType TEXT_PLAIN = MimeType.of("text/plain");
@Override
- public ParsedContent extractContent(InputStream inputStream, String contentType) throws Exception {
+ public ParsedContent extractContent(InputStream inputStream, ContentType contentType) throws Exception {
if (inputStream == null || contentType == null) {
return ParsedContent.empty();
}
- if (contentType.equals("text/html")) {
+ if (contentType.mimeType().equals(TEXT_HTML)) {
return parseHtmlContent(inputStream);
}
- if (contentType.equals("text/plain")) {
+ if (contentType.mimeType().equals(TEXT_PLAIN)) {
return parsePlainTextContent(inputStream);
}
return ParsedContent.empty();
diff --git a/mailbox/store/src/main/java/org/apache/james/mailbox/store/search/MessageSearches.java b/mailbox/store/src/main/java/org/apache/james/mailbox/store/search/MessageSearches.java
index 690fcbf..583ec25 100644
--- a/mailbox/store/src/main/java/org/apache/james/mailbox/store/search/MessageSearches.java
+++ b/mailbox/store/src/main/java/org/apache/james/mailbox/store/search/MessageSearches.java
@@ -267,8 +267,7 @@ public class MessageSearches implements Iterable<SimpleMessageSearchIndex.Search
return textExtractor
.extractContent(
rawData,
- // todo we likely want only the media type here
- attachment.getType().asString())
+ attachment.getType())
.getTextualContent()
.stream();
} catch (Exception e) {
diff --git a/mailbox/store/src/test/java/org/apache/james/mailbox/store/extractor/DefaultTextExtractorTest.java b/mailbox/store/src/test/java/org/apache/james/mailbox/store/extractor/DefaultTextExtractorTest.java
index 49ae39e..aaac5b9 100644
--- a/mailbox/store/src/test/java/org/apache/james/mailbox/store/extractor/DefaultTextExtractorTest.java
+++ b/mailbox/store/src/test/java/org/apache/james/mailbox/store/extractor/DefaultTextExtractorTest.java
@@ -24,6 +24,7 @@ import static org.assertj.core.api.Assertions.assertThat;
import java.io.InputStream;
import org.apache.james.mailbox.extractor.TextExtractor;
+import org.apache.james.mailbox.model.ContentType;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
@@ -39,18 +40,27 @@ class DefaultTextExtractorTest {
void textTest() throws Exception {
InputStream inputStream = ClassLoader.getSystemResourceAsStream("documents/Text.txt");
assertThat(inputStream).isNotNull();
- assertThat(textExtractor.extractContent(inputStream, "text/plain")
+ assertThat(textExtractor.extractContent(inputStream, ContentType.of("text/plain"))
.getTextualContent())
.contains("This is some awesome text text.\n\n");
}
@Test
+ void extractContentShouldTakeIntoAccountCharset() throws Exception {
+ InputStream inputStream = ClassLoader.getSystemResourceAsStream("documents/simple-text-iso-8859-1.txt");
+ assertThat(inputStream).isNotNull();
+ assertThat(textExtractor.extractContent(inputStream, ContentType.of("text/plain; charset=ISO-8859-1"))
+ .getTextualContent())
+ .contains("\"é\" This text is not UTF-8 \"à\"");
+ }
+
+ @Test
void textMicrosoftWorldTest() throws Exception {
InputStream inputStream = ClassLoader.getSystemResourceAsStream("documents/writter.docx");
assertThat(inputStream).isNotNull();
assertThat(textExtractor.extractContent(
inputStream,
- "application/vnd.openxmlformats-officedocument.wordprocessingml.document")
+ ContentType.of("application/vnd.openxmlformats-officedocument.wordprocessingml.document"))
.getTextualContent())
.isEmpty();
}
diff --git a/mailbox/store/src/test/java/org/apache/james/mailbox/store/extractor/JsoupTextExtractorTest.java b/mailbox/store/src/test/java/org/apache/james/mailbox/store/extractor/JsoupTextExtractorTest.java
index cfdd362..00d00f9 100644
--- a/mailbox/store/src/test/java/org/apache/james/mailbox/store/extractor/JsoupTextExtractorTest.java
+++ b/mailbox/store/src/test/java/org/apache/james/mailbox/store/extractor/JsoupTextExtractorTest.java
@@ -27,12 +27,13 @@ import java.nio.charset.StandardCharsets;
import org.apache.james.mailbox.extractor.ParsedContent;
import org.apache.james.mailbox.extractor.TextExtractor;
+import org.apache.james.mailbox.model.ContentType;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
class JsoupTextExtractorTest {
- private static final String TEXT_HTML_CONTENT_TYPE = "text/html";
+ private static final ContentType TEXT_HTML_CONTENT_TYPE = ContentType.of("text/html");
private static final String HTML_TEXT_CONTENT = "HTML pages can include a lot of null '\0' character. But still expecting the content can be parsed." +
"Jsoup 1.12.1 thinks a file containing more than 10 null characters can be a binary file";
private static final String NULL_CHARACTERS = "\0\0\0\0\0\0\0\0\0\0";
@@ -57,7 +58,15 @@ class JsoupTextExtractorTest {
void extractContentShouldHandlePlainText() throws Exception {
InputStream inputStream = new ByteArrayInputStream("myText".getBytes(StandardCharsets.UTF_8));
- assertThat(textExtractor.extractContent(inputStream, "text/plain").getTextualContent())
+ assertThat(textExtractor.extractContent(inputStream, ContentType.of("text/plain")).getTextualContent())
+ .contains("myText");
+ }
+
+ @Test
+ void extractContentShouldHandlePlainTextWithCharset() throws Exception {
+ InputStream inputStream = new ByteArrayInputStream("myText".getBytes(StandardCharsets.UTF_8));
+
+ assertThat(textExtractor.extractContent(inputStream, ContentType.of("text/plain; charset=utf-8")).getTextualContent())
.contains("myText");
}
@@ -65,7 +74,7 @@ class JsoupTextExtractorTest {
void extractContentShouldHandleArbitraryTextMediaType() throws Exception {
InputStream inputStream = new ByteArrayInputStream("myText".getBytes(StandardCharsets.UTF_8));
- assertThat(textExtractor.extractContent(inputStream, "text/arbitrary").getTextualContent())
+ assertThat(textExtractor.extractContent(inputStream, ContentType.of("text/arbitrary")).getTextualContent())
.isEmpty();
}
@@ -93,6 +102,24 @@ class JsoupTextExtractorTest {
.contains(HTML_TEXT_CONTENT));
}
+ @Test
+ void extractContentShouldTakeIntoAccountCharsetWhenPlainText() throws Exception {
+ InputStream inputStream = ClassLoader.getSystemResourceAsStream("documents/simple-text-iso-8859-1.txt");
+ assertThat(inputStream).isNotNull();
+ assertThat(textExtractor.extractContent(inputStream, ContentType.of("text/plain; charset=ISO-8859-1"))
+ .getTextualContent())
+ .contains("\"é\" This text is not UTF-8 \"à\"");
+ }
+
+ @Test
+ void extractContentTakeIntoAccountCharsetWhenHTML() throws Exception {
+ InputStream inputStream = ClassLoader.getSystemResourceAsStream("documents/html-iso-8859-1.html");
+ assertThat(inputStream).isNotNull();
+ assertThat(textExtractor.extractContent(inputStream, ContentType.of("text/html; charset=ISO-8859-1"))
+ .getTextualContent())
+ .contains("\"é\" this is a simple HTML text \"à\"");
+ }
+
private InputStream textContentWithManyNullCharacters() {
return new ByteArrayInputStream(FULL_CONTENT.getBytes(StandardCharsets.UTF_8));
}
diff --git a/mailbox/api/src/main/java/org/apache/james/mailbox/extractor/TextExtractor.java b/mailbox/store/src/test/resources/documents/html-iso-8859-1.html
similarity index 69%
copy from mailbox/api/src/main/java/org/apache/james/mailbox/extractor/TextExtractor.java
copy to mailbox/store/src/test/resources/documents/html-iso-8859-1.html
index b33d853..ad6c89d 100644
--- a/mailbox/api/src/main/java/org/apache/james/mailbox/extractor/TextExtractor.java
+++ b/mailbox/store/src/test/resources/documents/html-iso-8859-1.html
@@ -1,4 +1,4 @@
-/****************************************************************
+<!--
* Licensed to the Apache Software Foundation (ASF) under one *
* or more contributor license agreements. See the NOTICE file *
* distributed with this work for additional information *
@@ -15,14 +15,17 @@
* KIND, either express or implied. See the License for the *
* specific language governing permissions and limitations *
* under the License. *
- ****************************************************************/
+-->
+<!doctype html>
+<html xmlns="http://www.w3.org/1999/xhtml" xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office">
+ <head>
+ <![endif]-->
+ <meta charset="UTF-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>*|MC:SUBJECT|*</title>
-package org.apache.james.mailbox.extractor;
-
-import java.io.InputStream;
-
-public interface TextExtractor {
-
- ParsedContent extractContent(InputStream inputStream, String contentType) throws Exception;
-
-}
+ <body>
+ <p>"�" this is a simple HTML text "�"</p>
+ </body>
+</html>
diff --git a/mailbox/store/src/test/resources/documents/simple-text-iso-8859-1.txt b/mailbox/store/src/test/resources/documents/simple-text-iso-8859-1.txt
new file mode 100644
index 0000000..3be593c
--- /dev/null
+++ b/mailbox/store/src/test/resources/documents/simple-text-iso-8859-1.txt
@@ -0,0 +1 @@
+"�" This text is not UTF-8 "�"
\ No newline at end of file
diff --git a/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/CachingTextExtractor.java b/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/CachingTextExtractor.java
index 9feaf55..30cd94f 100644
--- a/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/CachingTextExtractor.java
+++ b/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/CachingTextExtractor.java
@@ -29,6 +29,7 @@ import java.util.concurrent.TimeUnit;
import org.apache.commons.io.IOUtils;
import org.apache.james.mailbox.extractor.ParsedContent;
import org.apache.james.mailbox.extractor.TextExtractor;
+import org.apache.james.mailbox.model.ContentType;
import org.apache.james.metrics.api.GaugeRegistry;
import org.apache.james.metrics.api.Metric;
import org.apache.james.metrics.api.MetricFactory;
@@ -108,7 +109,7 @@ public class CachingTextExtractor implements TextExtractor {
}
@Override
- public ParsedContent extractContent(InputStream inputStream, String contentType) throws Exception {
+ public ParsedContent extractContent(InputStream inputStream, ContentType contentType) throws Exception {
byte[] bytes = IOUtils.toByteArray(inputStream);
String key = Hashing.sha256().hashBytes(bytes).toString();
@@ -119,7 +120,7 @@ public class CachingTextExtractor implements TextExtractor {
}
}
- private ParsedContent retrieveAndUpdateWeight(byte[] bytes, String contentType) throws Exception {
+ private ParsedContent retrieveAndUpdateWeight(byte[] bytes, ContentType contentType) throws Exception {
ParsedContent parsedContent = underlying.extractContent(new ByteArrayInputStream(bytes), contentType);
weightMetric.add(computeWeight(parsedContent));
return parsedContent;
diff --git a/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/ContentTypeFilteringTextExtractor.java b/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/ContentTypeFilteringTextExtractor.java
index c648610..d846b75 100644
--- a/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/ContentTypeFilteringTextExtractor.java
+++ b/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/ContentTypeFilteringTextExtractor.java
@@ -23,28 +23,30 @@ import java.io.InputStream;
import org.apache.james.mailbox.extractor.ParsedContent;
import org.apache.james.mailbox.extractor.TextExtractor;
+import org.apache.james.mailbox.model.ContentType;
+import org.apache.james.mailbox.model.ContentType.MimeType;
import com.google.common.collect.ImmutableSet;
public class ContentTypeFilteringTextExtractor implements TextExtractor {
private final TextExtractor textExtractor;
- private final ImmutableSet<String> contentTypeBlacklist;
+ private final ImmutableSet<MimeType> contentTypeBlacklist;
- public ContentTypeFilteringTextExtractor(TextExtractor textExtractor, ImmutableSet<String> contentTypeBlacklist) {
+ public ContentTypeFilteringTextExtractor(TextExtractor textExtractor, ImmutableSet<MimeType> contentTypeBlacklist) {
this.textExtractor = textExtractor;
this.contentTypeBlacklist = contentTypeBlacklist;
}
@Override
- public ParsedContent extractContent(InputStream inputStream, String contentType) throws Exception {
- if (isBlacklisted(contentType)) {
+ public ParsedContent extractContent(InputStream inputStream, ContentType contentType) throws Exception {
+ if (isBlacklisted(contentType.mimeType())) {
return ParsedContent.empty();
}
return textExtractor.extractContent(inputStream, contentType);
}
- private boolean isBlacklisted(String contentType) {
+ private boolean isBlacklisted(MimeType contentType) {
return contentTypeBlacklist.contains(contentType);
}
diff --git a/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaConfiguration.java b/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaConfiguration.java
index e95a8e8..590461c 100644
--- a/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaConfiguration.java
+++ b/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaConfiguration.java
@@ -25,6 +25,7 @@ import java.util.Optional;
import java.util.Set;
import java.util.concurrent.TimeUnit;
+import org.apache.james.mailbox.model.ContentType.MimeType;
import org.apache.james.util.Port;
import com.google.common.base.Preconditions;
@@ -41,7 +42,7 @@ public class TikaConfiguration {
private Optional<Integer> timeoutInMillis;
private Optional<Duration> cacheEvictionPeriod;
private Optional<Long> cacheWeightInBytes;
- private ImmutableSet.Builder<String> contentTypeBlacklist;
+ private ImmutableSet.Builder<MimeType> contentTypeBlacklist;
private Builder() {
isEnabled = Optional.empty();
@@ -140,7 +141,7 @@ public class TikaConfiguration {
return this;
}
- public Builder contentTypeBlacklist(Set<String> contentTypeBlacklist) {
+ public Builder contentTypeBlacklist(Set<MimeType> contentTypeBlacklist) {
Preconditions.checkNotNull(contentTypeBlacklist);
this.contentTypeBlacklist.addAll(contentTypeBlacklist);
return this;
@@ -179,9 +180,9 @@ public class TikaConfiguration {
private final int timeoutInMillis;
private final Duration cacheEvictionPeriod;
private final long cacheWeightInBytes;
- private final ImmutableSet<String> contentTypeBlacklist;
+ private final ImmutableSet<MimeType> contentTypeBlacklist;
- private TikaConfiguration(boolean enabled, boolean cacheEnabled, String host, int port, int timeoutInMillis, Duration cacheEvictionPeriod, long cacheWeightInBytes, ImmutableSet<String> contentTypeBlacklist) {
+ private TikaConfiguration(boolean enabled, boolean cacheEnabled, String host, int port, int timeoutInMillis, Duration cacheEvictionPeriod, long cacheWeightInBytes, ImmutableSet<MimeType> contentTypeBlacklist) {
this.enabled = enabled;
this.cacheEnabled = cacheEnabled;
this.host = host;
@@ -220,7 +221,7 @@ public class TikaConfiguration {
return cacheWeightInBytes;
}
- public ImmutableSet<String> getContentTypeBlacklist() {
+ public ImmutableSet<MimeType> getContentTypeBlacklist() {
return contentTypeBlacklist;
}
diff --git a/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaHttpClient.java b/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaHttpClient.java
index 9e490db..ceae8ff 100644
--- a/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaHttpClient.java
+++ b/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaHttpClient.java
@@ -21,7 +21,9 @@ package org.apache.james.mailbox.tika;
import java.io.InputStream;
import java.util.Optional;
+import org.apache.james.mailbox.model.ContentType;
+
public interface TikaHttpClient {
- Optional<InputStream> recursiveMetaDataAsJson(InputStream inputStream, String contentType);
+ Optional<InputStream> recursiveMetaDataAsJson(InputStream inputStream, ContentType contentType);
}
diff --git a/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaHttpClientImpl.java b/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaHttpClientImpl.java
index 402eb6b..5f00ca9 100644
--- a/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaHttpClientImpl.java
+++ b/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaHttpClientImpl.java
@@ -22,6 +22,7 @@ import java.io.IOException;
import java.io.InputStream;
import java.net.URI;
import java.net.URISyntaxException;
+import java.nio.charset.Charset;
import java.util.Optional;
import org.apache.http.client.fluent.Request;
@@ -52,12 +53,16 @@ public class TikaHttpClientImpl implements TikaHttpClient {
}
@Override
- public Optional<InputStream> recursiveMetaDataAsJson(InputStream inputStream, String contentType) {
+ public Optional<InputStream> recursiveMetaDataAsJson(InputStream inputStream, org.apache.james.mailbox.model.ContentType contentType) {
try {
+ ContentType httpContentType = ContentType.create(contentType.mimeType().asString(),
+ contentType.charset()
+ .map(Charset::name)
+ .orElse(null));
return Optional.ofNullable(
Request.Put(recursiveMetaData)
.socketTimeout(tikaConfiguration.getTimeoutInMillis())
- .bodyStream(inputStream, ContentType.create(contentType))
+ .bodyStream(inputStream, httpContentType)
.execute()
.returnContent()
.asStream());
diff --git a/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaTextExtractor.java b/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaTextExtractor.java
index d12bfa4..b20da09 100644
--- a/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaTextExtractor.java
+++ b/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaTextExtractor.java
@@ -33,6 +33,7 @@ import javax.inject.Inject;
import org.apache.commons.lang3.StringUtils;
import org.apache.james.mailbox.extractor.ParsedContent;
import org.apache.james.mailbox.extractor.TextExtractor;
+import org.apache.james.mailbox.model.ContentType;
import org.apache.james.mailbox.store.extractor.JsoupTextExtractor;
import org.apache.james.metrics.api.MetricFactory;
@@ -56,6 +57,7 @@ import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableMap;
public class TikaTextExtractor implements TextExtractor {
+ private static final ContentType.MediaType TEXT = ContentType.MediaType.of("text");
private final MetricFactory metricFactory;
private final TikaHttpClient tikaHttpClient;
@@ -79,8 +81,8 @@ public class TikaTextExtractor implements TextExtractor {
}
@Override
- public ParsedContent extractContent(InputStream inputStream, String contentType) throws Exception {
- if (contentType.startsWith("text/")) {
+ public ParsedContent extractContent(InputStream inputStream, ContentType contentType) throws Exception {
+ if (contentType.mediaType().equals(TEXT)) {
return jsoupTextExtractor.extractContent(inputStream, contentType);
}
return metricFactory.runPublishingTimerMetric("tikaTextExtraction", Throwing.supplier(
@@ -88,7 +90,7 @@ public class TikaTextExtractor implements TextExtractor {
.sneakyThrow());
}
- public ParsedContent performContentExtraction(InputStream inputStream, String contentType) throws IOException {
+ public ParsedContent performContentExtraction(InputStream inputStream, ContentType contentType) throws IOException {
ContentAndMetadata contentAndMetadata = convert(tikaHttpClient.recursiveMetaDataAsJson(inputStream, contentType));
return new ParsedContent(contentAndMetadata.getContent(), contentAndMetadata.getMetadata());
}
diff --git a/mailbox/tika/src/test/java/org/apache/james/mailbox/tika/CachingTextExtractorTest.java b/mailbox/tika/src/test/java/org/apache/james/mailbox/tika/CachingTextExtractorTest.java
index 791a7ed..6722733 100644
--- a/mailbox/tika/src/test/java/org/apache/james/mailbox/tika/CachingTextExtractorTest.java
+++ b/mailbox/tika/src/test/java/org/apache/james/mailbox/tika/CachingTextExtractorTest.java
@@ -42,6 +42,7 @@ import java.util.stream.IntStream;
import org.apache.james.mailbox.extractor.ParsedContent;
import org.apache.james.mailbox.extractor.TextExtractor;
+import org.apache.james.mailbox.model.ContentType;
import org.apache.james.metrics.api.NoopGaugeRegistry;
import org.apache.james.metrics.tests.RecordingMetricFactory;
import org.apache.james.util.concurrency.ConcurrentTestRunner;
@@ -62,7 +63,7 @@ public class CachingTextExtractorTest {
i -> new ByteArrayInputStream(String.format("content%d", i).getBytes(StandardCharsets.UTF_8));
private static final Supplier<InputStream> INPUT_STREAM = () -> STREAM_GENERATOR.apply(1);
private static final long CACHE_LIMIT_10_MiB = 10 * 1024 * 1024;
- private static final String CONTENT_TYPE = "application/bytes";
+ private static final ContentType CONTENT_TYPE = ContentType.of("application/bytes");
private CachingTextExtractor textExtractor;
private TextExtractor wrappedTextExtractor;
diff --git a/mailbox/tika/src/test/java/org/apache/james/mailbox/tika/ContentTypeFilteringTextExtractorTest.java b/mailbox/tika/src/test/java/org/apache/james/mailbox/tika/ContentTypeFilteringTextExtractorTest.java
index ebea68c..89825a2 100644
--- a/mailbox/tika/src/test/java/org/apache/james/mailbox/tika/ContentTypeFilteringTextExtractorTest.java
+++ b/mailbox/tika/src/test/java/org/apache/james/mailbox/tika/ContentTypeFilteringTextExtractorTest.java
@@ -31,6 +31,8 @@ import java.nio.charset.StandardCharsets;
import org.apache.commons.io.IOUtils;
import org.apache.james.mailbox.extractor.ParsedContent;
import org.apache.james.mailbox.extractor.TextExtractor;
+import org.apache.james.mailbox.model.ContentType;
+import org.apache.james.mailbox.model.ContentType.MimeType;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.mockito.Mock;
@@ -52,10 +54,22 @@ class ContentTypeFilteringTextExtractorTest {
void extractContentReturnEmptyWithContentTypeInBlacklist() throws Exception {
ContentTypeFilteringTextExtractor contentTypeFilteringTextExtractor =
new ContentTypeFilteringTextExtractor(textExtractor,
- ImmutableSet.of("application/ics", "application/zip"));
+ ImmutableSet.of(MimeType.of("application/ics"), MimeType.of("application/zip")));
assertThat(contentTypeFilteringTextExtractor
- .extractContent(IOUtils.toInputStream("", StandardCharsets.UTF_8), "application/ics"))
+ .extractContent(IOUtils.toInputStream("", StandardCharsets.UTF_8), ContentType.of("application/ics")))
+ .isEqualTo(ParsedContent.empty());
+ verifyNoMoreInteractions(textExtractor);
+ }
+
+ @Test
+ void extractContentShouldIgnoreContentTypeParameters() throws Exception {
+ ContentTypeFilteringTextExtractor contentTypeFilteringTextExtractor =
+ new ContentTypeFilteringTextExtractor(textExtractor,
+ ImmutableSet.of(MimeType.of("application/ics"), MimeType.of("application/zip")));
+
+ assertThat(contentTypeFilteringTextExtractor
+ .extractContent(IOUtils.toInputStream("", StandardCharsets.UTF_8), ContentType.of("application/ics; charset=utf-8")))
.isEqualTo(ParsedContent.empty());
verifyNoMoreInteractions(textExtractor);
}
@@ -65,8 +79,8 @@ class ContentTypeFilteringTextExtractorTest {
InputStream inputStream = ClassLoader.getSystemResourceAsStream("documents/Text.txt");
ContentTypeFilteringTextExtractor contentTypeFilteringTextExtractor =
new ContentTypeFilteringTextExtractor(textExtractor,
- ImmutableSet.of("application/ics", "application/zip"));
- contentTypeFilteringTextExtractor.extractContent(inputStream, "text/plain");
+ ImmutableSet.of(MimeType.of("application/ics"), MimeType.of("application/zip")));
+ contentTypeFilteringTextExtractor.extractContent(inputStream, ContentType.of("text/plain"));
verify(textExtractor, times(1)).extractContent(any(), any());
}
diff --git a/mailbox/tika/src/test/java/org/apache/james/mailbox/tika/TikaTextExtractorTest.java b/mailbox/tika/src/test/java/org/apache/james/mailbox/tika/TikaTextExtractorTest.java
index 4ecde3d..e47cee2 100644
--- a/mailbox/tika/src/test/java/org/apache/james/mailbox/tika/TikaTextExtractorTest.java
+++ b/mailbox/tika/src/test/java/org/apache/james/mailbox/tika/TikaTextExtractorTest.java
@@ -31,6 +31,7 @@ import java.util.Optional;
import org.apache.commons.io.IOUtils;
import org.apache.james.mailbox.extractor.ParsedContent;
import org.apache.james.mailbox.extractor.TextExtractor;
+import org.apache.james.mailbox.model.ContentType;
import org.apache.james.mailbox.tika.TikaTextExtractor.ContentAndMetadataDeserializer;
import org.apache.james.metrics.tests.RecordingMetricFactory;
import org.junit.jupiter.api.BeforeEach;
@@ -59,7 +60,8 @@ class TikaTextExtractorTest {
@Test
void textualContentShouldReturnEmptyWhenInputStreamIsEmpty() throws Exception {
- assertThat(textExtractor.extractContent(IOUtils.toInputStream("", StandardCharsets.UTF_8), "text/plain").getTextualContent())
+ assertThat(textExtractor.extractContent(IOUtils.toInputStream("", StandardCharsets.UTF_8), ContentType.of("text/plain"))
+ .getTextualContent())
.contains("");
}
@@ -67,7 +69,7 @@ class TikaTextExtractorTest {
void textTest() throws Exception {
InputStream inputStream = ClassLoader.getSystemResourceAsStream("documents/Text.txt");
assertThat(inputStream).isNotNull();
- assertThat(textExtractor.extractContent(inputStream, "text/plain").getTextualContent())
+ assertThat(textExtractor.extractContent(inputStream, ContentType.of("text/plain")).getTextualContent())
.isPresent()
.asString()
.contains("This is some awesome text text.");
@@ -77,7 +79,9 @@ class TikaTextExtractorTest {
void textMicrosoftWorldTest() throws Exception {
InputStream inputStream = ClassLoader.getSystemResourceAsStream("documents/writter.docx");
assertThat(inputStream).isNotNull();
- assertThat(textExtractor.extractContent(inputStream, "application/vnd.openxmlformats-officedocument.wordprocessingml.document").getTextualContent())
+ assertThat(textExtractor.extractContent(inputStream,
+ ContentType.of("application/vnd.openxmlformats-officedocument.wordprocessingml.document"))
+ .getTextualContent())
.isPresent()
.asString()
.contains("This is an awesome document on libroffice writter !");
@@ -87,7 +91,8 @@ class TikaTextExtractorTest {
void textOdtTest() throws Exception {
InputStream inputStream = ClassLoader.getSystemResourceAsStream("documents/writter.odt");
assertThat(inputStream).isNotNull();
- assertThat(textExtractor.extractContent(inputStream, "application/vnd.oasis.opendocument.text").getTextualContent())
+ assertThat(textExtractor.extractContent(inputStream, ContentType.of("application/vnd.oasis.opendocument.text"))
+ .getTextualContent())
.isPresent()
.asString()
.contains("This is an awesome document on libroffice writter !");
@@ -97,7 +102,8 @@ class TikaTextExtractorTest {
void documentWithBadDeclaredMetadataShouldBeWellHandled() throws Exception {
InputStream inputStream = ClassLoader.getSystemResourceAsStream("documents/fake.txt");
assertThat(inputStream).isNotNull();
- assertThat(textExtractor.extractContent(inputStream, "application/vnd.oasis.opendocument.text").getTextualContent())
+ assertThat(textExtractor.extractContent(inputStream, ContentType.of("application/vnd.oasis.opendocument.text"))
+ .getTextualContent())
.isPresent()
.asString()
.contains("This is an awesome document on libroffice writter !");
@@ -107,7 +113,7 @@ class TikaTextExtractorTest {
void slidePowerPointTest() throws Exception {
InputStream inputStream = ClassLoader.getSystemResourceAsStream("documents/slides.pptx");
assertThat(inputStream).isNotNull();
- assertThat(textExtractor.extractContent(inputStream, "application/vnd.openxmlformats-officedocument.presentationml.presentation").getTextualContent())
+ assertThat(textExtractor.extractContent(inputStream, ContentType.of("application/vnd.openxmlformats-officedocument.presentationml.presentation")).getTextualContent())
.isPresent()
.asString()
.contains("James is awesome")
@@ -118,7 +124,8 @@ class TikaTextExtractorTest {
void slideOdpTest() throws Exception {
InputStream inputStream = ClassLoader.getSystemResourceAsStream("documents/slides.odp");
assertThat(inputStream).isNotNull();
- assertThat(textExtractor.extractContent(inputStream, "application/vnd.oasis.opendocument.presentation").getTextualContent())
+ assertThat(textExtractor.extractContent(inputStream, ContentType.of("application/vnd.oasis.opendocument.presentation"))
+ .getTextualContent())
.isPresent()
.asString()
.contains("James is awesome")
@@ -129,7 +136,8 @@ class TikaTextExtractorTest {
void pdfTest() throws Exception {
InputStream inputStream = ClassLoader.getSystemResourceAsStream("documents/PDF.pdf");
assertThat(inputStream).isNotNull();
- assertThat(textExtractor.extractContent(inputStream, "application/pdf").getTextualContent())
+ assertThat(textExtractor.extractContent(inputStream, ContentType.of("application/pdf"))
+ .getTextualContent())
.isPresent()
.asString()
.contains("This is an awesome document on libroffice writter !");
@@ -139,7 +147,8 @@ class TikaTextExtractorTest {
void odsTest() throws Exception {
InputStream inputStream = ClassLoader.getSystemResourceAsStream("documents/calc.ods");
assertThat(inputStream).isNotNull();
- assertThat(textExtractor.extractContent(inputStream, "application/vnd.oasis.opendocument.spreadsheet").getTextualContent())
+ assertThat(textExtractor.extractContent(inputStream, ContentType.of("application/vnd.oasis.opendocument.spreadsheet"))
+ .getTextualContent())
.isPresent()
.asString()
.contains("This is an aesome LibreOffice document !");
@@ -149,7 +158,8 @@ class TikaTextExtractorTest {
void excelTest() throws Exception {
InputStream inputStream = ClassLoader.getSystemResourceAsStream("documents/calc.xlsx");
assertThat(inputStream).isNotNull();
- assertThat(textExtractor.extractContent(inputStream, "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet").getTextualContent())
+ assertThat(textExtractor.extractContent(inputStream, ContentType.of("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"))
+ .getTextualContent())
.isPresent()
.asString()
.contains("Feuille1")
@@ -165,7 +175,7 @@ class TikaTextExtractorTest {
.getBytes(StandardCharsets.UTF_8))));
InputStream inputStream = null;
- textExtractor.extractContent(inputStream, "text/plain");
+ textExtractor.extractContent(inputStream, ContentType.of("text/plain"));
}
@Test
@@ -178,7 +188,8 @@ class TikaTextExtractorTest {
.getBytes(StandardCharsets.UTF_8))));
InputStream inputStream = new ByteArrayInputStream("toto".getBytes(StandardCharsets.UTF_8));
- ParsedContent parsedContent = textExtractor.extractContent(inputStream, "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet");
+ ParsedContent parsedContent = textExtractor.extractContent(inputStream,
+ ContentType.of("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"));
assertThat(parsedContent.getTextualContent()).contains(expectedExtractedContent);
}
@@ -192,7 +203,8 @@ class TikaTextExtractorTest {
InputStream inputStream = new ByteArrayInputStream("toto".getBytes(StandardCharsets.UTF_8));
- assertThatThrownBy(() -> textExtractor.extractContent(inputStream, "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"))
+ assertThatThrownBy(() -> textExtractor.extractContent(inputStream,
+ ContentType.of("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet")))
.isInstanceOf(IllegalStateException.class)
.hasMessage("The element should be a Json object");
}
diff --git a/server/container/guice/cassandra-guice/src/main/java/org/apache/james/modules/mailbox/TikaConfigurationReader.java b/server/container/guice/cassandra-guice/src/main/java/org/apache/james/modules/mailbox/TikaConfigurationReader.java
index ae7bc0e..341f838 100644
--- a/server/container/guice/cassandra-guice/src/main/java/org/apache/james/modules/mailbox/TikaConfigurationReader.java
+++ b/server/container/guice/cassandra-guice/src/main/java/org/apache/james/modules/mailbox/TikaConfigurationReader.java
@@ -25,6 +25,7 @@ import java.util.Optional;
import java.util.Set;
import org.apache.commons.configuration2.Configuration;
+import org.apache.james.mailbox.model.ContentType.MimeType;
import org.apache.james.mailbox.tika.TikaConfiguration;
import org.apache.james.util.DurationParser;
import org.apache.james.util.Size;
@@ -69,9 +70,10 @@ public class TikaConfigurationReader {
.map(Throwing.function(Size::parse))
.map(Size::asBytes);
- Set<String> contentTypeBlacklist = StreamUtils
+ Set<MimeType> contentTypeBlacklist = StreamUtils
.ofNullable(configuration.getStringArray(TIKA_CONTENT_TYPE_BLACKLIST))
.map(String::trim)
+ .map(MimeType::of)
.collect(ImmutableSet.toImmutableSet());
return TikaConfiguration.builder()
diff --git a/server/container/guice/cassandra-guice/src/test/java/org/apache/james/modules/mailbox/TikaConfigurationReaderTest.java b/server/container/guice/cassandra-guice/src/test/java/org/apache/james/modules/mailbox/TikaConfigurationReaderTest.java
index 03b8a99..aefa878 100644
--- a/server/container/guice/cassandra-guice/src/test/java/org/apache/james/modules/mailbox/TikaConfigurationReaderTest.java
+++ b/server/container/guice/cassandra-guice/src/test/java/org/apache/james/modules/mailbox/TikaConfigurationReaderTest.java
@@ -26,6 +26,7 @@ import java.time.Duration;
import org.apache.commons.configuration2.PropertiesConfiguration;
import org.apache.commons.configuration2.convert.DefaultListDelimiterHandler;
+import org.apache.james.mailbox.model.ContentType.MimeType;
import org.apache.james.mailbox.tika.TikaConfiguration;
import org.junit.Test;
@@ -272,7 +273,7 @@ public class TikaConfigurationReaderTest {
.port(889)
.timeoutInMillis(500)
.cacheWeightInBytes(1520000)
- .contentTypeBlacklist(ImmutableSet.of("application/ics", "application/zip"))
+ .contentTypeBlacklist(ImmutableSet.of(MimeType.of("application/ics"), MimeType.of("application/zip")))
.build());
}
@@ -297,7 +298,7 @@ public class TikaConfigurationReaderTest {
.port(889)
.timeoutInMillis(500)
.cacheWeightInBytes(1520000)
- .contentTypeBlacklist(ImmutableSet.of("application/ics", "application/zip"))
+ .contentTypeBlacklist(ImmutableSet.of(MimeType.of("application/ics"), MimeType.of("application/zip")))
.build());
}
diff --git a/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/draft/methods/MIMEMessageConverter.java b/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/draft/methods/MIMEMessageConverter.java
index a7c9dc7..64f9dd3 100644
--- a/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/draft/methods/MIMEMessageConverter.java
+++ b/server/protocols/jmap-draft/src/main/java/org/apache/james/jmap/draft/methods/MIMEMessageConverter.java
@@ -328,9 +328,7 @@ public class MIMEMessageConverter {
}
private ContentTypeField contentTypeField(MessageAttachmentMetadata att) {
- // todo mailbox pojo should be able to expose itself as a mime4j object
- String type = att.getAttachment().getType().asString();
- ContentTypeField typeAsField = Fields.contentType(type);
+ ContentTypeField typeAsField = att.getAttachment().getType().asMime4J();
if (att.getName().isPresent()) {
return Fields.contentType(typeAsField.getMimeType(),
ImmutableMap.<String, String>builder()
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[james-project] 10/12: JAMES-3105 Expose
trustMessageDenormalization over webadmin
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 7e873ebef6c542a857fb7d9ed2e35a67d19b0007
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Thu Apr 23 17:36:55 2020 +0700
JAMES-3105 Expose trustMessageDenormalization over webadmin
`trustMessageDenormalization` query parameter can be set to `true`. This will result in a better performance running the
task at the cost of safety in the face of message denormalization inconsistencies. Defaults to false, which generates
additional checks.
---
.../mail/task/RecomputeMailboxCountersService.java | 15 +++++
.../task/RecomputeMailboxCountersServiceTest.java | 18 ++++--
.../RecomputeMailboxCountersRequestToTask.java | 32 ++++++++++-
.../RecomputeMailboxCountersRequestToTaskTest.java | 66 ++++++++++++++++++++++
src/site/markdown/server/manage-webadmin.md | 4 ++
5 files changed, 129 insertions(+), 6 deletions(-)
diff --git a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java
index b39db27..7970a32 100644
--- a/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java
+++ b/mailbox/cassandra/src/main/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersService.java
@@ -77,6 +77,21 @@ public class RecomputeMailboxCountersService {
public boolean isMessageDenormalizationTrusted() {
return trustMessageDenormalization;
}
+
+ @Override
+ public final boolean equals(Object o) {
+ if (o instanceof Options) {
+ Options options = (Options) o;
+
+ return Objects.equals(this.trustMessageDenormalization, options.trustMessageDenormalization);
+ }
+ return false;
+ }
+
+ @Override
+ public final int hashCode() {
+ return Objects.hash(trustMessageDenormalization);
+ }
}
private static class Counter {
diff --git a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersServiceTest.java b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersServiceTest.java
index 10382a8..4df80a4 100644
--- a/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersServiceTest.java
+++ b/mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/task/RecomputeMailboxCountersServiceTest.java
@@ -39,6 +39,7 @@ import org.apache.james.mailbox.cassandra.mail.CassandraMailboxDAO;
import org.apache.james.mailbox.cassandra.mail.CassandraMessageIdDAO;
import org.apache.james.mailbox.cassandra.mail.CassandraMessageIdToImapUidDAO;
import org.apache.james.mailbox.cassandra.mail.task.RecomputeMailboxCountersService.Context;
+import org.apache.james.mailbox.cassandra.mail.task.RecomputeMailboxCountersService.Options;
import org.apache.james.mailbox.cassandra.modules.CassandraAclModule;
import org.apache.james.mailbox.cassandra.modules.CassandraMailboxCounterModule;
import org.apache.james.mailbox.cassandra.modules.CassandraMailboxModule;
@@ -56,6 +57,8 @@ import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
+import nl.jqno.equalsverifier.EqualsVerifier;
+
class RecomputeMailboxCountersServiceTest {
private static final UidValidity UID_VALIDITY_1 = UidValidity.of(145);
private static final Username USER = Username.of("user");
@@ -91,6 +94,11 @@ class RecomputeMailboxCountersServiceTest {
testee = new RecomputeMailboxCountersService(mailboxDAO, imapUidToMessageIdDAO, messageIdToImapUidDAO, counterDAO);
}
+ @Test
+ void optionsShouldMatchBeanContract() {
+ EqualsVerifier.forClass(Options.class).verify();
+ }
+
@Nested
class TrustMessageDenormalizationTest implements Contract {
@Override
@@ -104,8 +112,8 @@ class RecomputeMailboxCountersServiceTest {
}
@Override
- public RecomputeMailboxCountersService.Options options() {
- return RecomputeMailboxCountersService.Options.trustMessageDenormalization();
+ public Options options() {
+ return Options.trustMessageDenormalization();
}
@Override
@@ -155,8 +163,8 @@ class RecomputeMailboxCountersServiceTest {
}
@Override
- public RecomputeMailboxCountersService.Options options() {
- return RecomputeMailboxCountersService.Options.recheckMessageDenormalization();
+ public Options options() {
+ return Options.recheckMessageDenormalization();
}
@Override
@@ -180,7 +188,7 @@ class RecomputeMailboxCountersServiceTest {
CassandraMailboxDAO mailboxDAO();
- RecomputeMailboxCountersService.Options options();
+ Options options();
CassandraMessageIdDAO imapUidToMessageIdDAO();
diff --git a/server/protocols/webadmin/webadmin-cassandra/src/main/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTask.java b/server/protocols/webadmin/webadmin-cassandra/src/main/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTask.java
index 2d60e61..a090913 100644
--- a/server/protocols/webadmin/webadmin-cassandra/src/main/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTask.java
+++ b/server/protocols/webadmin/webadmin-cassandra/src/main/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTask.java
@@ -19,6 +19,8 @@
package org.apache.james.webadmin.routes;
+import java.util.Optional;
+
import javax.inject.Inject;
import org.apache.james.mailbox.cassandra.mail.task.RecomputeMailboxCountersService;
@@ -27,12 +29,40 @@ import org.apache.james.mailbox.cassandra.mail.task.RecomputeMailboxCountersTask
import org.apache.james.webadmin.tasks.TaskFromRequestRegistry;
import org.apache.james.webadmin.tasks.TaskRegistrationKey;
+import com.google.common.annotations.VisibleForTesting;
+import com.google.common.base.Preconditions;
+
+import spark.Request;
+
public class RecomputeMailboxCountersRequestToTask extends TaskFromRequestRegistry.TaskRegistration {
private static final TaskRegistrationKey REGISTRATION_KEY = TaskRegistrationKey.of("RecomputeMailboxCounters");
+ public static final String TRUST_PARAM = "trustMessageDenormalization";
@Inject
public RecomputeMailboxCountersRequestToTask(RecomputeMailboxCountersService service) {
super(REGISTRATION_KEY,
- request -> new RecomputeMailboxCountersTask(service, Options.recheckMessageDenormalization()));
+ request -> new RecomputeMailboxCountersTask(service, parseOptions(request)));
+ }
+
+ private static Options parseOptions(Request request) {
+ var stringValue = Optional.ofNullable(request.queryParams(TRUST_PARAM));
+ return parseOptions(stringValue);
+ }
+
+ @VisibleForTesting
+ static Options parseOptions(Optional<String> stringValue) {
+ return stringValue
+ .map(RecomputeMailboxCountersRequestToTask::parseOptions)
+ .orElse(Options.recheckMessageDenormalization());
+ }
+
+ private static Options parseOptions(String stringValue) {
+ Preconditions.checkArgument(isValid(stringValue), "'%s' needs to be a valid boolean", TRUST_PARAM);
+ return Options.of(Boolean.valueOf(stringValue));
+ }
+
+ private static boolean isValid(String stringValue) {
+ return stringValue.equalsIgnoreCase("true")
+ || stringValue.equalsIgnoreCase("false");
}
}
diff --git a/server/protocols/webadmin/webadmin-cassandra/src/test/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTaskTest.java b/server/protocols/webadmin/webadmin-cassandra/src/test/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTaskTest.java
new file mode 100644
index 0000000..780fb71
--- /dev/null
+++ b/server/protocols/webadmin/webadmin-cassandra/src/test/java/org/apache/james/webadmin/routes/RecomputeMailboxCountersRequestToTaskTest.java
@@ -0,0 +1,66 @@
+/****************************************************************
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information *
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the *
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the *
+ * specific language governing permissions and limitations *
+ * under the License. *
+ ****************************************************************/
+
+package org.apache.james.webadmin.routes;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+import java.util.Optional;
+
+import org.apache.james.mailbox.cassandra.mail.task.RecomputeMailboxCountersService.Options;
+import org.junit.jupiter.api.Test;
+
+class RecomputeMailboxCountersRequestToTaskTest {
+ @Test
+ void parseOptionsShouldReturnRecheckWhenEmpty() {
+ assertThat(RecomputeMailboxCountersRequestToTask.parseOptions(Optional.empty()))
+ .isEqualTo(Options.recheckMessageDenormalization());
+ }
+
+ @Test
+ void parseOptionsShouldReturnRecheckWhenFalse() {
+ assertThat(RecomputeMailboxCountersRequestToTask.parseOptions(Optional.of("false")))
+ .isEqualTo(Options.recheckMessageDenormalization());
+ }
+
+ @Test
+ void parseOptionsShouldBeCaseIncentive() {
+ assertThat(RecomputeMailboxCountersRequestToTask.parseOptions(Optional.of("False")))
+ .isEqualTo(Options.recheckMessageDenormalization());
+ }
+
+ @Test
+ void parseOptionsShouldReturnTrueWhenTrust() {
+ assertThat(RecomputeMailboxCountersRequestToTask.parseOptions(Optional.of("true")))
+ .isEqualTo(Options.trustMessageDenormalization());
+ }
+
+ @Test
+ void parseOptionsShouldFailWhenEmpty() {
+ assertThatThrownBy(() -> RecomputeMailboxCountersRequestToTask.parseOptions(Optional.of("")))
+ .isInstanceOf(IllegalArgumentException.class);
+ }
+
+ @Test
+ void parseOptionsShouldFailWhenNotAValidBoolean() {
+ assertThatThrownBy(() -> RecomputeMailboxCountersRequestToTask.parseOptions(Optional.of("zz")))
+ .isInstanceOf(IllegalArgumentException.class);
+ }
+}
\ No newline at end of file
diff --git a/src/site/markdown/server/manage-webadmin.md b/src/site/markdown/server/manage-webadmin.md
index 384d4f6..e3823fa 100644
--- a/src/site/markdown/server/manage-webadmin.md
+++ b/src/site/markdown/server/manage-webadmin.md
@@ -491,6 +491,10 @@ Rerunning the task will *eventually* provide the consistent result. As such we a
In order to ensure being offline, stop the traffic on SMTP, JMAP and IMAP ports, for example via re-configuration or
firewall rules.
+`trustMessageDenormalization` query parameter can be set to `true`. This will result in a better performance running the
+task at the cost of safety in the face of message denormalization inconsistencies. Defaults to false, which generates
+additional checks.
+
#### Recomputing Global JMAP fast message view projection
This action is only available for backends supporting JMAP protocol.
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[james-project] 07/12: JAMES-2997 Strong type for ContentType Field
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit ad5b0b6e3c5f0e90a1630a6f8a99d5454537707f
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Tue Apr 28 16:16:01 2020 +0700
JAMES-2997 Strong type for ContentType Field
---
.../src/main/java/org/apache/james/mailbox/model/ContentType.java | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/mailbox/api/src/main/java/org/apache/james/mailbox/model/ContentType.java b/mailbox/api/src/main/java/org/apache/james/mailbox/model/ContentType.java
index 09f1002..228d646 100644
--- a/mailbox/api/src/main/java/org/apache/james/mailbox/model/ContentType.java
+++ b/mailbox/api/src/main/java/org/apache/james/mailbox/model/ContentType.java
@@ -38,7 +38,7 @@ import com.google.common.base.Strings;
* Follows syntax and usage as defined in https://tools.ietf.org/html/rfc2045#section-5
* Thus includes mime type, defined by its media type and subtype as well as contentType fields parameters,
* including charset
- *
+ *
* Example: text/plain; charset=utf-8
*/
public class ContentType {
@@ -228,13 +228,13 @@ public class ContentType {
if (o instanceof ContentType) {
ContentType that = (ContentType) o;
- return java.util.Objects.equals(this.value, that.value);
+ return Objects.equals(this.value, that.value);
}
return false;
}
@Override
public final int hashCode() {
- return java.util.Objects.hash(value);
+ return Objects.hash(value);
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[james-project] 08/12: JAMES-2997 Specifiy charset for some emails
missing them
Posted by bt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
btellier pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/james-project.git
commit 2af5c40ea8ead11a391cb9efa6d521ab34944b98
Author: Benoit Tellier <bt...@linagora.com>
AuthorDate: Wed Apr 29 15:40:31 2020 +0700
JAMES-2997 Specifiy charset for some emails missing them
---
.../store/src/test/resources/eml/emailWithNonIndexableAttachment.eml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/mailbox/store/src/test/resources/eml/emailWithNonIndexableAttachment.eml b/mailbox/store/src/test/resources/eml/emailWithNonIndexableAttachment.eml
index 2dae3f2..bb15079 100644
--- a/mailbox/store/src/test/resources/eml/emailWithNonIndexableAttachment.eml
+++ b/mailbox/store/src/test/resources/eml/emailWithNonIndexableAttachment.eml
@@ -11,7 +11,7 @@ Message-ID: 1480502737913
Date: Wed, 30 Nov 2016 10:45:38 +0000
---=Part.0.16b49c355c1c1b54.158b4d6fd1b.bd3741f90275bb0a=-
-Content-Type: text/html; name="=?US-ASCII?Q?test.html?="
+Content-Type: text/html; charset=utf-8; name="=?US-ASCII?Q?test.html?="
Content-Disposition: attachment
Content-Transfer-Encoding: base64
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org