You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by "Benoit Tellier (Jira)" <se...@james.apache.org> on 2022/05/23 02:50:00 UTC
[jira] [Closed] (JAMES-3769) ElasticSearch search overrides
[ https://issues.apache.org/jira/browse/JAMES-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benoit Tellier closed JAMES-3769.
---------------------------------
Resolution: Fixed
> ElasticSearch search overrides
> ------------------------------
>
> Key: JAMES-3769
> URL: https://issues.apache.org/jira/browse/JAMES-3769
> Project: James Server
> Issue Type: Improvement
> Components: elasticsearch
> Reporter: Benoit Tellier
> Priority: Major
> Attachments: 74cec3c3-305a-4dd9-a2bd-f54ff210ecfc.png
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> h3. Context
> IMAP SEARCH requests can of course be used for user searches. But many applications (eg Samsung email client) also uses
> IMAP SEARCH for resynchronization.
> The use of ElasticSearch in the context of resynchronization is a problem as:
> - Resynchronization incurs many requests that ElasticSearch is not made to cope with.
> - This have a performance impact and some 60+ seconds can be spotted.
> - ElasticSearch indexing is asynchronous wich is not ideal for resynchronization
> - An ElasticSearch downtime would imply a full downtime for these clients.
>
> Those resynchronization queries would better be served by the metadata database
> h3. Decision
> Define an API, `SearchOverride` that can be called on matching search queries and thus offload ElasticSearch.
> Provide an extension mechanism for search overrides, where potentially custom implementations can be loaded from the
> configuration.
> Provide well targeted `SearchOverride`, implemented on top of `Cassandra` database.
> h3. Consequences
> We expect to :
> - Observe less IMAP SEARCH slow requests
> - ElasticSearch downtime should not impact IMAP clients using IMAP SEARCH like IMAP Samsung application
> - Thus also be able to support a higher IMAP workload as Cassandra is more suited to resynchronisation requests
> ## Sample IMAP requests
> {code:java}
> SearchOperation{key=SearchKey{type=TYPE_UID, date=null, size=0, value=null, seconds=-1, modSeq=-1, uids=[IdRange : TYPE: FROM UID: MessageUid{uid=1}:MessageUid{uid=9223372036854775807}], sequences=null, keys=Optional.empty}, options=[]}
> {code}
> {code:java}
> SearchOperation{key=SearchKey{type=TYPE_AND, date=null, size=0, value=null, seconds=-1, modSeq=-1, uids=null, sequences=null, keys=Optional[[
> SearchKey{type=TYPE_SEQUENCE_SET, date=null, size=0, value=null, seconds=-1, modSeq=-1, uids=null, sequences=[IdRange ( 1->9223372036854775807 )], keys=Optional.empty},
> SearchKey{type=TYPE_DELETED, date=null, size=0, value=null, seconds=-1, modSeq=-1, uids=null, sequences=null, keys=Optional.empty}]]},
> options=[COUNT]}
> {code}
> {code:java}
> SearchOperation{key=SearchKey{type=TYPE_AND, date=null, size=0, value=null, seconds=-1, modSeq=-1, uids=null, sequences=null, keys=Optional[[
> SearchKey{type=TYPE_UID, date=null, size=0, value=null, seconds=-1, modSeq=-1, uids=[IdRange : TYPE: FROM UID: MessageUid{uid=1}:MessageUid{uid=9223372036854775807}], sequences=null, keys=Optional.empty},
> SearchKey{type=TYPE_UNSEEN, date=null, size=0, value=null, seconds=-1, modSeq=-1, uids=null, sequences=null, keys=Optional.empty}]]}, options=[COUNT]}
> {code}
> {code:java}
> SearchOperation{key=SearchKey{type=TYPE_AND, date=null, size=0, value=null, seconds=-1, modSeq=-1, uids=null, sequences=null, keys=Optional[[
> SearchKey{type=TYPE_SEQUENCE_SET, date=null, size=0, value=null, seconds=-1, modSeq=-1, uids=null, sequences=[IdRange ( 1->9223372036854775807 )], keys=Optional.empty},
> SearchKey{type=TYPE_NOT, date=null, size=0, value=null, seconds=-1, modSeq=-1, uids=null, sequences=null, keys=Optional[[
> SearchKey{type=TYPE_DELETED, date=null, size=0, value=null, seconds=-1, modSeq=-1, uids=null, sequences=null, keys=Optional.empty}]]}]]}, options=[ALL]}
> {code}
> {code:java}
> SearchOperation{key=SearchKey{type=TYPE_ALL, date=null, size=0, value=null, seconds=-1, modSeq=-1, uids=null, sequences=null, keys=Optional.empty}, options=[]}
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org