[jira] [Commented] (JAMES-3435) Relaxing LWT usage: domain, users

["Benoit Tellier (Jira)" <se...@james.apache.org>]

    [ https://issues.apache.org/jira/browse/JAMES-3435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17316850#comment-17316850 ] 

Benoit Tellier commented on JAMES-3435:
---------------------------------------

I merged https://github.com/apache/james-project/pull/350

I offers the following configuration options:

{code:java}
| mailbox.read.strong.consistency
| Optional. Boolean, defaults to true. Disabling should be considered experimental.
If enabled, regular consistency level is used for read transactions for mailbox. Not doing so might result
in stale reads as the system.paxos table will not be checked for latest updates. Better performance are expected
by turning it off. Note that reads performed as part of write transactions are always performed with a strong
consistency.

| message.read.strong.consistency
| Optional. Boolean, defaults to true. Disabling should be considered experimental.
If enabled, regular consistency level is used for read transactions for message. Not doing so might result
in stale reads as the system.paxos table will not be checked for latest updates. Better performance are expected
by turning it off. Note that reads performed as part of write transactions are always performed with a strong
consistency.

| message.write.strong.consistency.unsafe
| Optional. Boolean, defaults to true. Disabling should be considered experimental and unsafe.
If enabled, Lightweight transactions will no longer be used upon messages operation (table `imapUidTable`).
As message flags updates relies so far on a read-before-write model, it exposes yourself to data races leading to
potentially update loss. Better performance are expected
by turning it off. Reads performed as part of write transaction are also performed with a relaxed consistency.
{code}


> Relaxing LWT usage: domain, users
> ---------------------------------
>
>                 Key: JAMES-3435
>                 URL: https://issues.apache.org/jira/browse/JAMES-3435
>             Project: James Server
>          Issue Type: Improvement
>          Components: cassandra
>    Affects Versions: master
>            Reporter: Benoit Tellier
>            Priority: Major
>             Fix For: master
>
>          Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> https://www.mail-archive.com/server-dev@james.apache.org/msg68713.html
> {code:java}
> Cassandra is an eventually consistent datastore, that can be used in a
> consistant fashion. To do so, we rely on a mechanism called "LightWeight
> Transactions (LWT)". Lightweight transactions relies on the PAXOS
> distributed consensus algorithm to enforce a condition upon data
> mutation. A table, system.paxos, is used to track the state of
> transactions. Furthermore, upon writes, several round-trips (two) are
> needed to ensure data integrity accross replica(minimum round trips to
> achieve consensus) and the system.paxos table is read / written to in
> addition to the applicative table.
> All of this causes LWT to be significantly slower than their lower
> consistency counterparts. On some Linagora owned production instances,
> regular reads takes 2ms while reads on tables relying on LWT takes 6ms.
> Similar figures are found for writes. We also noticed some high
> compaction throughtput on the paxos table, leading to many back-ground
> writes.
> Given the massive impact of LWT usage on performance, and given the lack
> of debate upon LWT adoption, I would like to re-challenge their usage...
> Here are the places we rely on LWT for the Distributed Server:
>  - IMAP UID generation (monotic integer) - strong consistency is
> strictly required to not loose data as overwriting a uid means
> overwriting a message.
>  - IMAP ModSeq generation (monotic integer) - strong consistency is
> required, as modseq overwrites can lead to some data not being well
> synchronised.
>  - Domain and users - we rely on LWT to return an error when deleting a
> user that do not exist, or creating an already existing user. It sounds
> unecessary.
>  - Message flags relies on LWT to ensure updates are not overwritten. As
> an often read metadata, the impact is high, for limited criticity for
> the end user. After all, no data is lost, only a user action like
> marking a message as Seen, an action that he can very well perform again
>  - Mailbox path registration, ACL - required to prevent data races
> My proposal would be:
>  - Keep using LWT for UID and modseq generation, as well as Mailbox path
> registration.
>  - Make the use of LWT for message flags update configurable - as an
> admin I can choose to disable it.
>  - I am also fine with completly removing LWT usage for message flags
> update.
>  - No longer use LWT on domain or users. Instead use idempotent create /
> delete. The contract test will thus need to be relaxed.
>  - On the long term, relying on a CRDT to represent ACLs at the
> Cassandra level, instead of serialized JSON, would enable to get rid of
> LWT usage on the ACL table.
> {code}
> Let's start relaxing LWT transaction for users & domains.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org