[jira] [Commented] (JAMES-3296) Be able to rebuild RabbitMQ MailQueue

["Benoit Tellier (Jira)" <se...@james.apache.org>]

    [ https://issues.apache.org/jira/browse/JAMES-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17236046#comment-17236046 ] 

Benoit Tellier commented on JAMES-3296:
---------------------------------------

I conducted some chaos testingregarding RabbitMQ.

This task allows me to recover content published to RabbitMQ - but lost.

However, it do not allow me to recover content published during a RabbitMQ outage.

Consider:

{code:java}
Given a stopped RabbitMQ,
When I send an email
Then the email send is failing
And the sent email is stuck in Outbox

As an administrator, I want to recover from this outage
I run curl -XPOST 'http://ip:port/mailQueues/spool?action=RepublishNotProcessedMails&olderThan=10s'
And the email is still stuck in Outbox
{code}


Of course I would LOVE to be able to not "loose" such mails...

The reason:

{code:java}
    void enQueue(Mail mail) throws MailQueue.MailQueueException {
        EnqueueId enqueueId = EnqueueId.generate();
        saveMail(mail)
            .map(partIds -> new MailReference(enqueueId, mail, partIds))
            .flatMap(Throwing.function(this::publishReferenceToRabbit).sneakyThrow())
            .flatMap(mailQueueView::storeMail)
            .thenEmpty(Mono.fromRunnable(enqueueMetric::increment))
            .block();
    }
{code}


The publish is done after, thus content is lost during rabbitMQ donwtime....

I we perform the storage in the view in Cassandra simultaneously with the content publishing in RabbitMQ the above implemented task will allow recovering **fully** from the outage, which seems like a very desirable behaviour to me!

> Be able to rebuild RabbitMQ MailQueue
> -------------------------------------
>
>                 Key: JAMES-3296
>                 URL: https://issues.apache.org/jira/browse/JAMES-3296
>             Project: James Server
>          Issue Type: Improvement
>          Components: Queue, SpoolManager & Processors
>            Reporter: Gautier DI FOLCO
>            Priority: Major
>             Fix For: 3.6.0
>
>
> In order to survive RabbitMQ data-loss, we should be able to rebuild RabbitMQ from Cassandra view to be able to continue mail processing.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org