[jira] [Commented] (JAMES-2733) WebAdmin is slow under heavyload

["Tellier Benoit (JIRA)" <se...@james.apache.org>]

    [ https://issues.apache.org/jira/browse/JAMES-2733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16832236#comment-16832236 ] 

Tellier Benoit commented on JAMES-2733:
---------------------------------------

See https://github.com/linagora/james-project/pull/2325 which rooted the issue to an inefficient MailQueue::getSize operation.

The overall proposed solution is incorect as RabbitMQ messages may represent some deleted emails (ManageableMailQueue::remove/clear).

Having a global counter for queue size is not a good idea either as  reconciliation oof that view will be impossible.

However I though about a possible solution: *Per-Bucket mail counter*

Upon enqueue:
 - Increment the counter of the bucket the mail is assigned to

Upon dequeue:
 - Decrement the counter of the bucket the mail is assigned to

Upon get size
 - Starting from the *browseStart* point...
 - Sum all the bucket counts starting from browseStart

Primary key: queueName + sliceId + bucketId // Collumn: mailCount

An interesting property is that this solution is *eventually consistent*: if a counter decrement is missed in slice t, getSize will be incorrect as long as browseStart will be before time t. However, the error will have no more impact when browseStart is after time t.

Finally this solution complexity is in O(bucketCount * sliceCount) (getSize current version is in O(mailCount)), and thus not related to mailCount, but more to mailQueue configuration value, and most ancient item. Big improvments expected!

> WebAdmin is slow under heavyload
> --------------------------------
>
>                 Key: JAMES-2733
>                 URL: https://issues.apache.org/jira/browse/JAMES-2733
>             Project: James Server
>          Issue Type: Improvement
>            Reporter: Gautier DI FOLCO
>            Priority: Minor
>
> Under heavy load, for a prolonged period of time (9 mail/s for 6h), WebAdmin takes up to 45 seconds like
>  
> {code:java}
> curl -XGET http://james-charge-01:8000/mailQueues/spool
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org