You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by "Benoit Tellier (Jira)" <se...@james.apache.org> on 2024/08/28 08:24:00 UTC
[jira] [Commented] (JAMES-4064) Mailqueue filters should not use durable subscriptions
[ https://issues.apache.org/jira/browse/JAMES-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877285#comment-17877285 ]
Benoit Tellier commented on JAMES-4064:
---------------------------------------
+1
We had similar issues on the Rabbit implementation where we wanted to create 1 queue per James server.
It turns out the management of the cleanup of that per-james server queue (and cleanup after reboot!) was a surprisingly annoying task.
- built in auto-delete queues are prone to being destroyed too early upon network failures
- we ended up relying on queue TTL wich was more resiliant
- a cleanup upon the shutdown process (@Predestroy) helped mitigating a bit also the issue - at least clean shutdowns do not leak!
I do not know how this past experience converts to the Pulsar context and hope the pulsar handling will be more straight forward.
> Mailqueue filters should not use durable subscriptions
> ------------------------------------------------------
>
> Key: JAMES-4064
> URL: https://issues.apache.org/jira/browse/JAMES-4064
> Project: James Server
> Issue Type: Sub-task
> Components: pulsar, Queue
> Reporter: Jean Helou
> Priority: Major
>
> Removal in a distributed system requires a way to distribute the filters to the various instances of the mailqueue. This was implemented by pushing filters in a dedicated pulsar topic.
>
> Since each instance needs to read the topic from the start to compile all the filters, it is necessary to have unique names for an exclusive subscription (see [https://pulsar.apache.org/docs/3.3.x/concepts-messaging/#subscriptions)]
> If the instances used a shared subscription they would not receive filters that have already been received by other members of the subscription.
>
> The current implementation uses Durable subscriptions and the uniqueness of the name is generated by adding an UUID. Unfortunately this means that a server restart will create a new subscription and the previous one will remain forever.
>
> I recently discovered that pulsar has a limit on the total number of durable subscriptions :D eventually preventing the creation of new ones.
>
> The filters should be migrated to non durable subscriptions (see [https://pulsar.apache.org/docs/3.3.x/concepts-messaging/#subscription-modes)]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org