You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by Matthieu Baechler <ma...@apache.org> on 2019/12/09 16:25:41 UTC
About Mailbox event listeners
Hi,
Most of the readers of this mailing list probably know that James has
been using project reactor [1], an implementation of Reactive Streams,
to improve its usage of resources and lower the latency of operations
for some time.
For a long time, the Mailbox part of James has a Pub/Sub system to
allow for modular extension of James default behavior like learning for
Spam, indexing in ElasticSearch, etc.
Recenlty, we reworked that Pub/Sub to be able to implement a
distributed version of it by integrating RabbitMQ.
During this work we wrote an extensive test suite to define the right
contracts a Mailbox Pub/Sub system should support and we reworked the
single-node version (called InVmEventBus) to rely on reactor.
However, listeners are still not able to make use of reactive streams
because we didn't changed the way listeners work.
You can either have a synchronous listener: it means that for a given
event, the delivery method call with return only when all listeners got
called (or an error occured) or an asynchronous listener, which
basically means we won't wait for the listener return to go ahead.
When a listener does some IO, like for example ElasticSearch indexer,
we don't leverage the fact that IO are asynchronous and thus we block
threads waiting for the IO.
Given everything is reactor based, it would be a much better use of
resources to let the listeners work with `Publisher` type and use non-
block APIs.
We could probably provide an adapter API to allow non-reactive
listeners to use the traditional API and wrapping listener calls into
some reactive types.
What would you think about this change? Feedback is more than welcome.
Cheers,
[1] https://projectreactor.io
--
Matthieu Baechler
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: About Mailbox event listeners
Posted by Tellier Benoit <bt...@apache.org>.
On 10/12/2019 15:28, matthieu.baechler@gmail.com wrote:
> [...]
> Writing an Architecture Decision Record would mean it's a decision we
> took (and implemented). It's not the case for now so I didn't wrote it.
>
> Now, either we agree about it and it can become an improvement issue in
> JIRA or we don't and we will let this mail thread die.
>
> Does it make sense?
Yes it makes sense.
+1 for the JIRA.
> Cheers,
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: About Mailbox event listeners
Posted by ma...@gmail.com.
Hi,
On Tue, 2019-12-10 at 13:11 +0700, Tellier Benoit wrote:
> Hi Matthieu,
>
> I think this is a great suggestion.
>
> On 09/12/2019 23:25, Matthieu Baechler wrote:
> > Hi,
> >
> > [...]
> >
> > Given everything is reactor based, it would be a much better use of
> > resources to let the listeners work with `Publisher` type and use
> > non-
> > block APIs.
> I think this is a great suggestion.
>
> I did try to contribute something similar in the past but did not
> carry
> out this changeset.
>
> You have my +1.
> > We could probably provide an adapter API to allow non-reactive
> > listeners to use the traditional API and wrapping listener calls
> > into
> > some reactive types.
> >
> > What would you think about this change? Feedback is more than
> > welcome.
> >
> > Cheers,
> >
> >
> > [1] https://projectreactor.io
> >
> Can you provide a little Architecture Decision Record ?
>
To be honest, I don't expect to write that myself for now. I was
working around that topic and gained some insights about it that I will
probably forget in the coming weeks.
Usually, these ideas are just lost so I now write about it to share.
Writing an Architecture Decision Record would mean it's a decision we
took (and implemented). It's not the case for now so I didn't wrote it.
Now, either we agree about it and it can become an improvement issue in
JIRA or we don't and we will let this mail thread die.
Does it make sense?
Cheers,
--
Matthieu Baechler
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: About Mailbox event listeners
Posted by Tellier Benoit <bt...@apache.org>.
Hi Matthieu,
I think this is a great suggestion.
On 09/12/2019 23:25, Matthieu Baechler wrote:
> Hi,
>
> [...]
>
> Given everything is reactor based, it would be a much better use of
> resources to let the listeners work with `Publisher` type and use non-
> block APIs.
I think this is a great suggestion.
I did try to contribute something similar in the past but did not carry
out this changeset.
You have my +1.
>
> We could probably provide an adapter API to allow non-reactive
> listeners to use the traditional API and wrapping listener calls into
> some reactive types.
>
> What would you think about this change? Feedback is more than welcome.
>
> Cheers,
>
>
> [1] https://projectreactor.io
>
Can you provide a little Architecture Decision Record ?
Regards,
Benoit Tellier
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: About Mailbox event listeners
Posted by Đức Trần Tiến <tr...@gmail.com>.
+1
Duc
Vào Th 2, 9 thg 12, 2019 vào lúc 23:25 Matthieu Baechler <
matthieu@apache.org> đã viết:
> Hi,
>
> Most of the readers of this mailing list probably know that James has
> been using project reactor [1], an implementation of Reactive Streams,
> to improve its usage of resources and lower the latency of operations
> for some time.
>
> For a long time, the Mailbox part of James has a Pub/Sub system to
> allow for modular extension of James default behavior like learning for
> Spam, indexing in ElasticSearch, etc.
>
> Recenlty, we reworked that Pub/Sub to be able to implement a
> distributed version of it by integrating RabbitMQ.
>
> During this work we wrote an extensive test suite to define the right
> contracts a Mailbox Pub/Sub system should support and we reworked the
> single-node version (called InVmEventBus) to rely on reactor.
>
> However, listeners are still not able to make use of reactive streams
> because we didn't changed the way listeners work.
>
> You can either have a synchronous listener: it means that for a given
> event, the delivery method call with return only when all listeners got
> called (or an error occured) or an asynchronous listener, which
> basically means we won't wait for the listener return to go ahead.
>
> When a listener does some IO, like for example ElasticSearch indexer,
> we don't leverage the fact that IO are asynchronous and thus we block
> threads waiting for the IO.
>
> Given everything is reactor based, it would be a much better use of
> resources to let the listeners work with `Publisher` type and use non-
> block APIs.
>
> We could probably provide an adapter API to allow non-reactive
> listeners to use the traditional API and wrapping listener calls into
> some reactive types.
>
> What would you think about this change? Feedback is more than welcome.
>
> Cheers,
>
>
> [1] https://projectreactor.io
>
>
> --
> Matthieu Baechler
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
>