You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by Benoit TELLIER <bt...@apache.org> on 2022/01/07 11:24:56 UTC

About further adoption of Apache Pulsar to back the distributed server messaging use case.

Hello all,

Thanks to the very nice work of Matthieu and Jean we now have a working
Pulsar implementation for the James Mail Queue component [1]. Koodo!

[1] https://github.com/apache/james-project/pull/808

I did put together an ADR [2] providing details over:

 - Context over MailQueue and its historical implementation
 - Why would someone want Pulsar to back the mailqueue
 - Propose follow up steps for the Pulsar adoption regarding the
distributed James server
 - And finally technical details on this implementation.

[2] https://github.com/apache/james-project/pull/829

I think a good long term objective for the PMC is to drop RabbitMQ in
favor of pulsar (third parties could package their own components using
RabbitMQ if they wishes...)

This means:
 - Solve the bugs that were found during the Pulsar MailQueue review
 - Pulsar MailQueue need to allow listing blobs in order to be
deduplication friendly.
 - Provide an event bus based on Pulsar
 - Provide a task manager based on Pulsar
 - Package a distributed server backed by pulsar, deprecate then replace
the current one.
 - (optionally) support mail queue priorities

While contributions would of course be welcomed on this topic, we could
offer it as part of GSOC 2022, and we could co-mentor it with mentors of
the Pulsar community (see [3])

[3] https://lists.apache.org/thread/y9s7f6hmh51ky30l20yx0dlz458gw259

Would such a plan gain traction around here ?

Best regards,

Benoit TELLIER


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Re: About further adoption of Apache Pulsar to back the distributed server messaging use case.

Posted by Rene Cordier <rc...@apache.org>.
+1,

Rene.

On 07/01/2022 18:24, Benoit TELLIER wrote:
> Hello all,
> 
> Thanks to the very nice work of Matthieu and Jean we now have a working
> Pulsar implementation for the James Mail Queue component [1]. Koodo!
> 
> [1] https://github.com/apache/james-project/pull/808
> 
> I did put together an ADR [2] providing details over:
> 
>   - Context over MailQueue and its historical implementation
>   - Why would someone want Pulsar to back the mailqueue
>   - Propose follow up steps for the Pulsar adoption regarding the
> distributed James server
>   - And finally technical details on this implementation.
> 
> [2] https://github.com/apache/james-project/pull/829
> 
> I think a good long term objective for the PMC is to drop RabbitMQ in
> favor of pulsar (third parties could package their own components using
> RabbitMQ if they wishes...)
> 
> This means:
>   - Solve the bugs that were found during the Pulsar MailQueue review
>   - Pulsar MailQueue need to allow listing blobs in order to be
> deduplication friendly.
>   - Provide an event bus based on Pulsar
>   - Provide a task manager based on Pulsar
>   - Package a distributed server backed by pulsar, deprecate then replace
> the current one.
>   - (optionally) support mail queue priorities
> 
> While contributions would of course be welcomed on this topic, we could
> offer it as part of GSOC 2022, and we could co-mentor it with mentors of
> the Pulsar community (see [3])
> 
> [3] https://lists.apache.org/thread/y9s7f6hmh51ky30l20yx0dlz458gw259
> 
> Would such a plan gain traction around here ?
> 
> Best regards,
> 
> Benoit TELLIER
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Re: About further adoption of Apache Pulsar to back the distributed server messaging use case.

Posted by Benoit TELLIER <bt...@apache.org>.
Hello Jean,

On 11/01/2022 17:55, Jean Helou wrote:
> Hello,
>
> I'm obviously favorable to using pulsar :)
>
> You mention :
>> I think a good long term objective for the PMC is to drop RabbitMQ in
> favor of pulsar
>
> And in the ADR you also mention deprecating the RabbitMQ component.
>
> Can you clarify which implementations would be preserved/dropped (I'm
> obviously thinking of JMS implementations)
JMS makes sens for single node, zero dependency setup, with the embedded
ActiveMQ broker and should be kept.

I have a bad experience overall with RabbitMQ. Thus I "may" be a bit
biased toward this. I'm happy seeing alternatives to it emerge. I would
be happier seeing it not being the default implementation for the
distributed server. Over time we might eventually be able to drop it. Of
course this is in the (very) long run and we could have a co-existence
of both technologies in the meantime.

Best regards
>
> Cheers and happy new year james team
> jean
> On Fri, Jan 7, 2022 at 12:25 PM Benoit TELLIER <bt...@apache.org> wrote:
>
>> Hello all,
>>
>> Thanks to the very nice work of Matthieu and Jean we now have a working
>> Pulsar implementation for the James Mail Queue component [1]. Koodo!
>>
>> [1] https://github.com/apache/james-project/pull/808
>>
>> I did put together an ADR [2] providing details over:
>>
>>  - Context over MailQueue and its historical implementation
>>  - Why would someone want Pulsar to back the mailqueue
>>  - Propose follow up steps for the Pulsar adoption regarding the
>> distributed James server
>>  - And finally technical details on this implementation.
>>
>> [2] https://github.com/apache/james-project/pull/829
>>
>> I think a good long term objective for the PMC is to drop RabbitMQ in
>> favor of pulsar (third parties could package their own components using
>> RabbitMQ if they wishes...)
>>
>> This means:
>>  - Solve the bugs that were found during the Pulsar MailQueue review
>>  - Pulsar MailQueue need to allow listing blobs in order to be
>> deduplication friendly.
>>  - Provide an event bus based on Pulsar
>>  - Provide a task manager based on Pulsar
>>  - Package a distributed server backed by pulsar, deprecate then replace
>> the current one.
>>  - (optionally) support mail queue priorities
>>
>> While contributions would of course be welcomed on this topic, we could
>> offer it as part of GSOC 2022, and we could co-mentor it with mentors of
>> the Pulsar community (see [3])
>>
>> [3] https://lists.apache.org/thread/y9s7f6hmh51ky30l20yx0dlz458gw259
>>
>> Would such a plan gain traction around here ?
>>
>> Best regards,
>>
>> Benoit TELLIER
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>> For additional commands, e-mail: server-dev-help@james.apache.org
>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org


Re: About further adoption of Apache Pulsar to back the distributed server messaging use case.

Posted by Jean Helou <je...@gmail.com>.
Hello,

I'm obviously favorable to using pulsar :)

You mention :
> I think a good long term objective for the PMC is to drop RabbitMQ in
favor of pulsar

And in the ADR you also mention deprecating the RabbitMQ component.

Can you clarify which implementations would be preserved/dropped (I'm
obviously thinking of JMS implementations)

Cheers and happy new year james team
jean
On Fri, Jan 7, 2022 at 12:25 PM Benoit TELLIER <bt...@apache.org> wrote:

> Hello all,
>
> Thanks to the very nice work of Matthieu and Jean we now have a working
> Pulsar implementation for the James Mail Queue component [1]. Koodo!
>
> [1] https://github.com/apache/james-project/pull/808
>
> I did put together an ADR [2] providing details over:
>
>  - Context over MailQueue and its historical implementation
>  - Why would someone want Pulsar to back the mailqueue
>  - Propose follow up steps for the Pulsar adoption regarding the
> distributed James server
>  - And finally technical details on this implementation.
>
> [2] https://github.com/apache/james-project/pull/829
>
> I think a good long term objective for the PMC is to drop RabbitMQ in
> favor of pulsar (third parties could package their own components using
> RabbitMQ if they wishes...)
>
> This means:
>  - Solve the bugs that were found during the Pulsar MailQueue review
>  - Pulsar MailQueue need to allow listing blobs in order to be
> deduplication friendly.
>  - Provide an event bus based on Pulsar
>  - Provide a task manager based on Pulsar
>  - Package a distributed server backed by pulsar, deprecate then replace
> the current one.
>  - (optionally) support mail queue priorities
>
> While contributions would of course be welcomed on this topic, we could
> offer it as part of GSOC 2022, and we could co-mentor it with mentors of
> the Pulsar community (see [3])
>
> [3] https://lists.apache.org/thread/y9s7f6hmh51ky30l20yx0dlz458gw259
>
> Would such a plan gain traction around here ?
>
> Best regards,
>
> Benoit TELLIER
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
>

Re: About further adoption of Apache Pulsar to back the distributed server messaging use case.

Posted by Tung Tran Van <tu...@gmail.com>.
+1,

Tung, Tran Van

On Fri, Jan 7, 2022 at 6:25 PM Benoit TELLIER <bt...@apache.org> wrote:

> Hello all,
>
> Thanks to the very nice work of Matthieu and Jean we now have a working
> Pulsar implementation for the James Mail Queue component [1]. Koodo!
>
> [1] https://github.com/apache/james-project/pull/808
>
> I did put together an ADR [2] providing details over:
>
>  - Context over MailQueue and its historical implementation
>  - Why would someone want Pulsar to back the mailqueue
>  - Propose follow up steps for the Pulsar adoption regarding the
> distributed James server
>  - And finally technical details on this implementation.
>
> [2] https://github.com/apache/james-project/pull/829
>
> I think a good long term objective for the PMC is to drop RabbitMQ in
> favor of pulsar (third parties could package their own components using
> RabbitMQ if they wishes...)
>
> This means:
>  - Solve the bugs that were found during the Pulsar MailQueue review
>  - Pulsar MailQueue need to allow listing blobs in order to be
> deduplication friendly.
>  - Provide an event bus based on Pulsar
>  - Provide a task manager based on Pulsar
>  - Package a distributed server backed by pulsar, deprecate then replace
> the current one.
>  - (optionally) support mail queue priorities
>
> While contributions would of course be welcomed on this topic, we could
> offer it as part of GSOC 2022, and we could co-mentor it with mentors of
> the Pulsar community (see [3])
>
> [3] https://lists.apache.org/thread/y9s7f6hmh51ky30l20yx0dlz458gw259
>
> Would such a plan gain traction around here ?
>
> Best regards,
>
> Benoit TELLIER
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
>

-- 
Tung, Tran Van
*Phone:* (+84) 35 757 6258
*Skype:* tung.tv202