You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@james.apache.org by "quantranhong1999 (via GitHub)" <gi...@apache.org> on 2024/02/20 10:33:59 UTC
[PR] [WIP] JAMES 3996 Redis event bus POC [james-project]
quantranhong1999 opened a new pull request, #2028:
URL: https://github.com/apache/james-project/pull/2028
WIP: not much yet, mostly trying Redis features related to our event bus use case.
So far it seems for group registration, Redis Streams would be a good fit:
- at least once delivery
- unicast to only 1 consumer in a consumer group
- support ack message
- support (manual) dead letter
While Redis Pub/sub can be used for key registration:
- at most once delivery (no ack, no dead letter btw)
- broadcast to all connected consumers
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] [WIP] JAMES 3996 Redis event bus POC [james-project]
Posted by "chibenwa (via GitHub)" <gi...@apache.org>.
chibenwa commented on code in PR #2028:
URL: https://github.com/apache/james-project/pull/2028#discussion_r1502996220
##########
event-bus/distributed/src/main/java/org/apache/james/events/EventDispatcher.java:
##########
@@ -168,18 +178,32 @@ private Mono<Void> remoteGroupsDispatch(byte[] serializedEvent, Event event) {
.then(Mono.error(ex)));
}
- private Mono<Void> remoteKeysDispatch(byte[] serializedEvent, Set<RegistrationKey> keys) {
- return remoteDispatch(serializedEvent,
+ private Mono<Void> remoteKeysDispatch(String eventAsJson, Set<RegistrationKey> keys) {
+ return remoteDispatch(eventAsJson,
keys.stream()
.map(RoutingKey::of)
.collect(ImmutableList.toImmutableList()));
}
- private Mono<Void> remoteDispatch(byte[] serializedEvent, Collection<RoutingKey> routingKeys) {
+ private Mono<Void> remoteDispatch(String eventAsJson, Collection<RoutingKey> routingKeys) {
if (routingKeys.isEmpty()) {
return Mono.empty();
}
- return sender.send(toMessages(serializedEvent, routingKeys));
+
+ return Flux.fromIterable(routingKeys)
+ .flatMap(routingKey -> getTargetChannels(routingKey)
+ .flatMap(channel -> {
+ // message format: event|||eventbusId|||routingKey.
+ // It seems quite dummy but the general idea is to embed the eventBusId and routingKey into the channel message without refactoring the Event (not good to put them into Event IMO).
+ String channelMessageToPublish = eventAsJson + REDIS_CHANNEL_MESSAGE_DELIMITER + eventBusId.asString() + REDIS_CHANNEL_MESSAGE_DELIMITER + routingKey.asString();
Review Comment:
And if for whatever reason we have a mailbox named `|||` then crazy thing happens...
We could reverse the order: eventBusId.asString() then routingKey.asString() then the JSON
and lookup the two first delimiters
Which would make this way more robust
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] [WIP] JAMES 3996 Redis event bus POC [james-project]
Posted by "quantranhong1999 (via GitHub)" <gi...@apache.org>.
quantranhong1999 commented on code in PR #2028:
URL: https://github.com/apache/james-project/pull/2028#discussion_r1497433283
##########
event-bus/redis/src/main/java/org/apache/james/events/GroupRegistrationHandler.java:
##########
@@ -104,68 +105,62 @@ GroupRegistration retrieveGroupRegistration(Group group) {
}
public void start() {
- scheduler = Schedulers.newBoundedElastic(EventBus.EXECUTION_RATE, ReactorUtils.DEFAULT_BOUNDED_ELASTIC_QUEUESIZE, "groups-handler");
- channelPool.createWorkQueue(
- QueueSpecification.queue(queueName.asString())
- .durable(DURABLE)
- .exclusive(!EXCLUSIVE)
- .autoDelete(!AUTO_DELETE)
- .arguments(configuration.workQueueArgumentsBuilder(!ALLOW_QUORUM)
- .deadLetter(namingStrategy.deadLetterExchange())
- .build()),
- BindingSpecification.binding()
- .exchange(namingStrategy.exchange())
- .queue(queueName.asString())
- .routingKey(EMPTY_ROUTING_KEY))
+ scheduler = Schedulers.newBoundedElastic(EventBus.EXECUTION_RATE, ReactorUtils.DEFAULT_BOUNDED_ELASTIC_QUEUESIZE, "groups-handler"); // create thread pool for Event bus
+
+ redisStreamReactiveCommands
+ .xgroupCreate(XReadArgs.StreamOffset.from(streamName, "0-0"),
+ CONSUMER_GROUP_NAME,
+ XGroupCreateArgs.Builder.mkstream()) // create Redis stream if not exist
.retryWhen(Retry.backoff(retryBackoff.getMaxRetries(), retryBackoff.getFirstBackoff()).jitter(retryBackoff.getJitterFactor()).scheduler(Schedulers.boundedElastic()))
.block();
- this.consumer = Optional.of(consumeWorkQueue());
+ this.consumer = Optional.of(consumeRedisStream());
}
- private Disposable consumeWorkQueue() {
- return Flux.using(
- receiverProvider::createReceiver,
- receiver -> receiver.consumeManualAck(queueName.asString(), new ConsumeOptions().qos(EventBus.EXECUTION_RATE)),
- Receiver::close)
- .filter(delivery -> Objects.nonNull(delivery.getBody()))
- .flatMap(this::deliver, EventBus.EXECUTION_RATE)
+ public Disposable consumeRedisStream() {
+ return Flux.interval(Duration.ofSeconds(1)) // pooling for new messages every 1 second. not sure it would be a good idea. Anyway just start simple...
Review Comment:
Ok, I will focus on the "key" impl with Redis.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] JAMES 3996 Redis event bus POC [james-project]
Posted by "vttranlina (via GitHub)" <gi...@apache.org>.
vttranlina commented on PR #2028:
URL: https://github.com/apache/james-project/pull/2028#issuecomment-1968502423
Good job
I tried to run RabbitMQEventBusTest on local,
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] JAMES 3996 Redis event bus POC [james-project]
Posted by "Maxxx873 (via GitHub)" <gi...@apache.org>.
Maxxx873 commented on PR #2028:
URL: https://github.com/apache/james-project/pull/2028#issuecomment-1968287999
> > Hello! If I understand correctly, then after completing this task, the task that I previously started https://issues.apache.org/jira/browse/JAMES-3956 will be closed. You do it much faster)
>
> Hi, in this POC I focused on the key part using Redis Pub/Sub but not the group part (our focus is to test the stability of Redis event bus key part first). So likely I would not move on the group part soon.
>
> You are welcome to take this work and continue on the group part ^^.
Thanks for your reply! I'll keep an eye on developments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] JAMES 3996 Redis event bus POC [james-project]
Posted by "quantranhong1999 (via GitHub)" <gi...@apache.org>.
quantranhong1999 commented on PR #2028:
URL: https://github.com/apache/james-project/pull/2028#issuecomment-1968513440
> I tried to run RabbitMQEventBusTest on local,
Well the `OutOfMemoryError` may be related to the `Flux<ChannelMessage<K, V>> observeChannels();` usage (we should use it more carefully):
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] JAMES 3996 Redis event bus POC [james-project]
Posted by "quantranhong1999 (via GitHub)" <gi...@apache.org>.
quantranhong1999 commented on PR #2028:
URL: https://github.com/apache/james-project/pull/2028#issuecomment-1975885147
> Well the OutOfMemoryError may be related to the Flux<ChannelMessage<K, V>> observeChannels(); usage (we should use it more carefully):
`OverflowStrategy.BUFFER` should be fine. We are using that within the RabbitMQ impl too:
Other OverflowStrategy(s) seems not fine as they could drop messages.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] JAMES 3996 Redis event bus POC [james-project]
Posted by "quantranhong1999 (via GitHub)" <gi...@apache.org>.
quantranhong1999 commented on PR #2028:
URL: https://github.com/apache/james-project/pull/2028#issuecomment-1968250395
> Hello! If I understand correctly, then after completing this task, the task that I previously started https://issues.apache.org/jira/browse/JAMES-3956 will be closed. You do it much faster)
Hi, in this POC I focused on the key part using Redis Pub/Sub but not the group part (our focus is to test the stability of Redis event bus key part first). So likely I would not move on the group part soon.
You are welcome to take this work and continue on the group part ^^.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] [WIP] JAMES 3996 Redis event bus POC [james-project]
Posted by "quantranhong1999 (via GitHub)" <gi...@apache.org>.
quantranhong1999 commented on PR #2028:
URL: https://github.com/apache/james-project/pull/2028#issuecomment-1959198347
TODO next: store notifications' metadata (EventBusId, RegistrationKey) in Redis.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] [WIP] JAMES 3996 Redis event bus POC [james-project]
Posted by "quantranhong1999 (via GitHub)" <gi...@apache.org>.
quantranhong1999 commented on code in PR #2028:
URL: https://github.com/apache/james-project/pull/2028#discussion_r1503637115
##########
event-bus/distributed/src/main/java/org/apache/james/events/KeyRegistrationBinder.java:
##########
@@ -19,36 +19,30 @@
package org.apache.james.events;
+import io.lettuce.core.api.reactive.RedisSetReactiveCommands;
import reactor.core.publisher.Mono;
-import reactor.rabbitmq.BindingSpecification;
-import reactor.rabbitmq.Sender;
-class RegistrationBinder {
- private final NamingStrategy namingStrategy;
- private final Sender sender;
- private final RegistrationQueueName registrationQueue;
+class KeyRegistrationBinder {
+ private final RedisSetReactiveCommands<String, String> redisSetReactiveCommands;
+ private final RegistrationChannelName registrationChannel;
- RegistrationBinder(NamingStrategy namingStrategy, Sender sender, RegistrationQueueName registrationQueue) {
- this.namingStrategy = namingStrategy;
- this.sender = sender;
- this.registrationQueue = registrationQueue;
+ KeyRegistrationBinder(RedisSetReactiveCommands<String, String> redisSetReactiveCommands,
+ RegistrationChannelName registrationChannel) {
+ this.redisSetReactiveCommands = redisSetReactiveCommands;
+ this.registrationChannel = registrationChannel;
}
Mono<Void> bind(RegistrationKey key) {
- return sender.bind(bindingSpecification(key))
+ // Use Redis Set to store 1 registrationKey -> n channel(s) mapping in Redis
+ RoutingKeyConverter.RoutingKey routingKey = RoutingKeyConverter.RoutingKey.of(key);
+ return redisSetReactiveCommands.sadd(routingKey.asString(), registrationChannel.asString())
.then();
}
Mono<Void> unbind(RegistrationKey key) {
- return sender.unbind(bindingSpecification(key))
- .then();
- }
-
- private BindingSpecification bindingSpecification(RegistrationKey key) {
+ // delete the registrationKey -> channel mapping in Redis
RoutingKeyConverter.RoutingKey routingKey = RoutingKeyConverter.RoutingKey.of(key);
- return BindingSpecification.binding()
- .exchange(namingStrategy.exchange())
- .queue(registrationQueue.asString())
- .routingKey(routingKey.asString());
+ return redisSetReactiveCommands.srem(routingKey.asString(), registrationChannel.asString())
Review Comment:
Likely we can write a task/cron job to clean up the mappings in that case.
Using Redis `PUBSUB CHANNEL` can list all active channels.
Then scan the mappings and delete the ones that mapped to an unactive channel.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] [WIP] JAMES 3996 Redis event bus POC [james-project]
Posted by "quantranhong1999 (via GitHub)" <gi...@apache.org>.
quantranhong1999 commented on code in PR #2028:
URL: https://github.com/apache/james-project/pull/2028#discussion_r1496801126
##########
event-bus/redis/src/main/java/org/apache/james/events/KeyReconnectionHandler.java:
##########
@@ -0,0 +1,67 @@
+/****************************************************************
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information *
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the *
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the *
+ * specific language governing permissions and limitations *
+ * under the License. *
+ ****************************************************************/
+
+package org.apache.james.events;
+
+import static org.apache.james.backends.rabbitmq.Constants.ALLOW_QUORUM;
+import static org.apache.james.backends.rabbitmq.Constants.AUTO_DELETE;
+import static org.apache.james.backends.rabbitmq.Constants.DURABLE;
+import static org.apache.james.backends.rabbitmq.Constants.EXCLUSIVE;
+
+import javax.inject.Inject;
+
+import org.apache.james.backends.rabbitmq.QueueArguments;
+import org.apache.james.backends.rabbitmq.RabbitMQConfiguration;
+import org.apache.james.backends.rabbitmq.SimpleConnectionPool;
+import org.reactivestreams.Publisher;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.rabbitmq.client.Channel;
+import com.rabbitmq.client.Connection;
+
+import reactor.core.publisher.Mono;
+
+public class KeyReconnectionHandler implements SimpleConnectionPool.ReconnectionHandler {
Review Comment:
(just dummy duplication first)
I am not sure we need this eventually with Redis implementation. But as a POC likely I can remove the advanced parts and just focus on the basic publish/subscribe part first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] [WIP] JAMES 3996 Redis event bus POC [james-project]
Posted by "quantranhong1999 (via GitHub)" <gi...@apache.org>.
quantranhong1999 commented on code in PR #2028:
URL: https://github.com/apache/james-project/pull/2028#discussion_r1503663261
##########
event-bus/distributed/src/main/java/org/apache/james/events/KeyRegistrationBinder.java:
##########
@@ -19,36 +19,30 @@
package org.apache.james.events;
+import io.lettuce.core.api.reactive.RedisSetReactiveCommands;
import reactor.core.publisher.Mono;
-import reactor.rabbitmq.BindingSpecification;
-import reactor.rabbitmq.Sender;
-class RegistrationBinder {
- private final NamingStrategy namingStrategy;
- private final Sender sender;
- private final RegistrationQueueName registrationQueue;
+class KeyRegistrationBinder {
+ private final RedisSetReactiveCommands<String, String> redisSetReactiveCommands;
+ private final RegistrationChannelName registrationChannel;
- RegistrationBinder(NamingStrategy namingStrategy, Sender sender, RegistrationQueueName registrationQueue) {
- this.namingStrategy = namingStrategy;
- this.sender = sender;
- this.registrationQueue = registrationQueue;
+ KeyRegistrationBinder(RedisSetReactiveCommands<String, String> redisSetReactiveCommands,
+ RegistrationChannelName registrationChannel) {
+ this.redisSetReactiveCommands = redisSetReactiveCommands;
+ this.registrationChannel = registrationChannel;
}
Mono<Void> bind(RegistrationKey key) {
- return sender.bind(bindingSpecification(key))
+ // Use Redis Set to store 1 registrationKey -> n channel(s) mapping in Redis
+ RoutingKeyConverter.RoutingKey routingKey = RoutingKeyConverter.RoutingKey.of(key);
+ return redisSetReactiveCommands.sadd(routingKey.asString(), registrationChannel.asString())
.then();
}
Mono<Void> unbind(RegistrationKey key) {
- return sender.unbind(bindingSpecification(key))
- .then();
- }
-
- private BindingSpecification bindingSpecification(RegistrationKey key) {
+ // delete the registrationKey -> channel mapping in Redis
RoutingKeyConverter.RoutingKey routingKey = RoutingKeyConverter.RoutingKey.of(key);
- return BindingSpecification.binding()
- .exchange(namingStrategy.exchange())
- .queue(registrationQueue.asString())
- .routingKey(routingKey.asString());
+ return redisSetReactiveCommands.srem(routingKey.asString(), registrationChannel.asString())
Review Comment:
example: https://github.com/apache/james-project/pull/2028/commits/273ac491863403c64a802ceccfb879bf2e5f8eeb
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] JAMES 3996 Redis event bus POC [james-project]
Posted by "Arsnael (via GitHub)" <gi...@apache.org>.
Arsnael closed pull request #2028: JAMES 3996 Redis event bus POC
URL: https://github.com/apache/james-project/pull/2028
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] [WIP] JAMES 3996 Redis event bus POC [james-project]
Posted by "quantranhong1999 (via GitHub)" <gi...@apache.org>.
quantranhong1999 commented on PR #2028:
URL: https://github.com/apache/james-project/pull/2028#issuecomment-1961002365
`RabbitMQEventBusTest` is 99% green. (except 1 resilient test)
Distributed James can run with the Redis event bus key within IT.
Next: build docker image and try this work on a staging env.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] [WIP] JAMES 3996 Redis event bus POC [james-project]
Posted by "chibenwa (via GitHub)" <gi...@apache.org>.
chibenwa commented on code in PR #2028:
URL: https://github.com/apache/james-project/pull/2028#discussion_r1502997593
##########
event-bus/distributed/src/main/java/org/apache/james/events/KeyRegistrationBinder.java:
##########
@@ -19,36 +19,30 @@
package org.apache.james.events;
+import io.lettuce.core.api.reactive.RedisSetReactiveCommands;
import reactor.core.publisher.Mono;
-import reactor.rabbitmq.BindingSpecification;
-import reactor.rabbitmq.Sender;
-class RegistrationBinder {
- private final NamingStrategy namingStrategy;
- private final Sender sender;
- private final RegistrationQueueName registrationQueue;
+class KeyRegistrationBinder {
+ private final RedisSetReactiveCommands<String, String> redisSetReactiveCommands;
+ private final RegistrationChannelName registrationChannel;
- RegistrationBinder(NamingStrategy namingStrategy, Sender sender, RegistrationQueueName registrationQueue) {
- this.namingStrategy = namingStrategy;
- this.sender = sender;
- this.registrationQueue = registrationQueue;
+ KeyRegistrationBinder(RedisSetReactiveCommands<String, String> redisSetReactiveCommands,
+ RegistrationChannelName registrationChannel) {
+ this.redisSetReactiveCommands = redisSetReactiveCommands;
+ this.registrationChannel = registrationChannel;
}
Mono<Void> bind(RegistrationKey key) {
- return sender.bind(bindingSpecification(key))
+ // Use Redis Set to store 1 registrationKey -> n channel(s) mapping in Redis
+ RoutingKeyConverter.RoutingKey routingKey = RoutingKeyConverter.RoutingKey.of(key);
+ return redisSetReactiveCommands.sadd(routingKey.asString(), registrationChannel.asString())
.then();
}
Mono<Void> unbind(RegistrationKey key) {
- return sender.unbind(bindingSpecification(key))
- .then();
- }
-
- private BindingSpecification bindingSpecification(RegistrationKey key) {
+ // delete the registrationKey -> channel mapping in Redis
RoutingKeyConverter.RoutingKey routingKey = RoutingKeyConverter.RoutingKey.of(key);
- return BindingSpecification.binding()
- .exchange(namingStrategy.exchange())
- .queue(registrationQueue.asString())
- .routingKey(routingKey.asString());
+ return redisSetReactiveCommands.srem(routingKey.asString(), registrationChannel.asString())
Review Comment:
Question: how clean up is done in case of james node crash?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] JAMES 3996 Redis event bus POC [james-project]
Posted by "quantranhong1999 (via GitHub)" <gi...@apache.org>.
quantranhong1999 commented on PR #2028:
URL: https://github.com/apache/james-project/pull/2028#issuecomment-1968540260
Hi,
I want to share the performance test and metrics results for this POC.
## Performance test
Perf test result generally is good as it is a bit better than 0.8.2. No related error logs.
### IMAP
The **SELECT** command (which uses `eventBus.register`) has better perf on Redis POC than RabbitMQ one (from TMail 0.8.2 release): p99 911ms compared to 1590ms.
## JMAP
## Event bus key metrics
Redis metrics generally outperform RabbitMQ.
### Redis POC
```
# HELP redis_register Generated from Dropwizard metric import (metric=redis-register, type=com.codahale.metrics.Timer)
# TYPE redis_register summary
redis_register{quantile="0.5",} 9.29791E-4
redis_register{quantile="0.75",} 0.0013598710000000001
redis_register{quantile="0.95",} 0.0033751030000000004
redis_register{quantile="0.98",} 0.005308415
redis_register{quantile="0.99",} 0.0077004790000000005
redis_register{quantile="0.999",} 0.044564479000000004
redis_register_count 21670.0
```
=> p99 7ms for registering a key
```
# HELP redis_dispatch Generated from Dropwizard metric import (metric=redis-dispatch, type=com.codahale.metrics.Timer)
# TYPE redis_dispatch summary
redis_dispatch{quantile="0.5",} 0.0016138230000000001
redis_dispatch{quantile="0.75",} 0.002506751
redis_dispatch{quantile="0.95",} 0.004849663000000001
redis_dispatch{quantile="0.98",} 0.007143423
redis_dispatch{quantile="0.99",} 0.009961471000000001
redis_dispatch{quantile="0.999",} 0.045350911
redis_dispatch_count 205453.0
```
=> p99 10ms for dispatching an event
```
# HELP redis_unregister Generated from Dropwizard metric import (metric=redis-unregister, type=com.codahale.metrics.Timer)
# TYPE rabbit_unregister summary
redis_unregister{quantile="0.5",} 0.001040383
redis_unregister{quantile="0.75",} 0.0017121270000000001
redis_unregister{quantile="0.95",} 0.025296895000000003
redis_unregister{quantile="0.98",} 0.034865151000000004
redis_unregister{quantile="0.99",} 0.05845811100000001
redis_unregister{quantile="0.999",} 0.070254591
redis_unregister_count 21670.0
```
=> p99 58ms for unregistering a key (a bit surprising as it is higher than 2 above)
### RabbitMQ
```
# HELP rabbit_register Generated from Dropwizard metric import (metric=rabbit-register, type=com.codahale.metrics.Timer)
# TYPE rabbit_register summary
rabbit_register{quantile="0.5",} 3.135E-6
rabbit_register{quantile="0.75",} 0.027918335000000002
rabbit_register{quantile="0.95",} 3.0198988790000003
rabbit_register{quantile="0.98",} 6.677331967000001
rabbit_register{quantile="0.99",} 9.328132095
rabbit_register{quantile="0.999",} 13.623099391
rabbit_register_count 18846.0
```
=> p99 9 seconds 0.0 for registering a key
```
# HELP rabbit_dispatch Generated from Dropwizard metric import (metric=rabbit-dispatch, type=com.codahale.metrics.Timer)
# TYPE rabbit_dispatch summary
rabbit_dispatch{quantile="0.5",} 5.57055E-4
rabbit_dispatch{quantile="0.75",} 7.98719E-4
rabbit_dispatch{quantile="0.95",} 0.001941503
rabbit_dispatch{quantile="0.98",} 0.0034078710000000003
rabbit_dispatch{quantile="0.99",} 0.005931007
rabbit_dispatch{quantile="0.999",} 6.5095598070000005
rabbit_dispatch_count 174339.0
```
=> p99 5ms for dispatching an event
```
# HELP rabbit_unregister Generated from Dropwizard metric import (metric=rabbit-unregister, type=com.codahale.metrics.Timer)
# TYPE rabbit_unregister summary
rabbit_unregister{quantile="0.5",} 0.11481907100000001
rabbit_unregister{quantile="0.75",} 0.327155711
rabbit_unregister{quantile="0.95",} 2.0384317430000003
rabbit_unregister{quantile="0.98",} 4.177526783
rabbit_unregister{quantile="0.99",} 10.737418239
rabbit_unregister{quantile="0.999",} 25.904021503000003
rabbit_unregister_count 1295.0
```
=> p99 10 seconds 0.0 for unregistering a key
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] [WIP] JAMES 3996 Redis event bus POC [james-project]
Posted by "chibenwa (via GitHub)" <gi...@apache.org>.
chibenwa commented on code in PR #2028:
URL: https://github.com/apache/james-project/pull/2028#discussion_r1496397813
##########
event-bus/redis/src/main/java/org/apache/james/events/KeyReconnectionHandler.java:
##########
@@ -0,0 +1,67 @@
+/****************************************************************
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information *
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the *
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the *
+ * specific language governing permissions and limitations *
+ * under the License. *
+ ****************************************************************/
+
+package org.apache.james.events;
+
+import static org.apache.james.backends.rabbitmq.Constants.ALLOW_QUORUM;
+import static org.apache.james.backends.rabbitmq.Constants.AUTO_DELETE;
+import static org.apache.james.backends.rabbitmq.Constants.DURABLE;
+import static org.apache.james.backends.rabbitmq.Constants.EXCLUSIVE;
+
+import javax.inject.Inject;
+
+import org.apache.james.backends.rabbitmq.QueueArguments;
+import org.apache.james.backends.rabbitmq.RabbitMQConfiguration;
+import org.apache.james.backends.rabbitmq.SimpleConnectionPool;
+import org.reactivestreams.Publisher;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.rabbitmq.client.Channel;
+import com.rabbitmq.client.Connection;
+
+import reactor.core.publisher.Mono;
+
+public class KeyReconnectionHandler implements SimpleConnectionPool.ReconnectionHandler {
Review Comment:
Needed with redis?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] [WIP] JAMES 3996 Redis event bus POC [james-project]
Posted by "quantranhong1999 (via GitHub)" <gi...@apache.org>.
quantranhong1999 commented on code in PR #2028:
URL: https://github.com/apache/james-project/pull/2028#discussion_r1497367581
##########
event-bus/redis/src/main/java/org/apache/james/events/GroupRegistrationHandler.java:
##########
@@ -104,68 +105,62 @@ GroupRegistration retrieveGroupRegistration(Group group) {
}
public void start() {
- scheduler = Schedulers.newBoundedElastic(EventBus.EXECUTION_RATE, ReactorUtils.DEFAULT_BOUNDED_ELASTIC_QUEUESIZE, "groups-handler");
- channelPool.createWorkQueue(
- QueueSpecification.queue(queueName.asString())
- .durable(DURABLE)
- .exclusive(!EXCLUSIVE)
- .autoDelete(!AUTO_DELETE)
- .arguments(configuration.workQueueArgumentsBuilder(!ALLOW_QUORUM)
- .deadLetter(namingStrategy.deadLetterExchange())
- .build()),
- BindingSpecification.binding()
- .exchange(namingStrategy.exchange())
- .queue(queueName.asString())
- .routingKey(EMPTY_ROUTING_KEY))
+ scheduler = Schedulers.newBoundedElastic(EventBus.EXECUTION_RATE, ReactorUtils.DEFAULT_BOUNDED_ELASTIC_QUEUESIZE, "groups-handler"); // create thread pool for Event bus
+
+ redisStreamReactiveCommands
+ .xgroupCreate(XReadArgs.StreamOffset.from(streamName, "0-0"),
+ CONSUMER_GROUP_NAME,
+ XGroupCreateArgs.Builder.mkstream()) // create Redis stream if not exist
.retryWhen(Retry.backoff(retryBackoff.getMaxRetries(), retryBackoff.getFirstBackoff()).jitter(retryBackoff.getJitterFactor()).scheduler(Schedulers.boundedElastic()))
.block();
- this.consumer = Optional.of(consumeWorkQueue());
+ this.consumer = Optional.of(consumeRedisStream());
}
- private Disposable consumeWorkQueue() {
- return Flux.using(
- receiverProvider::createReceiver,
- receiver -> receiver.consumeManualAck(queueName.asString(), new ConsumeOptions().qos(EventBus.EXECUTION_RATE)),
- Receiver::close)
- .filter(delivery -> Objects.nonNull(delivery.getBody()))
- .flatMap(this::deliver, EventBus.EXECUTION_RATE)
+ public Disposable consumeRedisStream() {
+ return Flux.interval(Duration.ofSeconds(1)) // pooling for new messages every 1 second. not sure it would be a good idea. Anyway just start simple...
Review Comment:
Maybe using `repeat` would be a better idea
##########
event-bus/redis/src/main/java/org/apache/james/events/GroupRegistrationHandler.java:
##########
@@ -104,68 +105,62 @@ GroupRegistration retrieveGroupRegistration(Group group) {
}
public void start() {
- scheduler = Schedulers.newBoundedElastic(EventBus.EXECUTION_RATE, ReactorUtils.DEFAULT_BOUNDED_ELASTIC_QUEUESIZE, "groups-handler");
- channelPool.createWorkQueue(
- QueueSpecification.queue(queueName.asString())
- .durable(DURABLE)
- .exclusive(!EXCLUSIVE)
- .autoDelete(!AUTO_DELETE)
- .arguments(configuration.workQueueArgumentsBuilder(!ALLOW_QUORUM)
- .deadLetter(namingStrategy.deadLetterExchange())
- .build()),
- BindingSpecification.binding()
- .exchange(namingStrategy.exchange())
- .queue(queueName.asString())
- .routingKey(EMPTY_ROUTING_KEY))
+ scheduler = Schedulers.newBoundedElastic(EventBus.EXECUTION_RATE, ReactorUtils.DEFAULT_BOUNDED_ELASTIC_QUEUESIZE, "groups-handler"); // create thread pool for Event bus
+
+ redisStreamReactiveCommands
+ .xgroupCreate(XReadArgs.StreamOffset.from(streamName, "0-0"),
+ CONSUMER_GROUP_NAME,
+ XGroupCreateArgs.Builder.mkstream()) // create Redis stream if not exist
.retryWhen(Retry.backoff(retryBackoff.getMaxRetries(), retryBackoff.getFirstBackoff()).jitter(retryBackoff.getJitterFactor()).scheduler(Schedulers.boundedElastic()))
.block();
- this.consumer = Optional.of(consumeWorkQueue());
+ this.consumer = Optional.of(consumeRedisStream());
}
- private Disposable consumeWorkQueue() {
- return Flux.using(
- receiverProvider::createReceiver,
- receiver -> receiver.consumeManualAck(queueName.asString(), new ConsumeOptions().qos(EventBus.EXECUTION_RATE)),
- Receiver::close)
- .filter(delivery -> Objects.nonNull(delivery.getBody()))
- .flatMap(this::deliver, EventBus.EXECUTION_RATE)
+ public Disposable consumeRedisStream() {
+ return Flux.interval(Duration.ofSeconds(1)) // pooling for new messages every 1 second. not sure it would be a good idea. Anyway just start simple...
+ .flatMap(ignore -> redisStreamReactiveCommands.xreadgroup(
+ Consumer.from(CONSUMER_GROUP_NAME, consumerName),
+ XReadArgs.Builder.block(0), // Waiting unlimited time for new messages to avoid too much pooling
Review Comment:
maybe limiting the count of messages would prevent a consumer from being overwhelmed upon too many messages published in a short time, and distribute the load better between consumers.
But I am not sure if the command would be blocked util received fully e.g. COUNT 10 messages yet. (need to test)
If is not waiting for the full 10 messages, then it is good to use COUNT.
Or change BLOCK to a non-forever duration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] [WIP] JAMES 3996 Redis event bus POC [james-project]
Posted by "chibenwa (via GitHub)" <gi...@apache.org>.
chibenwa commented on code in PR #2028:
URL: https://github.com/apache/james-project/pull/2028#discussion_r1497400494
##########
event-bus/redis/src/main/java/org/apache/james/events/GroupRegistrationHandler.java:
##########
@@ -104,68 +105,62 @@ GroupRegistration retrieveGroupRegistration(Group group) {
}
public void start() {
- scheduler = Schedulers.newBoundedElastic(EventBus.EXECUTION_RATE, ReactorUtils.DEFAULT_BOUNDED_ELASTIC_QUEUESIZE, "groups-handler");
- channelPool.createWorkQueue(
- QueueSpecification.queue(queueName.asString())
- .durable(DURABLE)
- .exclusive(!EXCLUSIVE)
- .autoDelete(!AUTO_DELETE)
- .arguments(configuration.workQueueArgumentsBuilder(!ALLOW_QUORUM)
- .deadLetter(namingStrategy.deadLetterExchange())
- .build()),
- BindingSpecification.binding()
- .exchange(namingStrategy.exchange())
- .queue(queueName.asString())
- .routingKey(EMPTY_ROUTING_KEY))
+ scheduler = Schedulers.newBoundedElastic(EventBus.EXECUTION_RATE, ReactorUtils.DEFAULT_BOUNDED_ELASTIC_QUEUESIZE, "groups-handler"); // create thread pool for Event bus
+
+ redisStreamReactiveCommands
+ .xgroupCreate(XReadArgs.StreamOffset.from(streamName, "0-0"),
+ CONSUMER_GROUP_NAME,
+ XGroupCreateArgs.Builder.mkstream()) // create Redis stream if not exist
.retryWhen(Retry.backoff(retryBackoff.getMaxRetries(), retryBackoff.getFirstBackoff()).jitter(retryBackoff.getJitterFactor()).scheduler(Schedulers.boundedElastic()))
.block();
- this.consumer = Optional.of(consumeWorkQueue());
+ this.consumer = Optional.of(consumeRedisStream());
}
- private Disposable consumeWorkQueue() {
- return Flux.using(
- receiverProvider::createReceiver,
- receiver -> receiver.consumeManualAck(queueName.asString(), new ConsumeOptions().qos(EventBus.EXECUTION_RATE)),
- Receiver::close)
- .filter(delivery -> Objects.nonNull(delivery.getBody()))
- .flatMap(this::deliver, EventBus.EXECUTION_RATE)
+ public Disposable consumeRedisStream() {
+ return Flux.interval(Duration.ofSeconds(1)) // pooling for new messages every 1 second. not sure it would be a good idea. Anyway just start simple...
Review Comment:
Don't implement group registration with redis please.
It's more of a job for a rabbit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] [WIP] JAMES 3996 Redis event bus POC [james-project]
Posted by "chibenwa (via GitHub)" <gi...@apache.org>.
chibenwa commented on code in PR #2028:
URL: https://github.com/apache/james-project/pull/2028#discussion_r1497401126
##########
event-bus/redis/src/main/java/org/apache/james/events/GroupRegistrationHandler.java:
##########
@@ -104,68 +105,62 @@ GroupRegistration retrieveGroupRegistration(Group group) {
}
public void start() {
- scheduler = Schedulers.newBoundedElastic(EventBus.EXECUTION_RATE, ReactorUtils.DEFAULT_BOUNDED_ELASTIC_QUEUESIZE, "groups-handler");
- channelPool.createWorkQueue(
- QueueSpecification.queue(queueName.asString())
- .durable(DURABLE)
- .exclusive(!EXCLUSIVE)
- .autoDelete(!AUTO_DELETE)
- .arguments(configuration.workQueueArgumentsBuilder(!ALLOW_QUORUM)
- .deadLetter(namingStrategy.deadLetterExchange())
- .build()),
- BindingSpecification.binding()
- .exchange(namingStrategy.exchange())
- .queue(queueName.asString())
- .routingKey(EMPTY_ROUTING_KEY))
+ scheduler = Schedulers.newBoundedElastic(EventBus.EXECUTION_RATE, ReactorUtils.DEFAULT_BOUNDED_ELASTIC_QUEUESIZE, "groups-handler"); // create thread pool for Event bus
+
+ redisStreamReactiveCommands
+ .xgroupCreate(XReadArgs.StreamOffset.from(streamName, "0-0"),
+ CONSUMER_GROUP_NAME,
+ XGroupCreateArgs.Builder.mkstream()) // create Redis stream if not exist
.retryWhen(Retry.backoff(retryBackoff.getMaxRetries(), retryBackoff.getFirstBackoff()).jitter(retryBackoff.getJitterFactor()).scheduler(Schedulers.boundedElastic()))
.block();
- this.consumer = Optional.of(consumeWorkQueue());
+ this.consumer = Optional.of(consumeRedisStream());
}
- private Disposable consumeWorkQueue() {
- return Flux.using(
- receiverProvider::createReceiver,
- receiver -> receiver.consumeManualAck(queueName.asString(), new ConsumeOptions().qos(EventBus.EXECUTION_RATE)),
- Receiver::close)
- .filter(delivery -> Objects.nonNull(delivery.getBody()))
- .flatMap(this::deliver, EventBus.EXECUTION_RATE)
+ public Disposable consumeRedisStream() {
+ return Flux.interval(Duration.ofSeconds(1)) // pooling for new messages every 1 second. not sure it would be a good idea. Anyway just start simple...
Review Comment:
No "group" implem with REDIS
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] [WIP] JAMES 3996 Redis event bus POC [james-project]
Posted by "quantranhong1999 (via GitHub)" <gi...@apache.org>.
quantranhong1999 commented on code in PR #2028:
URL: https://github.com/apache/james-project/pull/2028#discussion_r1503585953
##########
event-bus/distributed/src/main/java/org/apache/james/events/KeyRegistrationBinder.java:
##########
@@ -19,36 +19,30 @@
package org.apache.james.events;
+import io.lettuce.core.api.reactive.RedisSetReactiveCommands;
import reactor.core.publisher.Mono;
-import reactor.rabbitmq.BindingSpecification;
-import reactor.rabbitmq.Sender;
-class RegistrationBinder {
- private final NamingStrategy namingStrategy;
- private final Sender sender;
- private final RegistrationQueueName registrationQueue;
+class KeyRegistrationBinder {
+ private final RedisSetReactiveCommands<String, String> redisSetReactiveCommands;
+ private final RegistrationChannelName registrationChannel;
- RegistrationBinder(NamingStrategy namingStrategy, Sender sender, RegistrationQueueName registrationQueue) {
- this.namingStrategy = namingStrategy;
- this.sender = sender;
- this.registrationQueue = registrationQueue;
+ KeyRegistrationBinder(RedisSetReactiveCommands<String, String> redisSetReactiveCommands,
+ RegistrationChannelName registrationChannel) {
+ this.redisSetReactiveCommands = redisSetReactiveCommands;
+ this.registrationChannel = registrationChannel;
}
Mono<Void> bind(RegistrationKey key) {
- return sender.bind(bindingSpecification(key))
+ // Use Redis Set to store 1 registrationKey -> n channel(s) mapping in Redis
+ RoutingKeyConverter.RoutingKey routingKey = RoutingKeyConverter.RoutingKey.of(key);
+ return redisSetReactiveCommands.sadd(routingKey.asString(), registrationChannel.asString())
.then();
}
Mono<Void> unbind(RegistrationKey key) {
- return sender.unbind(bindingSpecification(key))
- .then();
- }
-
- private BindingSpecification bindingSpecification(RegistrationKey key) {
+ // delete the registrationKey -> channel mapping in Redis
RoutingKeyConverter.RoutingKey routingKey = RoutingKeyConverter.RoutingKey.of(key);
- return BindingSpecification.binding()
- .exchange(namingStrategy.exchange())
- .queue(registrationQueue.asString())
- .routingKey(routingKey.asString());
+ return redisSetReactiveCommands.srem(routingKey.asString(), registrationChannel.asString())
Review Comment:
Good question. I do not know the answer yet. But I knew that we need to carefully manage/clean up data in Redis.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] JAMES 3996 Redis event bus POC [james-project]
Posted by "Maxxx873 (via GitHub)" <gi...@apache.org>.
Maxxx873 commented on PR #2028:
URL: https://github.com/apache/james-project/pull/2028#issuecomment-1968243951
Hello! If I understand correctly, then after completing this task, the task that I previously started https://issues.apache.org/jira/browse/JAMES-3956 will be closed. You do it much faster)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org