Re: [DISCUSSION] PIP-130: Apply redelivery backoff policy for ack timeout

[PengHui Li <pe...@apache.org>]

Looks there is no objection, I will start the official vote for PIP-130

Regards,
Penghui

On Fri, Dec 31, 2021 at 8:25 PM Haiting Jiang <ji...@apache.org>
wrote:

> +1 for this PIP.
>
> Do we have plans for other languages clients? like go?
>
> Thanks,
> Haiting Jiang
>
> On 2021/12/27 13:24:52 PengHui Li wrote:
> > https://github.com/apache/pulsar/issues/13528
> >
> > Pasted below for quoting convenience.
> >
> > -----
> >
> > PIP 130: Apply redelivery backoff policy for ack timeout
> >
> > ## Motivation
> >
> > PIP 106
> >
> https://github.com/apache/pulsar/wiki/PIP-106%3A-Negative-acknowledgment-backoff
> > introduced negative acknowledgment message redelivery backoff which
> allows
> > users to achieve
> > more flexible message redelivery delay time control. But the redelivery
> > backoff policy only
> > apply to the negative acknowledgment API, for users who use ack timeout
> to
> > trigger the message
> > redelivery, not the negative acknowledgment API, they can't use the new
> > features introduced by
> > PIP 106.
> >
> > So the proposal is to apply the message redelivery policy for the ack
> > timeout mechanism.
> > Users can specify an ack timeout redelivery backoff, for example, apply
> an
> > exponential backoff
> > with 10 seconds ack timeout:
> >
> > ```java
> > client.newConsumer()
> >     .ackTimeout(10, TimeUnit.SECOND)
> >     .ackTimeoutRedeliveryBackoff(
> >         ExponentialRedeliveryBackoff.builder()
> >             .minDelayMs(1000)
> >             .maxDelayMs(60000).build());
> >     .subscribe();
> > ```
> >
> > The message redelivery behavior should be:
> >
> > |  Redelivery count   | Redelivery delay  |
> > |  ----  | ----  |
> > | 1 | 10 + 1 seconds |
> > | 2 | 10 + 2 seconds |
> > | 3 | 10 + 4 seconds |
> > | 4 | 10 + 8 seconds |
> > | 5 | 10 + 16 seconds |
> > | 6 | 10 + 32 seconds |
> > | 7 | 10 + 60 seconds |
> > | 8 | 10 + 60 seconds |
> >
> > ## Goal
> >
> > Add an API to the Java Client to provide the ability to specify the ack
> > timeout message redelivery
> > backoff and the message redelivery behavior should abide by the
> redelivery
> > backoff policy.
> >
> >
> > ## API Changes
> >
> > 1. Change `NegativeAckRedeliveryBackoff` to `RedeliveryBackoff`, so that
> we
> > can use the
> > MessageRedeliveryBackoff for both negative acknowledgment API and ack
> > timeout message redelivery.
> >
> > 2. Change `NegativeAckRedeliveryExponentialBackoff` to
> > `ExponentialRedeliveryBackoff`, and add `multiplier`
> > for `RedeliveryExponentialBackoff` with default value 2.
> >
> > ExponentialRedeliveryBackoff.builder()
> >             .minDelayMs(1000)
> >             .maxDelayMs(60000)
> >             .multiplier(5)
> >             .build()
> >
> > 3. Add `ackTimeoutRedeliveryBackoff` method for the `ConsumerBuilder`:
> >
> > ```java
> > client.newConsumer()
> >     .ackTimeout(10, TimeUnit.SECOND)
> >     .ackTimeoutRedeliveryBackoff(
> >         ExponentialRedeliveryBackoff.builder()
> >             .minDelayMs(1000)
> >             .maxDelayMs(60000).build());
> >     .subscribe();
> > ```
> >
> > ## Compatibility and migration plan
> >
> > Since the negative acknowledgment message, redelivery backoff has not
> been
> > released yet,
> > so we can modify the API directly.
> >
> > ## Tests plan
> >
> > - Verify the introduced `multiplier` of ExponentialRedeliveryBackoff
> > - Verify the ack timeout message redelivery work as expected
> >
> > Regards,
> > Penghui
> >
>