You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by Nitin Goyal <ni...@gmail.com> on 2023/01/08 04:27:02 UTC
[VOTE] PIP-239: Retry Mechanism per Message title
Hello community,
I'm starting the VOTE for PIP-239: Retry Mechanism per Message title.
Motivation: We are working on a project where each message has their retry
as per the requirements. like one message 100 times and other messages 5
times and so on. This feature also adds an extra functionality while
comparing existing queueing services like RabbitMQ or SQS etc.
For more details please find the detailed PIP at:
https://github.com/apache/pulsar/issues/19136
Discussion Threads:
1. https://lists.apache.org/thread/rn6114xxtx1j57yy2jbmdv4xzt80o1hz
2. https://lists.apache.org/thread/b9rfv6t307z439xx3zt2ym4p140qzp06
Proposed changes in pulsar go client library.
https://github.com/apache/pulsar-client-go/pull/939
Thanks
Nitin Goyal
Re: [VOTE] PIP-239: Retry Mechanism per Message title
Posted by Nitin Goyal <ni...@gmail.com>.
Hello all,
I am extremely apologies about the discussion hours. It was my first time
to have discussions and voting process and wasn’t aware about the policy.
Will keep this in mind from next time.
Once again apologies.
Sincere
Nitin Goyal
On Tue, 10 Jan 2023 at 10:04 PM, Dave Fisher <wa...@apache.org> wrote:
> -1 (binding) because the discussion thread is in progress and most
> important you only allowed for 71 hours of discussion.
>
> I think it is important that any change to the main producer / consumer
> flow be proven to have no performance regressions in the broker.
>
> Regards,
> Dave
>
> > On Jan 8, 2023, at 11:35 PM, Enrico Olivelli <eo...@gmail.com>
> wrote:
> >
> > I am sure that this is a good idea.
> >
> > -1 (binding)
> >
> > I will follow up on the discussion thread
> >
> > I am sorry I am catching up late on this thread
> >
> > Enrico
> >
> > Il giorno lun 9 gen 2023 alle ore 03:51 Ran Gao <rg...@apache.org> ha
> scritto:
> >>
> >> +1 (non-binding)
> >>
> >> Thanks,
> >> Ran Gao
> >>
> >> On 2023/01/08 15:05:31 Zixuan Liu wrote:
> >>> +1 (non-binding)
> >>>
> >>> Thanks,
> >>> Zixuan
> >>>
> >>> Yunze Xu <yz...@streamnative.io.invalid> 于2023年1月8日周日 22:38写道：
> >>>
> >>>> +1 (binding)
> >>>>
> >>>> Thanks,
> >>>> Yunze
> >>>>
> >>>> On Sun, Jan 8, 2023 at 4:46 PM Zike Yang <zi...@apache.org> wrote:
> >>>>>
> >>>>> +1 (non-binding)
> >>>>>
> >>>>> Thanks,
> >>>>> Zike Yang
> >>>>>
> >>>>> On Sun, Jan 8, 2023 at 3:22 PM Haiting Jiang <jianghaiting@gmail.com
> >
> >>>> wrote:
> >>>>>>
> >>>>>> +1 binding
> >>>>>>
> >>>>>> Although I think we should use more unique keys for system
> properties
> >>>> in pulsar,
> >>>>>> e.g. reserve all properties prefixed with "PULSAR_" for system
> >>>> internal usage.
> >>>>>> But it's another topic as we already have "RECONSUMETIMES".
> >>>>>>
> >>>>>>
> >>>>>> On Sun, Jan 8, 2023 at 12:27 PM Nitin Goyal <
> nitin.goyal.dev@gmail.com>
> >>>> wrote:
> >>>>>>>
> >>>>>>> Hello community,
> >>>>>>>
> >>>>>>> I'm starting the VOTE for PIP-239: Retry Mechanism per Message
> title.
> >>>>>>>
> >>>>>>> Motivation: We are working on a project where each message has
> their
> >>>> retry
> >>>>>>> as per the requirements. like one message 100 times and other
> >>>> messages 5
> >>>>>>> times and so on. This feature also adds an extra functionality
> while
> >>>>>>> comparing existing queueing services like RabbitMQ or SQS etc.
> >>>>>>>
> >>>>>>> For more details please find the detailed PIP at:
> >>>>>>> https://github.com/apache/pulsar/issues/19136
> >>>>>>>
> >>>>>>> Discussion Threads:
> >>>>>>> 1.
> https://lists.apache.org/thread/rn6114xxtx1j57yy2jbmdv4xzt80o1hz
> >>>>>>> 2.
> https://lists.apache.org/thread/b9rfv6t307z439xx3zt2ym4p140qzp06
> >>>>>>>
> >>>>>>> Proposed changes in pulsar go client library.
> >>>>>>> https://github.com/apache/pulsar-client-go/pull/939
> >>>>>>>
> >>>>>>> Thanks
> >>>>>>> Nitin Goyal
> >>>>
> >>>
>
> --
Regards
Nitin Goyal
Re: [VOTE] PIP-239: Retry Mechanism per Message title
Posted by Dave Fisher <wa...@apache.org>.
-1 (binding) because the discussion thread is in progress and most important you only allowed for 71 hours of discussion.
I think it is important that any change to the main producer / consumer flow be proven to have no performance regressions in the broker.
Regards,
Dave
> On Jan 8, 2023, at 11:35 PM, Enrico Olivelli <eo...@gmail.com> wrote:
>
> I am sure that this is a good idea.
>
> -1 (binding)
>
> I will follow up on the discussion thread
>
> I am sorry I am catching up late on this thread
>
> Enrico
>
> Il giorno lun 9 gen 2023 alle ore 03:51 Ran Gao <rg...@apache.org> ha scritto:
>>
>> +1 (non-binding)
>>
>> Thanks,
>> Ran Gao
>>
>> On 2023/01/08 15:05:31 Zixuan Liu wrote:
>>> +1 (non-binding)
>>>
>>> Thanks,
>>> Zixuan
>>>
>>> Yunze Xu <yz...@streamnative.io.invalid> 于2023年1月8日周日 22:38写道：
>>>
>>>> +1 (binding)
>>>>
>>>> Thanks,
>>>> Yunze
>>>>
>>>> On Sun, Jan 8, 2023 at 4:46 PM Zike Yang <zi...@apache.org> wrote:
>>>>>
>>>>> +1 (non-binding)
>>>>>
>>>>> Thanks,
>>>>> Zike Yang
>>>>>
>>>>> On Sun, Jan 8, 2023 at 3:22 PM Haiting Jiang <ji...@gmail.com>
>>>> wrote:
>>>>>>
>>>>>> +1 binding
>>>>>>
>>>>>> Although I think we should use more unique keys for system properties
>>>> in pulsar,
>>>>>> e.g. reserve all properties prefixed with "PULSAR_" for system
>>>> internal usage.
>>>>>> But it's another topic as we already have "RECONSUMETIMES".
>>>>>>
>>>>>>
>>>>>> On Sun, Jan 8, 2023 at 12:27 PM Nitin Goyal <ni...@gmail.com>
>>>> wrote:
>>>>>>>
>>>>>>> Hello community,
>>>>>>>
>>>>>>> I'm starting the VOTE for PIP-239: Retry Mechanism per Message title.
>>>>>>>
>>>>>>> Motivation: We are working on a project where each message has their
>>>> retry
>>>>>>> as per the requirements. like one message 100 times and other
>>>> messages 5
>>>>>>> times and so on. This feature also adds an extra functionality while
>>>>>>> comparing existing queueing services like RabbitMQ or SQS etc.
>>>>>>>
>>>>>>> For more details please find the detailed PIP at:
>>>>>>> https://github.com/apache/pulsar/issues/19136
>>>>>>>
>>>>>>> Discussion Threads:
>>>>>>> 1. https://lists.apache.org/thread/rn6114xxtx1j57yy2jbmdv4xzt80o1hz
>>>>>>> 2. https://lists.apache.org/thread/b9rfv6t307z439xx3zt2ym4p140qzp06
>>>>>>>
>>>>>>> Proposed changes in pulsar go client library.
>>>>>>> https://github.com/apache/pulsar-client-go/pull/939
>>>>>>>
>>>>>>> Thanks
>>>>>>> Nitin Goyal
>>>>
>>>
Re: [VOTE] PIP-239: Retry Mechanism per Message title
Posted by Nitin Goyal <ni...@gmail.com>.
Hello Xiaolong Ran,
Thanks for clarification.
>> about subsequent version iterations, i would like to discuss with you
whether we want discard the logic of ReconsumeLater
We can open a new PIP/discussion thread for it. And there we can consider
all types of scenarios related to i.e. retry, dlq, delays in that.
Because currently there is no PIP or discussion thread open to discuss
deprecation of the reconsumeLater function in favor of NACK Backoff
Interface.
And my guess is that because not all the client libraries are yet supported
NACKBackoff policy as of now.
On Tue, Jan 10, 2023 at 8:18 PM rxl@apache.org <ra...@gmail.com>
wrote:
> Hi Nitin:
>
> > 3. if a message NACKed then. How many times it retries is also in memory
> if
> > the consumer has failed or restarted all this information is lost. But in
> > the case of reconsumeLater all these things persisted in the broker.
>
> The RedeliverCount field is an attribute in the CommandMessage protocol.
> Its state is persisted on the broker side but not on the consumer side, so
> restarting the consumer will not cause the loss of the message
> RedeliverCount attribute. Of course, situations such as Broker downtime and
> restart are excluded.
>
> > 1. NACK does not send messages to DLQ as per current architecture.; it
> > keeps messages in consumer memory to redeliver them later to the
> > same consumer.
>
> My thoughts on this point may not be consistent with yours. The logic of
> Nack is to send messages to DLQ. We can configure how many Nacks will be
> sent to DLQ. The request of the Redeliver Command is in the memory state of
> the Consumer, but if the restart of the Consumer is considered, the Broker
> will continue to push the unack message after the consumer restarts, and
> will not discard this state. Whether the message will be sent to the same
> consumer after Nack is the same principle as ReconsumeLater, and also
> depends on what subscription type you use. It's just that because
> ReconsumeLater uses the implementation logic of delayed messages, it is
> only applicable to shared subscription types.
>
> > If a message is NACKed the message will get consumed infinite time based
> on backoff limit till it gets ACKed. where a reconsume later >message will
> be sent to DLQ onces it reaches the retry limit set by the consumer.
>
> Again, we expose the NackBackoff interface, this behavior can be controlled
> by the user.
>
> In response to what you said about increasing the number of retries at the
> message level or customizing the message `+1`, but here I actually want to
> implement this logic in the logic of Nack instead of adding more logic to
> the existing ReconsumeLater, because In subsequent version iterations, I
> would like to discuss with you whether we want to discard the logic of
> ReconsumeLater.
>
> --
> Thanks
> Xiaolong Ran
>
> Nitin Goyal <ni...@gmail.com> 于2023年1月10日周二 22:07写道：
>
> > Hello Xiaolong,
> >
> > About your concern on NACK and reconsumeLater. Also as per current
> > Architecture rheir work is completely different from each other. which is
> > as follows.
> >
> > 1. NACK does not send messages to DLQ as per current architecture.; it
> > keeps messages in consumer memory to redeliver them later to the
> > same consumer. but the retry will allow adding custom delay to the
> message
> > and the same msg can be processed by another consumer from the same
> > consumer group as per their configuration (i.e. shared types).
> >
> > 2. If a message is NACKed the message will get consumed infinite time
> based
> > on backoff limit till it gets ACKed. where a reconsume later message will
> > be sent to DLQ onces it reaches the retry limit set by the consumer.
> >
> > 3. if a message NACKed then. How many times it retries is also in memory
> if
> > the consumer has failed or restarted all this information is lost. But in
> > the case of reconsumeLater all these things persisted in the broker.
> >
> >
> > On Tue, Jan 10, 2023 at 6:53 PM rxl@apache.org <ranxiaolong716@gmail.com
> >
> > wrote:
> >
> > > -1(non-binding)
> > >
> > > As discussed in [DISCUSS], for ReconsumeLater, I prefer to discard this
> > > interface in later versions, and think about this issue from another
> > angle.
> > > From a user’s point of view, both ReconsumeLater and Nack have the
> > function
> > > of retweeting messages. From a functional point of view, it seems that
> > > there is not much difference between the two. So how do we advise users
> > > under what circumstances? It is more appropriate to use ReconsumeLater,
> > and
> > > under what circumstances is it more appropriate to use Nack? For me, I
> > > don't know how to introduce it to users. I can only explain it based on
> > > their implementation logic. ReconsumeLater sends messages to the Retry
> > > Topic, and Nack sends messages to the DLQ Topic. But Nack can
> completely
> > > replace the logic here. Is it necessary for us to introduce a new
> > > ReconsumeLater interface to confuse users.
> > >
> > > So here I hope we can discuss whether we need to support this feature
> or
> > > logic from the perspective of Nack.
> > >
> > > --
> > > Thanks
> > > Xiaolong Ran
> > >
> > > Enrico Olivelli <eo...@gmail.com> 于2023年1月9日周一 15:36写道：
> > >
> > > > I am sure that this is a good idea.
> > > >
> > > > -1 (binding)
> > > >
> > > > I will follow up on the discussion thread
> > > >
> > > > I am sorry I am catching up late on this thread
> > > >
> > > > Enrico
> > > >
> > > > Il giorno lun 9 gen 2023 alle ore 03:51 Ran Gao <rg...@apache.org> ha
> > > > scritto:
> > > > >
> > > > > +1 (non-binding)
> > > > >
> > > > > Thanks,
> > > > > Ran Gao
> > > > >
> > > > > On 2023/01/08 15:05:31 Zixuan Liu wrote:
> > > > > > +1 (non-binding)
> > > > > >
> > > > > > Thanks,
> > > > > > Zixuan
> > > > > >
> > > > > > Yunze Xu <yz...@streamnative.io.invalid> 于2023年1月8日周日 22:38写道：
> > > > > >
> > > > > > > +1 (binding)
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Yunze
> > > > > > >
> > > > > > > On Sun, Jan 8, 2023 at 4:46 PM Zike Yang <zi...@apache.org>
> > wrote:
> > > > > > > >
> > > > > > > > +1 (non-binding)
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Zike Yang
> > > > > > > >
> > > > > > > > On Sun, Jan 8, 2023 at 3:22 PM Haiting Jiang <
> > > > jianghaiting@gmail.com>
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > +1 binding
> > > > > > > > >
> > > > > > > > > Although I think we should use more unique keys for system
> > > > properties
> > > > > > > in pulsar,
> > > > > > > > > e.g. reserve all properties prefixed with "PULSAR_" for
> > system
> > > > > > > internal usage.
> > > > > > > > > But it's another topic as we already have "RECONSUMETIMES".
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Sun, Jan 8, 2023 at 12:27 PM Nitin Goyal <
> > > > nitin.goyal.dev@gmail.com>
> > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Hello community,
> > > > > > > > > >
> > > > > > > > > > I'm starting the VOTE for PIP-239: Retry Mechanism per
> > > Message
> > > > title.
> > > > > > > > > >
> > > > > > > > > > Motivation: We are working on a project where each
> message
> > > has
> > > > their
> > > > > > > retry
> > > > > > > > > > as per the requirements. like one message 100 times and
> > other
> > > > > > > messages 5
> > > > > > > > > > times and so on. This feature also adds an extra
> > > functionality
> > > > while
> > > > > > > > > > comparing existing queueing services like RabbitMQ or SQS
> > > etc.
> > > > > > > > > >
> > > > > > > > > > For more details please find the detailed PIP at:
> > > > > > > > > > https://github.com/apache/pulsar/issues/19136
> > > > > > > > > >
> > > > > > > > > > Discussion Threads:
> > > > > > > > > > 1.
> > > > https://lists.apache.org/thread/rn6114xxtx1j57yy2jbmdv4xzt80o1hz
> > > > > > > > > > 2.
> > > > https://lists.apache.org/thread/b9rfv6t307z439xx3zt2ym4p140qzp06
> > > > > > > > > >
> > > > > > > > > > Proposed changes in pulsar go client library.
> > > > > > > > > > https://github.com/apache/pulsar-client-go/pull/939
> > > > > > > > > >
> > > > > > > > > > Thanks
> > > > > > > > > > Nitin Goyal
> > > > > > >
> > > > > >
> > > >
> > >
> >
> >
> > --
> > Regards
> > Nitin Goyal
> >
>
--
Regards
Nitin Goyal
Re: [VOTE] PIP-239: Retry Mechanism per Message title
Posted by "rxl@apache.org" <ra...@gmail.com>.
Hi Nitin:
> 3. if a message NACKed then. How many times it retries is also in memory
if
> the consumer has failed or restarted all this information is lost. But in
> the case of reconsumeLater all these things persisted in the broker.
The RedeliverCount field is an attribute in the CommandMessage protocol.
Its state is persisted on the broker side but not on the consumer side, so
restarting the consumer will not cause the loss of the message
RedeliverCount attribute. Of course, situations such as Broker downtime and
restart are excluded.
> 1. NACK does not send messages to DLQ as per current architecture.; it
> keeps messages in consumer memory to redeliver them later to the
> same consumer.
My thoughts on this point may not be consistent with yours. The logic of
Nack is to send messages to DLQ. We can configure how many Nacks will be
sent to DLQ. The request of the Redeliver Command is in the memory state of
the Consumer, but if the restart of the Consumer is considered, the Broker
will continue to push the unack message after the consumer restarts, and
will not discard this state. Whether the message will be sent to the same
consumer after Nack is the same principle as ReconsumeLater, and also
depends on what subscription type you use. It's just that because
ReconsumeLater uses the implementation logic of delayed messages, it is
only applicable to shared subscription types.
> If a message is NACKed the message will get consumed infinite time based
on backoff limit till it gets ACKed. where a reconsume later >message will
be sent to DLQ onces it reaches the retry limit set by the consumer.
Again, we expose the NackBackoff interface, this behavior can be controlled
by the user.
In response to what you said about increasing the number of retries at the
message level or customizing the message `+1`, but here I actually want to
implement this logic in the logic of Nack instead of adding more logic to
the existing ReconsumeLater, because In subsequent version iterations, I
would like to discuss with you whether we want to discard the logic of
ReconsumeLater.
--
Thanks
Xiaolong Ran
Nitin Goyal <ni...@gmail.com> 于2023年1月10日周二 22:07写道：
> Hello Xiaolong,
>
> About your concern on NACK and reconsumeLater. Also as per current
> Architecture rheir work is completely different from each other. which is
> as follows.
>
> 1. NACK does not send messages to DLQ as per current architecture.; it
> keeps messages in consumer memory to redeliver them later to the
> same consumer. but the retry will allow adding custom delay to the message
> and the same msg can be processed by another consumer from the same
> consumer group as per their configuration (i.e. shared types).
>
> 2. If a message is NACKed the message will get consumed infinite time based
> on backoff limit till it gets ACKed. where a reconsume later message will
> be sent to DLQ onces it reaches the retry limit set by the consumer.
>
> 3. if a message NACKed then. How many times it retries is also in memory if
> the consumer has failed or restarted all this information is lost. But in
> the case of reconsumeLater all these things persisted in the broker.
>
>
> On Tue, Jan 10, 2023 at 6:53 PM rxl@apache.org <ra...@gmail.com>
> wrote:
>
> > -1(non-binding)
> >
> > As discussed in [DISCUSS], for ReconsumeLater, I prefer to discard this
> > interface in later versions, and think about this issue from another
> angle.
> > From a user’s point of view, both ReconsumeLater and Nack have the
> function
> > of retweeting messages. From a functional point of view, it seems that
> > there is not much difference between the two. So how do we advise users
> > under what circumstances? It is more appropriate to use ReconsumeLater,
> and
> > under what circumstances is it more appropriate to use Nack? For me, I
> > don't know how to introduce it to users. I can only explain it based on
> > their implementation logic. ReconsumeLater sends messages to the Retry
> > Topic, and Nack sends messages to the DLQ Topic. But Nack can completely
> > replace the logic here. Is it necessary for us to introduce a new
> > ReconsumeLater interface to confuse users.
> >
> > So here I hope we can discuss whether we need to support this feature or
> > logic from the perspective of Nack.
> >
> > --
> > Thanks
> > Xiaolong Ran
> >
> > Enrico Olivelli <eo...@gmail.com> 于2023年1月9日周一 15:36写道：
> >
> > > I am sure that this is a good idea.
> > >
> > > -1 (binding)
> > >
> > > I will follow up on the discussion thread
> > >
> > > I am sorry I am catching up late on this thread
> > >
> > > Enrico
> > >
> > > Il giorno lun 9 gen 2023 alle ore 03:51 Ran Gao <rg...@apache.org> ha
> > > scritto:
> > > >
> > > > +1 (non-binding)
> > > >
> > > > Thanks,
> > > > Ran Gao
> > > >
> > > > On 2023/01/08 15:05:31 Zixuan Liu wrote:
> > > > > +1 (non-binding)
> > > > >
> > > > > Thanks,
> > > > > Zixuan
> > > > >
> > > > > Yunze Xu <yz...@streamnative.io.invalid> 于2023年1月8日周日 22:38写道：
> > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > Thanks,
> > > > > > Yunze
> > > > > >
> > > > > > On Sun, Jan 8, 2023 at 4:46 PM Zike Yang <zi...@apache.org>
> wrote:
> > > > > > >
> > > > > > > +1 (non-binding)
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Zike Yang
> > > > > > >
> > > > > > > On Sun, Jan 8, 2023 at 3:22 PM Haiting Jiang <
> > > jianghaiting@gmail.com>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > +1 binding
> > > > > > > >
> > > > > > > > Although I think we should use more unique keys for system
> > > properties
> > > > > > in pulsar,
> > > > > > > > e.g. reserve all properties prefixed with "PULSAR_" for
> system
> > > > > > internal usage.
> > > > > > > > But it's another topic as we already have "RECONSUMETIMES".
> > > > > > > >
> > > > > > > >
> > > > > > > > On Sun, Jan 8, 2023 at 12:27 PM Nitin Goyal <
> > > nitin.goyal.dev@gmail.com>
> > > > > > wrote:
> > > > > > > > >
> > > > > > > > > Hello community,
> > > > > > > > >
> > > > > > > > > I'm starting the VOTE for PIP-239: Retry Mechanism per
> > Message
> > > title.
> > > > > > > > >
> > > > > > > > > Motivation: We are working on a project where each message
> > has
> > > their
> > > > > > retry
> > > > > > > > > as per the requirements. like one message 100 times and
> other
> > > > > > messages 5
> > > > > > > > > times and so on. This feature also adds an extra
> > functionality
> > > while
> > > > > > > > > comparing existing queueing services like RabbitMQ or SQS
> > etc.
> > > > > > > > >
> > > > > > > > > For more details please find the detailed PIP at:
> > > > > > > > > https://github.com/apache/pulsar/issues/19136
> > > > > > > > >
> > > > > > > > > Discussion Threads:
> > > > > > > > > 1.
> > > https://lists.apache.org/thread/rn6114xxtx1j57yy2jbmdv4xzt80o1hz
> > > > > > > > > 2.
> > > https://lists.apache.org/thread/b9rfv6t307z439xx3zt2ym4p140qzp06
> > > > > > > > >
> > > > > > > > > Proposed changes in pulsar go client library.
> > > > > > > > > https://github.com/apache/pulsar-client-go/pull/939
> > > > > > > > >
> > > > > > > > > Thanks
> > > > > > > > > Nitin Goyal
> > > > > >
> > > > >
> > >
> >
>
>
> --
> Regards
> Nitin Goyal
>
Re: [VOTE] PIP-239: Retry Mechanism per Message title
Posted by Nitin Goyal <ni...@gmail.com>.
Hello Xiaolong,
About your concern on NACK and reconsumeLater. Also as per current
Architecture rheir work is completely different from each other. which is
as follows.
1. NACK does not send messages to DLQ as per current architecture.; it
keeps messages in consumer memory to redeliver them later to the
same consumer. but the retry will allow adding custom delay to the message
and the same msg can be processed by another consumer from the same
consumer group as per their configuration (i.e. shared types).
2. If a message is NACKed the message will get consumed infinite time based
on backoff limit till it gets ACKed. where a reconsume later message will
be sent to DLQ onces it reaches the retry limit set by the consumer.
3. if a message NACKed then. How many times it retries is also in memory if
the consumer has failed or restarted all this information is lost. But in
the case of reconsumeLater all these things persisted in the broker.
On Tue, Jan 10, 2023 at 6:53 PM rxl@apache.org <ra...@gmail.com>
wrote:
> -1(non-binding)
>
> As discussed in [DISCUSS], for ReconsumeLater, I prefer to discard this
> interface in later versions, and think about this issue from another angle.
> From a user’s point of view, both ReconsumeLater and Nack have the function
> of retweeting messages. From a functional point of view, it seems that
> there is not much difference between the two. So how do we advise users
> under what circumstances? It is more appropriate to use ReconsumeLater, and
> under what circumstances is it more appropriate to use Nack? For me, I
> don't know how to introduce it to users. I can only explain it based on
> their implementation logic. ReconsumeLater sends messages to the Retry
> Topic, and Nack sends messages to the DLQ Topic. But Nack can completely
> replace the logic here. Is it necessary for us to introduce a new
> ReconsumeLater interface to confuse users.
>
> So here I hope we can discuss whether we need to support this feature or
> logic from the perspective of Nack.
>
> --
> Thanks
> Xiaolong Ran
>
> Enrico Olivelli <eo...@gmail.com> 于2023年1月9日周一 15:36写道：
>
> > I am sure that this is a good idea.
> >
> > -1 (binding)
> >
> > I will follow up on the discussion thread
> >
> > I am sorry I am catching up late on this thread
> >
> > Enrico
> >
> > Il giorno lun 9 gen 2023 alle ore 03:51 Ran Gao <rg...@apache.org> ha
> > scritto:
> > >
> > > +1 (non-binding)
> > >
> > > Thanks,
> > > Ran Gao
> > >
> > > On 2023/01/08 15:05:31 Zixuan Liu wrote:
> > > > +1 (non-binding)
> > > >
> > > > Thanks,
> > > > Zixuan
> > > >
> > > > Yunze Xu <yz...@streamnative.io.invalid> 于2023年1月8日周日 22:38写道：
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > Thanks,
> > > > > Yunze
> > > > >
> > > > > On Sun, Jan 8, 2023 at 4:46 PM Zike Yang <zi...@apache.org> wrote:
> > > > > >
> > > > > > +1 (non-binding)
> > > > > >
> > > > > > Thanks,
> > > > > > Zike Yang
> > > > > >
> > > > > > On Sun, Jan 8, 2023 at 3:22 PM Haiting Jiang <
> > jianghaiting@gmail.com>
> > > > > wrote:
> > > > > > >
> > > > > > > +1 binding
> > > > > > >
> > > > > > > Although I think we should use more unique keys for system
> > properties
> > > > > in pulsar,
> > > > > > > e.g. reserve all properties prefixed with "PULSAR_" for system
> > > > > internal usage.
> > > > > > > But it's another topic as we already have "RECONSUMETIMES".
> > > > > > >
> > > > > > >
> > > > > > > On Sun, Jan 8, 2023 at 12:27 PM Nitin Goyal <
> > nitin.goyal.dev@gmail.com>
> > > > > wrote:
> > > > > > > >
> > > > > > > > Hello community,
> > > > > > > >
> > > > > > > > I'm starting the VOTE for PIP-239: Retry Mechanism per
> Message
> > title.
> > > > > > > >
> > > > > > > > Motivation: We are working on a project where each message
> has
> > their
> > > > > retry
> > > > > > > > as per the requirements. like one message 100 times and other
> > > > > messages 5
> > > > > > > > times and so on. This feature also adds an extra
> functionality
> > while
> > > > > > > > comparing existing queueing services like RabbitMQ or SQS
> etc.
> > > > > > > >
> > > > > > > > For more details please find the detailed PIP at:
> > > > > > > > https://github.com/apache/pulsar/issues/19136
> > > > > > > >
> > > > > > > > Discussion Threads:
> > > > > > > > 1.
> > https://lists.apache.org/thread/rn6114xxtx1j57yy2jbmdv4xzt80o1hz
> > > > > > > > 2.
> > https://lists.apache.org/thread/b9rfv6t307z439xx3zt2ym4p140qzp06
> > > > > > > >
> > > > > > > > Proposed changes in pulsar go client library.
> > > > > > > > https://github.com/apache/pulsar-client-go/pull/939
> > > > > > > >
> > > > > > > > Thanks
> > > > > > > > Nitin Goyal
> > > > >
> > > >
> >
>
--
Regards
Nitin Goyal
Re: [VOTE] PIP-239: Retry Mechanism per Message title
Posted by "rxl@apache.org" <ra...@gmail.com>.
-1(non-binding)
As discussed in [DISCUSS], for ReconsumeLater, I prefer to discard this
interface in later versions, and think about this issue from another angle.
From a user’s point of view, both ReconsumeLater and Nack have the function
of retweeting messages. From a functional point of view, it seems that
there is not much difference between the two. So how do we advise users
under what circumstances? It is more appropriate to use ReconsumeLater, and
under what circumstances is it more appropriate to use Nack? For me, I
don't know how to introduce it to users. I can only explain it based on
their implementation logic. ReconsumeLater sends messages to the Retry
Topic, and Nack sends messages to the DLQ Topic. But Nack can completely
replace the logic here. Is it necessary for us to introduce a new
ReconsumeLater interface to confuse users.
So here I hope we can discuss whether we need to support this feature or
logic from the perspective of Nack.
--
Thanks
Xiaolong Ran
Enrico Olivelli <eo...@gmail.com> 于2023年1月9日周一 15:36写道：
> I am sure that this is a good idea.
>
> -1 (binding)
>
> I will follow up on the discussion thread
>
> I am sorry I am catching up late on this thread
>
> Enrico
>
> Il giorno lun 9 gen 2023 alle ore 03:51 Ran Gao <rg...@apache.org> ha
> scritto:
> >
> > +1 (non-binding)
> >
> > Thanks,
> > Ran Gao
> >
> > On 2023/01/08 15:05:31 Zixuan Liu wrote:
> > > +1 (non-binding)
> > >
> > > Thanks,
> > > Zixuan
> > >
> > > Yunze Xu <yz...@streamnative.io.invalid> 于2023年1月8日周日 22:38写道：
> > >
> > > > +1 (binding)
> > > >
> > > > Thanks,
> > > > Yunze
> > > >
> > > > On Sun, Jan 8, 2023 at 4:46 PM Zike Yang <zi...@apache.org> wrote:
> > > > >
> > > > > +1 (non-binding)
> > > > >
> > > > > Thanks,
> > > > > Zike Yang
> > > > >
> > > > > On Sun, Jan 8, 2023 at 3:22 PM Haiting Jiang <
> jianghaiting@gmail.com>
> > > > wrote:
> > > > > >
> > > > > > +1 binding
> > > > > >
> > > > > > Although I think we should use more unique keys for system
> properties
> > > > in pulsar,
> > > > > > e.g. reserve all properties prefixed with "PULSAR_" for system
> > > > internal usage.
> > > > > > But it's another topic as we already have "RECONSUMETIMES".
> > > > > >
> > > > > >
> > > > > > On Sun, Jan 8, 2023 at 12:27 PM Nitin Goyal <
> nitin.goyal.dev@gmail.com>
> > > > wrote:
> > > > > > >
> > > > > > > Hello community,
> > > > > > >
> > > > > > > I'm starting the VOTE for PIP-239: Retry Mechanism per Message
> title.
> > > > > > >
> > > > > > > Motivation: We are working on a project where each message has
> their
> > > > retry
> > > > > > > as per the requirements. like one message 100 times and other
> > > > messages 5
> > > > > > > times and so on. This feature also adds an extra functionality
> while
> > > > > > > comparing existing queueing services like RabbitMQ or SQS etc.
> > > > > > >
> > > > > > > For more details please find the detailed PIP at:
> > > > > > > https://github.com/apache/pulsar/issues/19136
> > > > > > >
> > > > > > > Discussion Threads:
> > > > > > > 1.
> https://lists.apache.org/thread/rn6114xxtx1j57yy2jbmdv4xzt80o1hz
> > > > > > > 2.
> https://lists.apache.org/thread/b9rfv6t307z439xx3zt2ym4p140qzp06
> > > > > > >
> > > > > > > Proposed changes in pulsar go client library.
> > > > > > > https://github.com/apache/pulsar-client-go/pull/939
> > > > > > >
> > > > > > > Thanks
> > > > > > > Nitin Goyal
> > > >
> > >
>
Re: [VOTE] PIP-239: Retry Mechanism per Message title
Posted by Enrico Olivelli <eo...@gmail.com>.
I am sure that this is a good idea.
-1 (binding)
I will follow up on the discussion thread
I am sorry I am catching up late on this thread
Enrico
Il giorno lun 9 gen 2023 alle ore 03:51 Ran Gao <rg...@apache.org> ha scritto:
>
> +1 (non-binding)
>
> Thanks,
> Ran Gao
>
> On 2023/01/08 15:05:31 Zixuan Liu wrote:
> > +1 (non-binding)
> >
> > Thanks,
> > Zixuan
> >
> > Yunze Xu <yz...@streamnative.io.invalid> 于2023年1月8日周日 22:38写道：
> >
> > > +1 (binding)
> > >
> > > Thanks,
> > > Yunze
> > >
> > > On Sun, Jan 8, 2023 at 4:46 PM Zike Yang <zi...@apache.org> wrote:
> > > >
> > > > +1 (non-binding)
> > > >
> > > > Thanks,
> > > > Zike Yang
> > > >
> > > > On Sun, Jan 8, 2023 at 3:22 PM Haiting Jiang <ji...@gmail.com>
> > > wrote:
> > > > >
> > > > > +1 binding
> > > > >
> > > > > Although I think we should use more unique keys for system properties
> > > in pulsar,
> > > > > e.g. reserve all properties prefixed with "PULSAR_" for system
> > > internal usage.
> > > > > But it's another topic as we already have "RECONSUMETIMES".
> > > > >
> > > > >
> > > > > On Sun, Jan 8, 2023 at 12:27 PM Nitin Goyal <ni...@gmail.com>
> > > wrote:
> > > > > >
> > > > > > Hello community,
> > > > > >
> > > > > > I'm starting the VOTE for PIP-239: Retry Mechanism per Message title.
> > > > > >
> > > > > > Motivation: We are working on a project where each message has their
> > > retry
> > > > > > as per the requirements. like one message 100 times and other
> > > messages 5
> > > > > > times and so on. This feature also adds an extra functionality while
> > > > > > comparing existing queueing services like RabbitMQ or SQS etc.
> > > > > >
> > > > > > For more details please find the detailed PIP at:
> > > > > > https://github.com/apache/pulsar/issues/19136
> > > > > >
> > > > > > Discussion Threads:
> > > > > > 1. https://lists.apache.org/thread/rn6114xxtx1j57yy2jbmdv4xzt80o1hz
> > > > > > 2. https://lists.apache.org/thread/b9rfv6t307z439xx3zt2ym4p140qzp06
> > > > > >
> > > > > > Proposed changes in pulsar go client library.
> > > > > > https://github.com/apache/pulsar-client-go/pull/939
> > > > > >
> > > > > > Thanks
> > > > > > Nitin Goyal
> > >
> >
Re: [VOTE] PIP-239: Retry Mechanism per Message title
Posted by Ran Gao <rg...@apache.org>.
+1 (non-binding)
Thanks,
Ran Gao
On 2023/01/08 15:05:31 Zixuan Liu wrote:
> +1 (non-binding)
>
> Thanks,
> Zixuan
>
> Yunze Xu <yz...@streamnative.io.invalid> 于2023年1月8日周日 22:38写道：
>
> > +1 (binding)
> >
> > Thanks,
> > Yunze
> >
> > On Sun, Jan 8, 2023 at 4:46 PM Zike Yang <zi...@apache.org> wrote:
> > >
> > > +1 (non-binding)
> > >
> > > Thanks,
> > > Zike Yang
> > >
> > > On Sun, Jan 8, 2023 at 3:22 PM Haiting Jiang <ji...@gmail.com>
> > wrote:
> > > >
> > > > +1 binding
> > > >
> > > > Although I think we should use more unique keys for system properties
> > in pulsar,
> > > > e.g. reserve all properties prefixed with "PULSAR_" for system
> > internal usage.
> > > > But it's another topic as we already have "RECONSUMETIMES".
> > > >
> > > >
> > > > On Sun, Jan 8, 2023 at 12:27 PM Nitin Goyal <ni...@gmail.com>
> > wrote:
> > > > >
> > > > > Hello community,
> > > > >
> > > > > I'm starting the VOTE for PIP-239: Retry Mechanism per Message title.
> > > > >
> > > > > Motivation: We are working on a project where each message has their
> > retry
> > > > > as per the requirements. like one message 100 times and other
> > messages 5
> > > > > times and so on. This feature also adds an extra functionality while
> > > > > comparing existing queueing services like RabbitMQ or SQS etc.
> > > > >
> > > > > For more details please find the detailed PIP at:
> > > > > https://github.com/apache/pulsar/issues/19136
> > > > >
> > > > > Discussion Threads:
> > > > > 1. https://lists.apache.org/thread/rn6114xxtx1j57yy2jbmdv4xzt80o1hz
> > > > > 2. https://lists.apache.org/thread/b9rfv6t307z439xx3zt2ym4p140qzp06
> > > > >
> > > > > Proposed changes in pulsar go client library.
> > > > > https://github.com/apache/pulsar-client-go/pull/939
> > > > >
> > > > > Thanks
> > > > > Nitin Goyal
> >
>
Re: [VOTE] PIP-239: Retry Mechanism per Message title
Posted by Zixuan Liu <no...@gmail.com>.
+1 (non-binding)
Thanks,
Zixuan
Yunze Xu <yz...@streamnative.io.invalid> 于2023年1月8日周日 22:38写道：
> +1 (binding)
>
> Thanks,
> Yunze
>
> On Sun, Jan 8, 2023 at 4:46 PM Zike Yang <zi...@apache.org> wrote:
> >
> > +1 (non-binding)
> >
> > Thanks,
> > Zike Yang
> >
> > On Sun, Jan 8, 2023 at 3:22 PM Haiting Jiang <ji...@gmail.com>
> wrote:
> > >
> > > +1 binding
> > >
> > > Although I think we should use more unique keys for system properties
> in pulsar,
> > > e.g. reserve all properties prefixed with "PULSAR_" for system
> internal usage.
> > > But it's another topic as we already have "RECONSUMETIMES".
> > >
> > >
> > > On Sun, Jan 8, 2023 at 12:27 PM Nitin Goyal <ni...@gmail.com>
> wrote:
> > > >
> > > > Hello community,
> > > >
> > > > I'm starting the VOTE for PIP-239: Retry Mechanism per Message title.
> > > >
> > > > Motivation: We are working on a project where each message has their
> retry
> > > > as per the requirements. like one message 100 times and other
> messages 5
> > > > times and so on. This feature also adds an extra functionality while
> > > > comparing existing queueing services like RabbitMQ or SQS etc.
> > > >
> > > > For more details please find the detailed PIP at:
> > > > https://github.com/apache/pulsar/issues/19136
> > > >
> > > > Discussion Threads:
> > > > 1. https://lists.apache.org/thread/rn6114xxtx1j57yy2jbmdv4xzt80o1hz
> > > > 2. https://lists.apache.org/thread/b9rfv6t307z439xx3zt2ym4p140qzp06
> > > >
> > > > Proposed changes in pulsar go client library.
> > > > https://github.com/apache/pulsar-client-go/pull/939
> > > >
> > > > Thanks
> > > > Nitin Goyal
>
Re: [VOTE] PIP-239: Retry Mechanism per Message title
Posted by Yunze Xu <yz...@streamnative.io.INVALID>.
+1 (binding)
Thanks,
Yunze
On Sun, Jan 8, 2023 at 4:46 PM Zike Yang <zi...@apache.org> wrote:
>
> +1 (non-binding)
>
> Thanks,
> Zike Yang
>
> On Sun, Jan 8, 2023 at 3:22 PM Haiting Jiang <ji...@gmail.com> wrote:
> >
> > +1 binding
> >
> > Although I think we should use more unique keys for system properties in pulsar,
> > e.g. reserve all properties prefixed with "PULSAR_" for system internal usage.
> > But it's another topic as we already have "RECONSUMETIMES".
> >
> >
> > On Sun, Jan 8, 2023 at 12:27 PM Nitin Goyal <ni...@gmail.com> wrote:
> > >
> > > Hello community,
> > >
> > > I'm starting the VOTE for PIP-239: Retry Mechanism per Message title.
> > >
> > > Motivation: We are working on a project where each message has their retry
> > > as per the requirements. like one message 100 times and other messages 5
> > > times and so on. This feature also adds an extra functionality while
> > > comparing existing queueing services like RabbitMQ or SQS etc.
> > >
> > > For more details please find the detailed PIP at:
> > > https://github.com/apache/pulsar/issues/19136
> > >
> > > Discussion Threads:
> > > 1. https://lists.apache.org/thread/rn6114xxtx1j57yy2jbmdv4xzt80o1hz
> > > 2. https://lists.apache.org/thread/b9rfv6t307z439xx3zt2ym4p140qzp06
> > >
> > > Proposed changes in pulsar go client library.
> > > https://github.com/apache/pulsar-client-go/pull/939
> > >
> > > Thanks
> > > Nitin Goyal
Re: [VOTE] PIP-239: Retry Mechanism per Message title
Posted by Zike Yang <zi...@apache.org>.
+1 (non-binding)
Thanks,
Zike Yang
On Sun, Jan 8, 2023 at 3:22 PM Haiting Jiang <ji...@gmail.com> wrote:
>
> +1 binding
>
> Although I think we should use more unique keys for system properties in pulsar,
> e.g. reserve all properties prefixed with "PULSAR_" for system internal usage.
> But it's another topic as we already have "RECONSUMETIMES".
>
>
> On Sun, Jan 8, 2023 at 12:27 PM Nitin Goyal <ni...@gmail.com> wrote:
> >
> > Hello community,
> >
> > I'm starting the VOTE for PIP-239: Retry Mechanism per Message title.
> >
> > Motivation: We are working on a project where each message has their retry
> > as per the requirements. like one message 100 times and other messages 5
> > times and so on. This feature also adds an extra functionality while
> > comparing existing queueing services like RabbitMQ or SQS etc.
> >
> > For more details please find the detailed PIP at:
> > https://github.com/apache/pulsar/issues/19136
> >
> > Discussion Threads:
> > 1. https://lists.apache.org/thread/rn6114xxtx1j57yy2jbmdv4xzt80o1hz
> > 2. https://lists.apache.org/thread/b9rfv6t307z439xx3zt2ym4p140qzp06
> >
> > Proposed changes in pulsar go client library.
> > https://github.com/apache/pulsar-client-go/pull/939
> >
> > Thanks
> > Nitin Goyal
Re: [VOTE] PIP-239: Retry Mechanism per Message title
Posted by Haiting Jiang <ji...@gmail.com>.
+1 binding
Although I think we should use more unique keys for system properties in pulsar,
e.g. reserve all properties prefixed with "PULSAR_" for system internal usage.
But it's another topic as we already have "RECONSUMETIMES".
On Sun, Jan 8, 2023 at 12:27 PM Nitin Goyal <ni...@gmail.com> wrote:
>
> Hello community,
>
> I'm starting the VOTE for PIP-239: Retry Mechanism per Message title.
>
> Motivation: We are working on a project where each message has their retry
> as per the requirements. like one message 100 times and other messages 5
> times and so on. This feature also adds an extra functionality while
> comparing existing queueing services like RabbitMQ or SQS etc.
>
> For more details please find the detailed PIP at:
> https://github.com/apache/pulsar/issues/19136
>
> Discussion Threads:
> 1. https://lists.apache.org/thread/rn6114xxtx1j57yy2jbmdv4xzt80o1hz
> 2. https://lists.apache.org/thread/b9rfv6t307z439xx3zt2ym4p140qzp06
>
> Proposed changes in pulsar go client library.
> https://github.com/apache/pulsar-client-go/pull/939
>
> Thanks
> Nitin Goyal