You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@rocketmq.apache.org by 柳尘 <yu...@gmail.com> on 2022/10/19 06:01:38 UTC

[DISCUSS] RIP-53 Producer idempotent send

Hi, RocketMQ Community,
This question is about the producer idempotent send, the purpose is to do
it by setting up a transaction coordinator

We have written the proposal and you and see it by the link below:

https://docs.google.com/document/d/1nResLevPbeGmKwSQiId_jw0tfBhJtoPZTduOQL3qxNg/edit

[ RIP-53 Producer idempotent send ]

Please reply to this email if you have any questions

Re: [DISCUSS] RIP-53 Producer idempotent send

Posted by 柳尘 <yu...@gmail.com>.

From this, I think producerId + epoch can guarantee the order of sending or
consuming the same message

柳尘 <yu...@gmail.com> 于2022年10月21日周五 11:12写道：

> Hello, in the example diagram in RIP-53, sequences 4.1 and 4.3 synchronize
> the mapping of <TID, PID, Epoch> on the producer side after opening the
> transaction and on the consumer side after committing the transaction,
> respectively. Among these information, the transcationId represents the
> consistency across partitions, and the epoch can represent the time
> sequence in which the message was submitted.
>
>
> [image: W1)EOE{}Z]V]}AV@J44(U2.png]
> Not sure if this answers your question?
>
> Xinyu Zhou <yu...@apache.org> 于2022年10月20日周四 10:40写道：
>
>> I noticed that the introduced APIs are similar to Kafka. I have a
>> technical
>> question:
>>
>> Kafka commits offset through sending messages to an internal topic, so
>> Kafka can combine the send and consume operation in a single transaction
>> through the atomic multi-partition writes feature. But RocketMQ doesn't
>> have the same mechanism, do we have any other ingenious solutions to reach
>> a similar effect like that?
>>
>> Regards,
>> Xinyu Zhou
>>
>>
>>
>> On Wed, Oct 19, 2022 at 4:03 PM 柳尘 <yu...@gmail.com> wrote:
>>
>> > ok i'm waiting for vote day to pass
>> >
>> > Amber Liu <lt...@gmail.com> 于2022年10月19日周三 15:01写道：
>> >
>> > > +1,  I am interested in this RIP too and maybe we can complete this
>> task
>> > > together.
>> > >
>> > > 柳尘 <yu...@gmail.com> 于2022年10月19日周三 14:01写道：
>> > >
>> > > > Hi, RocketMQ Community,
>> > > > This question is about the producer idempotent send, the purpose is
>> to
>> > do
>> > > > it by setting up a transaction coordinator
>> > > >
>> > > > We have written the proposal and you and see it by the link below:
>> > > >
>> > > >
>> > > >
>> > >
>> >
>> https://docs.google.com/document/d/1nResLevPbeGmKwSQiId_jw0tfBhJtoPZTduOQL3qxNg/edit
>> > > >
>> > > > [ RIP-53 Producer idempotent send ]
>> > > >
>> > > > Please reply to this email if you have any questions
>> > > >
>> > >
>> >
>>
>

Re: [DISCUSS] RIP-53 Producer idempotent send

Posted by 柳尘 <yu...@gmail.com>.

Hello, in the example diagram in RIP-53, sequences 4.1 and 4.3 synchronize
the mapping of <TID, PID, Epoch> on the producer side after opening the
transaction and on the consumer side after committing the transaction,
respectively. Among these information, the transcationId represents the
consistency across partitions, and the epoch can represent the time
sequence in which the message was submitted.


[image: W1)EOE{}Z]V]}AV@J44(U`2.png]
Not sure if this answers your question?

Xinyu Zhou <yu...@apache.org> 于2022年10月20日周四 10:40写道：

> I noticed that the introduced APIs are similar to Kafka. I have a technical
> question:
>
> Kafka commits offset through sending messages to an internal topic, so
> Kafka can combine the send and consume operation in a single transaction
> through the atomic multi-partition writes feature. But RocketMQ doesn't
> have the same mechanism, do we have any other ingenious solutions to reach
> a similar effect like that?
>
> Regards,
> Xinyu Zhou
>
>
>
> On Wed, Oct 19, 2022 at 4:03 PM 柳尘 <yu...@gmail.com> wrote:
>
> > ok i'm waiting for vote day to pass
> >
> > Amber Liu <lt...@gmail.com> 于2022年10月19日周三 15:01写道：
> >
> > > +1,  I am interested in this RIP too and maybe we can complete this
> task
> > > together.
> > >
> > > 柳尘 <yu...@gmail.com> 于2022年10月19日周三 14:01写道：
> > >
> > > > Hi, RocketMQ Community,
> > > > This question is about the producer idempotent send, the purpose is
> to
> > do
> > > > it by setting up a transaction coordinator
> > > >
> > > > We have written the proposal and you and see it by the link below:
> > > >
> > > >
> > > >
> > >
> >
> https://docs.google.com/document/d/1nResLevPbeGmKwSQiId_jw0tfBhJtoPZTduOQL3qxNg/edit
> > > >
> > > > [ RIP-53 Producer idempotent send ]
> > > >
> > > > Please reply to this email if you have any questions
> > > >
> > >
> >
>

Re: [DISCUSS] RIP-53 Producer idempotent send

Posted by Xinyu Zhou <yu...@apache.org>.

I noticed that the introduced APIs are similar to Kafka. I have a technical
question:

Kafka commits offset through sending messages to an internal topic, so
Kafka can combine the send and consume operation in a single transaction
through the atomic multi-partition writes feature. But RocketMQ doesn't
have the same mechanism, do we have any other ingenious solutions to reach
a similar effect like that?

Regards,
Xinyu Zhou

On Wed, Oct 19, 2022 at 4:03 PM 柳尘 <yu...@gmail.com> wrote:

> ok i'm waiting for vote day to pass
>
> Amber Liu <lt...@gmail.com> 于2022年10月19日周三 15:01写道：
>
> > +1,  I am interested in this RIP too and maybe we can complete this task
> > together.
> >
> > 柳尘 <yu...@gmail.com> 于2022年10月19日周三 14:01写道：
> >
> > > Hi, RocketMQ Community,
> > > This question is about the producer idempotent send, the purpose is to
> do
> > > it by setting up a transaction coordinator
> > >
> > > We have written the proposal and you and see it by the link below:
> > >
> > >
> > >
> >
> https://docs.google.com/document/d/1nResLevPbeGmKwSQiId_jw0tfBhJtoPZTduOQL3qxNg/edit
> > >
> > > [ RIP-53 Producer idempotent send ]
> > >
> > > Please reply to this email if you have any questions
> > >
> >
>

Re: [DISCUSS] RIP-53 Producer idempotent send

Posted by 柳尘 <yu...@gmail.com>.

ok i'm waiting for vote day to pass

Amber Liu <lt...@gmail.com> 于2022年10月19日周三 15:01写道：

> +1,  I am interested in this RIP too and maybe we can complete this task
> together.
>
> 柳尘 <yu...@gmail.com> 于2022年10月19日周三 14:01写道：
>
> > Hi, RocketMQ Community,
> > This question is about the producer idempotent send, the purpose is to do
> > it by setting up a transaction coordinator
> >
> > We have written the proposal and you and see it by the link below:
> >
> >
> >
> https://docs.google.com/document/d/1nResLevPbeGmKwSQiId_jw0tfBhJtoPZTduOQL3qxNg/edit
> >
> > [ RIP-53 Producer idempotent send ]
> >
> > Please reply to this email if you have any questions
> >
>

Re: [DISCUSS] RIP-53 Producer idempotent send

Posted by Amber Liu <lt...@gmail.com>.

+1,  I am interested in this RIP too and maybe we can complete this task
together.

柳尘 <yu...@gmail.com> 于2022年10月19日周三 14:01写道：

> Hi, RocketMQ Community,
> This question is about the producer idempotent send, the purpose is to do
> it by setting up a transaction coordinator
>
> We have written the proposal and you and see it by the link below:
>
>
> https://docs.google.com/document/d/1nResLevPbeGmKwSQiId_jw0tfBhJtoPZTduOQL3qxNg/edit
>
> [ RIP-53 Producer idempotent send ]
>
> Please reply to this email if you have any questions
>