You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by Rajan Dhabalia <rd...@apache.org> on 2023/06/20 06:59:08 UTC
[VOTE] PIP-267: Support multi-topic messageId deserialization to ack messages
Hi.
Pulsar api provides MessageId interface which is generally used by producer
and consumer applications to manage topic offset. Sometimes, these
applications would like to serialize and deserialize messageIds,
specifically consumer app which would like to persist messageId and ack
with those messageIds by deserializing them. However, right now Pulsar
doesn't support correct deserialization of multi-topic or partitioned-topic
because of that 1acknowledge` API call fails for those topics with below
error:
"Only TopicMessageId is allowed to acknowledge for a multi-topics consumer"
Please visit PIP for any suggestions:
https://github.com/apache/pulsar/issues/20221
This PIP is created with PR: https://github.com/apache/pulsar/pull/19944
Thanks,
Rajan
Re: [VOTE] PIP-267: Support multi-topic messageId deserialization to ack messages
Posted by PengHui Li <pe...@apache.org>.
Hi Rajan,
I think we discuss the newly added field in the
PulsarApi.proto at
https://github.com/apache/pulsar/pull/19944#discussion_r1153963425
But the proposal doesn't mention it.
Although I know why we need to add that field to the proto file
to avoid introducing many changes to the client side since the client
had mixed the public API and internal data structure. But, in order
to give everyone who wants to know why this field was added in the future.
It should be clearly explained. The PIP is not only for
Pulsar's experienced veterans.
Only when we provide the context more comprehensively and keep it
transparent enough,
contributors get involved in the PIP review rather than wait until he
thinks he is
familiar enough with Pulsar to participate.
I will cast +0 here. I agree with the motivation and the solution because I
have the context
from the previous discussion under the pull request. But the proposal could
be more friendly to
community users and contributors.
Thanks,
Penghui
On Thu, Jun 22, 2023 at 12:08 AM 徐昀泽 <xy...@gmail.com> wrote:
> +1 (binding)
>
> Though I agree with Asaf that this proposal itself is not clear, I think
> the design
> Is easy to understand from the PR that a new field is added only for
> serialization
> and deserialization for a MessageId.
>
> Thanks,
> Yunze
>
> > On Jun 21, 2023, at 03:08, Asaf Mesika <as...@gmail.com> wrote:
> >
> > -1 (non-binding)
> >
> > The reason I'm asking all these questions on the DISCUSS is that I still
> > haven't managed to understand how you plan to solve the pain described.
> > Not to mention the lack of information in the design document I mentioned
> > in my replies to the discussion
> >
> > This DISCUSS thread is not resolved yet from my point of view.
> > The design document is not clear to me at all.
> >
> > Hence I would like to continue to understand it in the discussion thread.
> >
> >
> > On Tue, Jun 20, 2023 at 10:00 AM Rajan Dhabalia <rd...@apache.org>
> > wrote:
> >
> >> Hi.
> >>
> >> Pulsar api provides MessageId interface which is generally used by
> producer
> >> and consumer applications to manage topic offset. Sometimes, these
> >> applications would like to serialize and deserialize messageIds,
> >> specifically consumer app which would like to persist messageId and ack
> >> with those messageIds by deserializing them. However, right now Pulsar
> >> doesn't support correct deserialization of multi-topic or
> partitioned-topic
> >> because of that 1acknowledge` API call fails for those topics with below
> >> error:
> >> "Only TopicMessageId is allowed to acknowledge for a multi-topics
> consumer"
> >>
> >> Please visit PIP for any suggestions:
> >> https://github.com/apache/pulsar/issues/20221
> >>
> >> This PIP is created with PR:
> https://github.com/apache/pulsar/pull/19944
> >>
> >> Thanks,
> >> Rajan
> >>
>
>
Re: [VOTE] PIP-267: Support multi-topic messageId deserialization to ack messages
Posted by 徐昀泽 <xy...@gmail.com>.
+1 (binding)
Though I agree with Asaf that this proposal itself is not clear, I think the design
Is easy to understand from the PR that a new field is added only for serialization
and deserialization for a MessageId.
Thanks,
Yunze
> On Jun 21, 2023, at 03:08, Asaf Mesika <as...@gmail.com> wrote:
>
> -1 (non-binding)
>
> The reason I'm asking all these questions on the DISCUSS is that I still
> haven't managed to understand how you plan to solve the pain described.
> Not to mention the lack of information in the design document I mentioned
> in my replies to the discussion
>
> This DISCUSS thread is not resolved yet from my point of view.
> The design document is not clear to me at all.
>
> Hence I would like to continue to understand it in the discussion thread.
>
>
> On Tue, Jun 20, 2023 at 10:00 AM Rajan Dhabalia <rd...@apache.org>
> wrote:
>
>> Hi.
>>
>> Pulsar api provides MessageId interface which is generally used by producer
>> and consumer applications to manage topic offset. Sometimes, these
>> applications would like to serialize and deserialize messageIds,
>> specifically consumer app which would like to persist messageId and ack
>> with those messageIds by deserializing them. However, right now Pulsar
>> doesn't support correct deserialization of multi-topic or partitioned-topic
>> because of that 1acknowledge` API call fails for those topics with below
>> error:
>> "Only TopicMessageId is allowed to acknowledge for a multi-topics consumer"
>>
>> Please visit PIP for any suggestions:
>> https://github.com/apache/pulsar/issues/20221
>>
>> This PIP is created with PR: https://github.com/apache/pulsar/pull/19944
>>
>> Thanks,
>> Rajan
>>
Re: [VOTE] PIP-267: Support multi-topic messageId deserialization to ack messages
Posted by Asaf Mesika <as...@gmail.com>.
-1 (non-binding)
The reason I'm asking all these questions on the DISCUSS is that I still
haven't managed to understand how you plan to solve the pain described.
Not to mention the lack of information in the design document I mentioned
in my replies to the discussion
This DISCUSS thread is not resolved yet from my point of view.
The design document is not clear to me at all.
Hence I would like to continue to understand it in the discussion thread.
On Tue, Jun 20, 2023 at 10:00 AM Rajan Dhabalia <rd...@apache.org>
wrote:
> Hi.
>
> Pulsar api provides MessageId interface which is generally used by producer
> and consumer applications to manage topic offset. Sometimes, these
> applications would like to serialize and deserialize messageIds,
> specifically consumer app which would like to persist messageId and ack
> with those messageIds by deserializing them. However, right now Pulsar
> doesn't support correct deserialization of multi-topic or partitioned-topic
> because of that 1acknowledge` API call fails for those topics with below
> error:
> "Only TopicMessageId is allowed to acknowledge for a multi-topics consumer"
>
> Please visit PIP for any suggestions:
> https://github.com/apache/pulsar/issues/20221
>
> This PIP is created with PR: https://github.com/apache/pulsar/pull/19944
>
> Thanks,
> Rajan
>