You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "shibd (via GitHub)" <gi...@apache.org> on 2023/02/16 01:51:51 UTC
[GitHub] [pulsar] shibd commented on pull request #18929: [improve][fn] Support processingGuarantees "EFFECTIVELY_ONCE" in python function
shibd commented on PR #18929:
URL: https://github.com/apache/pulsar/pull/18929#issuecomment-1432356247
Hi,
> Thanks, @shibd, based on your comment, I think the mechanism of the effectively-once semantics should be as follows.
>
> The user needs to ensure that one source topic corresponds to one sink topic, at this time the principle of message delivery is: `source topic -> sink topic` If partition topic is enabled, it is necessary to ensure that the number of partitions of source topic and sink topic is the same, and ensure that the principle of message delivery follows: `source topic Pn -> sink topic Pn; source topic Pm -> sink topic Pm`
>
> Therefore, before using the effectively-once semantics, we should confirm the following points.
>
> 1. deduplication is enabled
> 2. function has only one source topic and one sink topic
> 3. if partition topic is enabled, we need to make sure the number of partitions is the same
>
> Please feel free to correct me, and what do you think @freeznet @nlu90
Hi, I agree 1, 2.
For 3, I think even if the source topic and sink topic have the same number of partitions, the function framework also does not guarantee that the data of `source topic [Pn]` will be sent to `sink topic [Pn]`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org