[GitHub] [pulsar] stupidusername edited a comment on issue #5157: [docs] How does --processing-guarantees affect source connectors?

[GitBox <gi...@apache.org>]

stupidusername edited a comment on issue #5157:
URL: https://github.com/apache/pulsar/issues/5157#issuecomment-633390668


   I've been doing some tests using this source connector: [sequence_source](https://github.com/Antti-Kaikkonen/PulsarSequenceSource).
   
   I tested different combinations of `--processing-guarantees` and `--parallelism` and consumed the published messages.
   
   - Using `EFFECTIVELY_ONCE` and `--parallelism` set to `1` I got one sequence of ordered numbers.
   - Using `ATLEAST_ONCE` and `--parallelism` set to `1` I got one sequence of ordered numbers.
   - Using `EFFECTIVELY_ONCE` and `--parallelism` set to `4` I got one sequence of ordered numbers.
   - Using `ATLEAST_ONCE` and `--parallelism` set to `4` I got what it seems to be four sequences of numbers, each one of them starting from 0.
   - Using `EFFECTIVELY_ONCE` and `--parallelism` set to `4` with a modified version of the source connector that uses a different `partitionId` for each instance of the connector I also got four sequences of numbers.
   
   The tests were conducted on a standalone instance of Pulsar. Deduplication was not set.
   
   So It looks like when ussing `EFFECTIVELY_ONCE` duplicated messages are beign removed even though deduplication is not set on a namespace level. Is this observation correct?
   
   Also, I would like to know if each instance of the source connector has the guarantee of writing its messages in order to the topic. If that's true: How is it achieved?
   
   Thanks!


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org