You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2019/10/21 09:11:03 UTC
Slack digest for #general - 2019-10-21
2019-10-20 09:13:35 UTC - Retardust: Is there any contribution guide?)
----
2019-10-20 12:18:51 UTC - jia zhai: @Retardust here it it: <http://pulsar.apache.org/en/contributing/>
----
2019-10-20 12:51:39 UTC - Bhavesh Rajdev: @Bhavesh Rajdev has joined the channel
----
2019-10-20 13:07:08 UTC - Matteo Merli: I don’t understand the problem though. you can always attach an application defined UUID to every message, for example as part as message properties.
----
2019-10-20 13:11:56 UTC - Matteo Merli: > Avoiding the difficulties of generating sequential IDs in a distributed system is why people use UUIDs in the first place...
A sequence Id is relative to a single producer, so it’s not a “distributed sequencer” problem. Each process just use a local counter.
The combo “producer-name” plus “sequence-is”, though, is globally unique and also is sequential, which is what we need to perform message deduplication.
----
2019-10-20 13:25:15 UTC - Retardust: <http://pulsar.apache.org/en/coding_guide>
:disappointed: 404
----
2019-10-20 13:40:52 UTC - Sijie Guo: Currently Pulsar client doesn’t retry creating producer or consumer when Pulsar is not available. After a producer or consumer is created, the operations are retried.
Feel free to create a issue for improving this behavior. Also contributions are welcome
----
2019-10-20 14:06:07 UTC - Raman Gupta: If I understand you correctly, in both of your "solutions" your're essentially saying: "Pulsar can't help you, do it yourself."
Attaching the UUID as a property means that consumer application code needs to use that value to ensure any given message is not processed twice, as Pulsar won't de-dup on that value.
And generating a synthetic sequence id that is not an inherent property of the input (such as a line number in a file), and having the application do that in the face of things going wrong (crashes, OOMs, etc.) which is when you need de-dup in the first place, is not a simple matter of just incrementing a counter on every send.
----
2019-10-20 14:09:35 UTC - Retardust: where is a lot of problems highlighted by IDE:(
are you folks OK with refactoring related pull-requests?
Is there a better place to have a conversation about contribution? maybe <#C5ZSVEN4E|dev>?
----
2019-10-20 14:13:43 UTC - Matteo Merli: Problem of deduping with UUIDs is that you need to consider the entire space of all the generated UUIDs. That’s impractical or impossible is most cases, in the context of messaging. Sequence ids allows to retain the same guarantee but just consider a single number, for each producer.
----
2019-10-20 16:27:21 UTC - trekker: @trekker has joined the channel
----
2019-10-20 18:11:47 UTC - Nicolas Ha: I am seeing this in the Java client doc `void redeliverUnacknowledgedMessages()` - could someone give an overview of the typical use-case for this?
----
2019-10-20 18:14:43 UTC - Nicolas Ha: You mean “stop-the-world” garbage collect right? I see, thanks
----
2019-10-20 18:34:50 UTC - Nicolas Ha: I was under the impression that negativeAcknowledged messages were immediately re-delivered but that may not be the case after all?
----
2019-10-21 01:11:57 UTC - jia zhai: <#C5ZSVEN4E|dev> also OK for this kind of discussing.
----
2019-10-21 01:13:54 UTC - jia zhai: opened an issue in github for the 404 page.
----
2019-10-21 01:14:24 UTC - jia zhai: very welcome for your contribute to the refactoring.:grinning:
+1 : Retardust
----
2019-10-21 05:51:56 UTC - Retardust: also doesnt see any publishers in stats:) any ideas?
----
2019-10-21 06:18:00 UTC - Retardust: Doesn't find any info about MessageListener and acknowledgement. After researching sources I found that I should manually ack message. consumer doestn't do that. it's should be documented on interface level if tgat desired behaviour
----
2019-10-21 06:55:03 UTC - Yong Zhang: <https://github.com/apache/pulsar/blob/master/pulsar-broker/src/test/java/org/apache/pulsar/client/impl/UnAcknowledgedMessagesTimeoutTest.java>
----
2019-10-21 07:28:56 UTC - Nicolas Ha: I don’t see `redeliverUnacknowledgedMessages` used in there?
----
2019-10-21 07:30:01 UTC - Nicolas Ha: I see only <https://github.com/apache/pulsar/blob/master/pulsar-broker/src/test/java/org/apache/pulsar/client/impl/UnAcknowledgedMessagesTimeoutTest.java#L97> - so it seems to me from reading that that `redeliverUnacknowledgedMessages` is useless?
----
2019-10-21 07:35:49 UTC - Retardust: I will pr on that if Im right
----
2019-10-21 08:51:38 UTC - Vladimir Shchur: Why did you even expect it to be automatically acked?
----
2019-10-21 09:11:00 UTC - Nicolas Ha: Also I somethimes see `Reconnecting the client to redeliver the messages` in my logs even though I don’t call `redeliverUnacknowledgedMessages` myself
----