You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2019/07/18 09:11:03 UTC
Slack digest for #general - 2019-07-18
2019-07-17 09:11:28 UTC - Sijie Guo: @Penghui Li: so sticky consumer can address their problem. but it requires the consumer applications to implement their own hash ring assignment. that requires too many efforts in consumer applications.
----
2019-07-17 09:12:25 UTC - Sijie Guo: sticky consumer is helpful in *Flink* like use cases, where Flink has its own instance management and state management.
----
2019-07-17 09:12:28 UTC - Yuvaraj Loganathan: @Olivier Chicha FailOver Subscription with partitioning key Guarantees the ordering. Why not use Failover subscription with mutiple partitions for scaling ?
----
2019-07-17 09:13:11 UTC - Penghui Li: Ok, i understand, will take a look. @Olivier Chicha Thanks for your feedback.
----
2019-07-17 09:15:21 UTC - Olivier Chicha: Thanks a lot,
Another idea that I had that would be nice, is to be able to be notified that the hash range of a consumer has changed.
It is very interesting (for tests, for logs, ....) to know that the set of key that are address by this consumer has changed.
this is already available for allocated partitions within the ConsumerEventListener,
I guess it would be consistant to be able to have a similar information for hash range:
rangeUpdated(rangeStart, rangeStop)
regards
+1 : Sijie Guo, Penghui Li, Yuvaraj Loganathan
----
2019-07-17 09:16:22 UTC - Sijie Guo: We will add that as well.
----
2019-07-17 09:17:13 UTC - Penghui Li: This is ok, we can send the changes to all consumers.
----
2019-07-17 09:20:37 UTC - Olivier Chicha: @Yuvaraj Loganathan yes this is what we were planning to use initially, but when we saw this new feature, we thought that it was simpler, more adapted and almost matching our needs, so we wanted to explore the possibility to use this rather than to be forced to partition our topics.
+1 : Yuvaraj Loganathan, Penghui Li, Sijie Guo
----
2019-07-17 09:23:05 UTC - Penghui Li: @Olivier Chicha <https://github.com/apache/pulsar/issues/4077> will track 2 tasks mentioned above
+1 : Sijie Guo
----
2019-07-17 09:25:36 UTC - Sijie Guo: We should forget “partition” someday :slightly_smiling_face:
sunglasses : Yuvaraj Loganathan
----
2019-07-17 09:36:40 UTC - Olivier Chicha: guys you are great, thank you so much for you reactivity
----
2019-07-17 10:52:23 UTC - Alexandre DUVAL: WDYT about <https://engineering.linkedin.com/blog/2019/brooklin-open-source>?
----
2019-07-17 12:40:04 UTC - alphazero: @alphazero has joined the channel
----
2019-07-17 12:47:22 UTC - alphazero: Hi. Have a question regarding the in-mem state of functions (SDK). Is this state strictly bound to the node running the function or is it distributed?
----
2019-07-17 13:13:15 UTC - Alexandre DUVAL: As negativeAck is not released on my client library yet, can I force topic to resend messages to a custom subscription which haven't been ack using cli?
----
2019-07-17 13:20:03 UTC - Alexandre DUVAL: Or I only can do it by restarting the subscription with MessageId.earliest?
----
2019-07-17 15:19:40 UTC - Aaron: Is there a way to turn off auto-topic creation from the java client? (not admin api)
----
2019-07-17 15:20:20 UTC - Sijie Guo: auto-topic creation can be disabled at the broker configuration
----
2019-07-17 15:21:29 UTC - Aaron: I know. Is there any way to restrict the client builder from connecting to the broker with a topic that hasn't been created yet? (without fudging with broker settings)
----
2019-07-17 15:21:46 UTC - sirghost: @sirghost has joined the channel
----
2019-07-17 15:24:19 UTC - Sijie Guo: Currently not. the behavior is controlled at broker.
----
2019-07-17 15:47:03 UTC - Grant Wu: What does a TTL of 0 mean?
----
2019-07-17 16:10:07 UTC - Matteo Merli: no TTL applied
----
2019-07-17 18:47:23 UTC - D'Andre: @D'Andre has joined the channel
----
2019-07-17 19:04:05 UTC - D'Andre: I have a question about the Java API jars from maven2. I cannot find the PulsarClient class anywhere. In fact, the entire org.apache.pulsar.client.api package is empty. Do I need to build the client from github for this to work?
----
2019-07-17 19:30:04 UTC - Ali Ahmed: @D'Andre what version are you referring to ?
----
2019-07-17 19:57:39 UTC - Jerry Peng: @Devin G. Bost for thread and process runtime for functions when the broker/worker restarts JARs will re-download
+1 : Devin G. Bost, Yong Zhang
----
2019-07-17 20:12:57 UTC - Jerry Peng: Brooklin is a bridging layer on top of a messaging systems. Someone can contribute a plugin for Pulsar
----
2019-07-17 20:18:35 UTC - D'Andre: @Ali Ahmed The site says that the latest is 2.4.0. That is what I have in my dependencies. I also tried 2.3.2, with the same result.
----
2019-07-17 20:19:15 UTC - Ali Ahmed: is this what are you using ?
----
2019-07-17 20:19:16 UTC - Ali Ahmed: <https://mvnrepository.com/artifact/org.apache.pulsar/pulsar-client/2.4.0>
----
2019-07-17 20:26:09 UTC - D'Andre: Yes
----
2019-07-17 20:33:59 UTC - Ali Ahmed: ```
import org.apache.pulsar.client.api.PulsarClient;
PulsarClient client = PulsarClient.builder()
.serviceUrl("")
.build();
```
----
2019-07-17 20:35:01 UTC - Ali Ahmed: can you try compiling the code block above/
----
2019-07-17 20:36:06 UTC - Balachandar Paulraj: @Balachandar Paulraj has joined the channel
----
2019-07-17 20:37:45 UTC - D'Andre:
----
2019-07-17 20:38:45 UTC - D'Andre:
----
2019-07-17 20:39:58 UTC - Ali Ahmed: works fine for me not sure what it wrong on your machine
----
2019-07-17 20:41:38 UTC - D'Andre: Oh well. Thank you for trying.
----
2019-07-17 20:44:57 UTC - D'Andre: I tried downloading the JAR from <https://search.maven.org/artifact/org.apache.pulsar/pulsar-client/2.4.0/jar> and there are no class files in 'org/apache/pulsar/client/api' only a url subdirectory. Very odd.
----
2019-07-17 20:45:27 UTC - Matteo Merli: the library has dependencies as well, Maven will fetch all of them
----
2019-07-17 20:45:52 UTC - Ali Ahmed: you can use gradle as well
----
2019-07-17 20:46:06 UTC - Matteo Merli: PulsarClient is part of `org.apache.pulsar:pulsar-client-api` which is a dependency of `org.apache.pulsar:pulsar-client`
----
2019-07-17 20:56:56 UTC - D'Andre: That was the issue. Thank you. The correct project name is 'pulsar-client-api'.
----
2019-07-17 20:57:18 UTC - Matteo Merli: You need to depend on `pulsar-client`
----
2019-07-17 20:59:30 UTC - D'Andre: I did, but ivy did not pull the dependency, Which is odd, since every other transitive dependency in the project is being built. Ultimately, I included both entries and Eclipse is happy.
----
2019-07-18 07:49:12 UTC - Shivji Kumar Jha: I am not sure if i have hit a bug or if i just need to turn on some configuration (pulsar client 2.4.0)
1. consumer.receive()
2. do not ack until ack timeout
3. The message is redelivered but the redeliveryCount remains 0, never increases.
----
2019-07-18 07:49:20 UTC - Shivji Kumar Jha: @Penghui Li
----
2019-07-18 07:50:19 UTC - Penghui Li: Ok, i will take a look
----
2019-07-18 07:54:04 UTC - Shivji Kumar Jha: @Penghui Li Does incrementing redeliveryCount require enabling dead letter topic on broker side by any chance?
----
2019-07-18 07:57:05 UTC - Penghui Li: I’m checking the code :grin:
----
2019-07-18 07:59:45 UTC - Penghui Li: Can you check the option of `subscriptionRedeliveryTrackerEnabled` in broker.conf?
----
2019-07-18 08:06:31 UTC - Shivji Kumar Jha: @Penghui Li sure, trying..
+1 : Penghui Li
----
2019-07-18 08:07:13 UTC - Shivji Kumar Jha: Its enabled already!
# Enable subscription message redelivery tracker to send redelivery count to consumer (default is enabled)
subscriptionRedeliveryTrackerEnabled=true
----
2019-07-18 08:08:22 UTC - Shivji Kumar Jha: @Penghui Li you could test this with SimpleProducerConsumerTest::testConsumerBlockingWithUnAckedMessages()
A dev complained and i can verify in this test that this indeed a bug :slightly_smiling_face:
I verified in a debug session btw…
----
2019-07-18 08:11:22 UTC - Penghui Li: @CongBo have test the redelivery count, seems it works, can you paste your test case?
----
2019-07-18 08:58:23 UTC - Shivji Kumar Jha: What promise does redelivery of message make?
Message will be delivered exactly (configured maxdeliverycount) times
OR
Message will be delivered >= (configured maxdeliverycount) times
----