You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2020/08/08 09:11:04 UTC
Slack digest for #dev - 2020-08-08
2020-08-07 12:23:00 UTC - Nicolas Ha: Looking at pulsar internals out of curiosity, in particular ConsumerImpl - I get that there is a receiver queue that gets re-filled from the broker and that the user can set how big it is.
I don’t quite get why the queue would have to be emptied?
<https://github.com/apache/pulsar/blob/fffd9f144bb14a220d17e951fea29b16ad2db103/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java#L860-L864>
Maybe that’s because I don’t get the difference between `incomingMessages` and `pendingReceives`
<https://github.com/apache/pulsar/blob/48156ad9a5c2e0d85813367bcaaf6ea845fffc2c/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerBase.java#L70>
----
2020-08-08 04:30:13 UTC - Chris Wong: @Chris Wong has joined the channel
----
2020-08-08 04:38:07 UTC - Chris Wong: <!here> I found a potential bug for `2.6.0` on `KEY_SHARED` mode. Could anyone help verify it? Thanks a lot.
Let's move to the story. We wanna consume messages via `KEY_SHARED` mode, so we have the Producer as following and set the Consumer as `KEY_SHARED`.
```Producer<String> producer = client.newProducer(Schema.STRING)
.topic(prefix + 0)
.enableBatching(true)
.batchingMaxBytes(128 * 1024 * 1024)
.batchingMaxMessages(200000)
.roundRobinRouterBatchingPartitionSwitchFrequency(10)
.batchingMaxPublishDelay(1, TimeUnit.MILLISECONDS)
.compressionType(CompressionType.LZ4)
.batcherBuilder(BatcherBuilder.KEY_BASED)
.sendTimeout(10, TimeUnit.SECONDS)
.blockIfQueueFull(true)
.create();```
We found the messages couldn't be consumed with compression and it worked after we removed `compressionType(CompressionType.LZ4)`. However, those segments of code are working totally well on `2.5.2`.
----
2020-08-08 04:42:00 UTC - Sijie Guo: @Penghui Li ^ Can you help check the key_shared issue?
----
2020-08-08 05:27:09 UTC - Penghui Li: @Chris Wong Looks the issue is <https://github.com/apache/pulsar/pull/7416> that fixed and #7416 will release in 2.6.1 (next week).
----
2020-08-08 08:25:52 UTC - Chris Wong: Thanks @Penghui Li
----