You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2019/08/19 09:11:03 UTC
Slack digest for #general - 2019-08-19
2019-08-18 10:31:40 UTC - Sijie Guo: FYI. <https://github.com/apache/pulsar/pull/4975> the fix for localrun connecting with proxy
----
2019-08-18 12:47:43 UTC - Martin Svensson: Hi, I'm struggling to get good read-performance when reading from topics with offloaded ledgers. At first, I thought it had something to do with the offloading driver or the storage itself, but after switching from S3 to GCS and seeing the same performance issues, I wonder if it's something else.
Has anyone else had problems with read-performance on offloaded topics? Any ideas on how I can improve it?
I'm running pretty vanilla pulsar on gke.
----
2019-08-18 13:09:40 UTC - nxliu: @nxliu has joined the channel
----
2019-08-18 13:35:53 UTC - Chris Bartholomew: @Ming Fang does this PIP line up with your thinking for your Raspberry Pi scenario?<https://github.com/apache/pulsar/wiki/PIP-21%3A-Pulsar-Edge-Component>
----
2019-08-18 13:55:18 UTC - Ming Fang: @Chris Bartholomew Yes the Pulsar Edge looks great. My concern is it’s too elaborate for a limited target audience and therefore will not get done. I’m planning to hack something up with the current Pulsar client and function runtimes
----
2019-08-18 14:13:32 UTC - Chris Bartholomew: Sound cool. If you want help testing it, let me know.
----
2019-08-18 14:14:39 UTC - Ming Fang: :+1:
----
2019-08-18 16:45:55 UTC - Luke Lu: What’s the Pulsar version? Do you have any numbers? Would adjusting offloading threshold help?
----
2019-08-18 20:16:39 UTC - Martin Svensson: I'm using 2.4.0.
I've tried a bunch of different scenarios but I'll give you two examples:
• producing 100 msg/s at 0.8Mbit/s for a few hours results in several offloaded ledgers
• starting a new subscription with new consumers (Shared) (from earliest) gives me messages at a rate under 200 msg/s (barely catching up)
• producing 1000 msg/s for half a minute doesn't trigger any offloading
• consuming from the get-go will initially consume msgs at a rate of ~5k msg/s for a few seconds and then assume the producing rate of 1000 msg/s
----
2019-08-19 01:15:30 UTC - William Xu: @William Xu has joined the channel
----
2019-08-19 06:02:12 UTC - Luke Lu: Does increasing s3ManagedLedgerOffloadReadBufferSizeInBytes (from the default 1MB) help?
@Sijie Guo, anybody/progress working on <https://github.com/apache/pulsar/issues/4025>?
----