You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2020/03/27 09:11:02 UTC
Slack digest for #general - 2020-03-27
2020-03-26 09:14:46 UTC - Amit Aggarwal: • in broker conf auto topic creation is enabled
• in broker set ttlDurationDefaultInSeconds to 604800 (in all brokers and restart them)
• now create subscription to topic - persistent/public/default/amit_test
• close the subscription
• create producer to above topic - providing a name and setting timeout to 0 (also dedup has been enabled in broker conf)
• Now send 2 messages to the topic using the producer one after another
• Now close the producer and client
• Now using admin stats monitor the topic - pulsar-broker:8080/admin/v2/persistent/public/default/amit_test/stats
• under the subscription can see that 2 messages are present in msgBacklog and lastExpireTimestamp is 0 (and no active producers or consumers)
• After couple of minutes can observe msgBacklog reduced to 1 and lastExpireTimestamp is set to recent timestamp
• After 5 more min can observe msgBacklog reduced to 0 and lastExpireTimestamp is updated to recent timestamp
It could be a bug how ttl checker is expiring messages (instead of expiring offsett it's meant to, it is expiring that offsett + 1), leading to loss of a single message every time it is run
Will create a bug with above details (performing more tests)
----
2020-03-26 10:29:17 UTC - Arthur: I already put theses 3 configurations but I get the bookie error:
`managedLedgerDefaultEnsembleSize: "1"`
`managedLedgerDefaultWriteQuorum: "1"`
`managedLedgerDefaultAckQuorum: "1"`
----
2020-03-26 11:03:00 UTC - Amit Aggarwal: Filled <https://github.com/apache/pulsar/issues/6617>
ok_hand : Sijie Guo
----
2020-03-26 14:23:59 UTC - Casper Biering: @Casper Biering has joined the channel
----
2020-03-26 14:53:50 UTC - macneib: @macneib has joined the channel
----
2020-03-26 16:19:57 UTC - Sijie Guo: thank you for filing the issue
----
2020-03-26 16:21:18 UTC - Sijie Guo: Did you check if the bookies are alive by running “bin/bookkeeper shell listbookies -rw”？
----
2020-03-26 16:30:52 UTC - Arthur: Bookeeper pod is running. From where should I run this command ? (from broker pod ?)
----
2020-03-26 17:31:01 UTC - Guillaume: @Guillaume has joined the channel
----
2020-03-26 17:37:12 UTC - Guillaume: Pulsar throw Exceptions about Timeout in a unpredictable way. I use pulsar in docker as standalone.
When I try to call admin API using the Java client, I get timeout in Pulsar here some logs:
`17:24:15.622 [pulsar-web-57-1] INFO org.eclipse.jetty.server.RequestLog - 91.134.208.124 - - [26/Mar/2020:17:24:15 +0000] "GET /admin/v2/broker-stats/topics HTTP/1.1" 200 153611 "-" "Pulsar-Java-v2.4.1" 6`
17:24:17.858 [ForkJoinPool.commonPool-worker-11] ERROR org.apache.pulsar.broker.service.persistent.PersistentTopic - [`<persistent://xiotnx/agent_ns/data_up-d0314279-81c5-41c1-92c6-a2b70a75e6f3>] Policies update failed org.apache.pulsar.broker.service.BrokerServiceException$ServerMetadataException: java.util.concurrent.TimeoutException, scheduled retry in 60 seconds`
java.util.concurrent.CompletionException: org.apache.pulsar.broker.service.BrokerServiceException$ServerMetadataException: java.util.concurrent.TimeoutException
Can you help ? Thanks.
----
2020-03-26 17:37:57 UTC - Sijie Guo: you can go to the bookie pood and run the command.
----
2020-03-26 17:43:03 UTC - CLS: Hi all, quick question - once a ledger has been rolled off into tiered storage, documentation says that the bookkeeper will keep messages for four hours by default. After that time, how do I access old ledgers? Are those messages still available for clients or do I need to do something to recover ledgers that have been offloaded into long term storage?
----
2020-03-26 17:43:44 UTC - Matteo Merli: From client perspective, there would be no difference
----
2020-03-26 17:44:08 UTC - CLS: awesome - thanks!
----
2020-03-26 17:44:19 UTC - Matteo Merli: Brokers will read the data from the right place, transparently
----
2020-03-26 17:44:34 UTC - CLS: very nice.
----
2020-03-26 18:21:50 UTC - Ebere Abanonu: Hi, please I cannot create producer nor consumer using proxy if I deploy on kubernetes. The lookup returns broker cluster ip which cannot be accessed outside the cluster.
----
2020-03-26 18:23:01 UTC - Matteo Merli: What client lib are you using? If client goes to proxy, it will always connect through the same servieUrl
----
2020-03-26 18:24:29 UTC - Ebere Abanonu: I get connected but when I create producer, I get no response
----
2020-03-26 18:24:43 UTC - Ebere Abanonu: Using my own client port
----
2020-03-26 18:25:15 UTC - Ebere Abanonu: On a bare metal it works ok
----
2020-03-26 18:54:03 UTC - Richard Jefts: @Richard Jefts has joined the channel
----
2020-03-26 21:43:10 UTC - Matteo Merli: I mean, which Pulsar client library
----
2020-03-26 22:19:49 UTC - Jonathan Suever: @Jonathan Suever has joined the channel
----
2020-03-27 04:13:31 UTC - Hiroyuki Yamada: Thank you, @Penghui Li and @Sijie Guo, for the detail investigation.
BTW, I found another issue. It might not be an issue again, but I found it a bit weird.
<http://mail-archives.apache.org/mod_mbox/pulsar-users/202003.mbox/%3CCAPDOW74LN3WtdhpG_cgCCSg9MuMmkNV6giCGD5p%3DW1wWji0W7w%40mail.gmail.com%3E>
(The reason why I’m investigation Key_Shared behavior, I’m having some ordering issues in my middleware that uses Pulsar inside. so I’m trying to reproduce it with minimum code)
----