You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2020/10/08 09:11:05 UTC
Slack digest for #general - 2020-10-08
2020-10-07 10:26:41 UTC - denchie: @denchie has joined the channel
----
2020-10-07 10:47:16 UTC - denchie: Hey guys! I'm getting this error log in my Golang app almost always at around 20-30 mins after starting the application.
```{"level":"warning","local_addr":{"IP":"10.0.1.191","Port":42310,"Zone":""},"msg":"Detected stale connection to broker","remote_addr":{"Scheme":"pulsar","Opaque":"","User":null,"Host":"ip-10-0-2-101.ec2.internal:6650","Path":"","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":""},"time":"2020-10-07T10:21:27Z"}```
Now I know that there's nothing wrong with the pulsar setup as I'm able to connect back as soon as I restart the application. Also, I'm not able to see anything wrong in the code also. Can someone give me some pointers on this?
I'm using <https://github.com/apache/pulsar-client-go> to create the consumer
----
2020-10-07 13:00:25 UTC - Toktok Rambo: @Shivji Kumar Jha I know it’s late, still. Tried running it with some independent code, and I’m unable to reproduce the issue. Must be some configurations I messed up in my main application
----
2020-10-07 13:01:44 UTC - Shivji Kumar Jha: Ah glad it worked!
party-parrot : Toktok Rambo
----
2020-10-07 16:03:25 UTC - Evan Furman: What does BKVM do exactly?
----
2020-10-07 16:12:18 UTC - Sijie Guo: the blog post doesn’t do a good job for performance comparison. for some of the tests they compare kafka and pravega; for some of the tests they compare pulsar and pravega; for the tiered storage test, it compares Pravega with EFS (an expensive component) with Pulsar with S3. A lot of issues I can spot from the blog post.
+1 : Vil
----
2020-10-07 16:21:12 UTC - Sijie Guo: @Evan Furman you can find some dashboard templates for pulsar-datadog integration. <https://github.com/streamnative/pulsar-datadog>
pray : Evan Furman, Shivji Kumar Jha
----
2020-10-07 16:47:06 UTC - Evan Furman: This is awesome — thank you @Sijie Guo
----
2020-10-07 17:30:45 UTC - Shivji Kumar Jha: I am trying to put a forked python client (master + <https://github.com/apache/pulsar/pull/8173|my PR> ) on an internal pypi server. The upload works fine but the download fails with
``` running build_ext
error: [Errno 2] No such file or directory: '_pulsar.so'```
<https://github.com/apache/pulsar/blob/634e9b9920737925ad418e792f87eaef4c2fe24d/pulsar-client-cpp/python/setup.py#L77|This> is where the setup.py tries to get the .so file . The question is how (when) is the .so file generated?
----
2020-10-07 17:32:07 UTC - Frank Kelly: Message Ordering Question - I've switched from `producer.send()` to `producer.sendAsync()` is there anything I need to do on the Consumer side to make sure messages are received in order (I think I know the answer but want to be sure)?
----
2020-10-07 18:02:00 UTC - Addison Higham: nope, order should be the same, the only thing with sendAsync is just to make sure you are handling the futures resolving
----
2020-10-07 18:06:27 UTC - Shivji Kumar Jha: Tests in <https://github.com/apache/pulsar/pull/8173|my PR> are failing with the <https://pipelines.actions.githubusercontent.com/EXIZfX8ixSbQEfqDA00dMJrEtloTMSwjlUs9NqJqrTcJfd7NaR/_apis/pipelines/1/runs/110364/signedlogcontent/3?urlExpires=2020-10-07T18%3A04%3A47.3723358Z&urlSigningMethod=HMACV1&urlSignature=77xHrY17CPU%2B%2BKLEGgXlBMVP%2B8F7TIWuy5Onf0OYBII%3D|following error> despite many retries. Looking for an advice / help.
```2020-10-06T05:47:03.8287415Z /home/runner/work/_temp/bc84e325-9808-450b-afae-966a2d0cd8a7.sh: line 1: ./build/retry.sh: No such file or directory
2020-10-06T05:47:03.8294667Z ##[error]Process completed with exit code 127.```
----
2020-10-07 18:30:26 UTC - Frank Kelly: Thanks Addison - is there some code pattern / idiom that shows a recommended way to resolve the futures - I assume resolving them immediately is no better than the synchronous `.send()` case?
----
2020-10-07 18:58:44 UTC - Nick Rivera: Hello! Does the auto deletion of inactive topics apply only to non-partitioned topics? We are using 2.6.0 and seeing behavior where the underlying "partition-<number>" topics are being auto deleted but not the partitioned topics that own them. This means that when we attempt to create partitioned topics through the admin api, we receive an error that the topic already exists, but all producing and consuming fails because the underlying "partition" topics do not exist
----
2020-10-07 19:00:44 UTC - Nick Rivera: For further context, we have disabled the auto-creation of topics
----
2020-10-07 19:36:49 UTC - Addison Higham: it is pretty dependent upon your application, but if you are still talking about your use case of sending files split up, then you could do `CompletableFuture.allOf` and then wait for all the futures to resolve
----
2020-10-07 19:37:25 UTC - Frank Kelly: Great - thank you
----
2020-10-07 22:35:09 UTC - Nick Rivera: checking the logs on the brokers, I definitely see instances of log lines like:
```[bookkeeper-ml-workers-OrderedExecutor-7-0] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://<redacted>-partition-0] Topic deleted successfully due to inactivity```
----
2020-10-07 22:35:46 UTC - Nick Rivera: but there are no such log lines for the high level partitioned topic name and it just remains there
----
2020-10-08 00:38:13 UTC - tuteng: Bookkeeper Visual Manager(bkvm) is an open source visual interface for <https://bookkeeper.apache.org/|Apache Bookkeeper>. <https://github.com/diennea/bookkeeper-visual-manager>, you can enable it by update configuration file <https://github.com/apache/pulsar-manager/blob/master/src/main/resources/bkvm.conf>, <https://github.com/apache/pulsar-manager#use-docker>
----
2020-10-08 00:38:56 UTC - tuteng: ```# bookie
bookie.host=<http://bookie-1>-{{ cluster_name }}.{{ route53_zone }}:8000
bookie.enable=false```
The feature of this part has not been completed yet
----
2020-10-08 01:31:47 UTC - Alan Broddle: Update. We had a proxy getting in the way and preventing the curl command. Thanks
----
2020-10-08 03:21:44 UTC - Vincent Wong: thanks
----
2020-10-08 04:03:25 UTC - Shivji Kumar Jha: Hi @Matteo Merli do you remember how the .so file used by python client's setup.py is generated?
----
2020-10-08 08:06:16 UTC - Vincent Wong: @Addison Higham so those files are storing my unack message, which is "backlog" from the subscription right ? if I clear those backlog and when next-time BK major compaction run, it should clear those huge files in my filesystem? appreciate for the help
----