You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2018/12/01 09:11:02 UTC
Slack digest for #general - 2018-12-01
2018-11-30 16:08:36 UTC - Beast in Black: @Matteo Merli @Ali Ahmed @Sijie Guo reposting a question I had asked on Wednesday: I'm seeing some broker policy errors on a namespace (possibly related to partitioned topics, but I'm not using those) when creating producers (and also consumer) to a nonpersistent topic in a namespace. I'm using pulsar-2.2 and the 2.1 CPP client.
The cpp client library shows this error when trying to create a subscriber to the topic:
```
07:06:03.430 [0x7fa4cd197700] ERROR /myapp/3rdParty/pulsar-client-cpp/lib/ClientConnection.cc(699) - [100.96.3.9:35232 -> 100.96.3.5:6650] Failed partition-metadata lookup req_id: 33 error: 1
07:06:03.430 [0x7fa4cd197700] ERROR /myapp/3rdParty/pulsar-client-cpp/lib/ClientImpl.cc(138) - Error Checking/Getting Partition Metadata while creating producer on <non-persistent://diagnostics/local.diagnostics.guestnamespace/diagnostics_nonpersist_stream> -- 5
```
Broker log (one of the brokers, I have 3) shows this *repeatedly*:
```
22:26:28.416 [ForkJoinPool.commonPool-worker-0] ERROR org.apache.pulsar.broker.web.PulsarWebResource - Policies not found for diagnostics/local.diagnostics.guestnamespace namespace
22:26:28.416 [ForkJoinPool.commonPool-worker-0] WARN org.apache.pulsar.broker.service.ServerCnx - Failed to get Partitioned Metadata [/100.96.2.13:41676] <non-persistent://diagnostics/local.diagnostics.guestnamespace/_routing>: Policies not found for diagnostics/local.diagnostics.guestnamespace namespace
org.apache.pulsar.broker.web.RestException: Policies not found for diagnostics/local.diagnostics.guestnamespace namespace
at org.apache.pulsar.broker.web.PulsarWebResource.lambda$4(PulsarWebResource.java:657) ~[org.apache.pulsar-pulsar-broker-2.2.0.jar:2.2.0]
at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:656) ~[?:1.8.0_171]
at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:632) ~[?:1.8.0_171]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_171]
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) ~[?:1.8.0_171]
at org.apache.pulsar.zookeeper.ZooKeeperDataCache.lambda$0(ZooKeeperDataCache.java:66) ~[org.apache.pulsar-pulsar-zookeeper-utils-2.2.0.jar:2.2.0]
at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:656) ~[?:1.8.0_171]
at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:632) ~[?:1.8.0_171]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_171]
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) ~[?:1.8.0_171]
at org.apache.pulsar.zookeeper.ZooKeeperCache.lambda$7(ZooKeeperCache.java:349) ~[org.apache.pulsar-pulsar-zookeeper-utils-2.2.0.jar:2.2.0]
at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:656) ~[?:1.8.0_171]
at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:632) ~[?:1.8.0_171]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[?:1.8.0_171]
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) ~[?:1.8.0_171]
at org.apache.pulsar.zookeeper.ZooKeeperCache.lambda$13(ZooKeeperCache.java:334) ~[org.apache.pulsar-pulsar-zookeeper-utils-2.2.0.jar:2.2.0]
at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402) [?:1.8.0_171]
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) [?:1.8.0_171]
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) [?:1.8.0_171]
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) [?:1.8.0_171]
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) [?:1.8.0_171]
```
Any idea how I can fix this `Policies not found` issue? I checked the pulsar documentation but perhaps I missed something related to this...
----
2018-11-30 16:09:42 UTC - Beast in Black: s/subscriber/producer/
----
2018-11-30 17:31:15 UTC - Julien Plissonneau Duquène: good morning West Coast
I was wondering if there will be Pulsar talks/events planned in Europe next year, any chance to meet some of you guys in the old world?
----
2018-11-30 17:33:10 UTC - Julien Plissonneau Duquène: btw the calendar on <http://pulsar.apache.org|pulsar.apache.org> is broken
----
2018-11-30 17:39:57 UTC - Jon Bock: Hi Julien! A number of people, including from us here at Streamlio, have submitted conference talk proposals for events in Europe, as we hear back we’ll definitely make sure the accepted ones get on the calendar (and get the calendar fixed). Let us know if there are any particular events that you’d suggest we look into submitting for, or if there’s something you’d like to submit where any of us can help.
And of course if you want to propose a Pulsar talk at a local meetup, let us know how we can support that!
----
2018-11-30 17:47:06 UTC - Byron: @Matteo Merli I finally upgraded to macOS 10.14 and ran the `brew upgrade` using the raw URL and got an error:
```
% brew upgrade <https://raw.githubusercontent.com/apache/pulsar/master/pulsar-client-cpp/homebrew/libpulsar.rb>
######################################################################## 100.0%
==> Upgrading 1 outdated package:
libpulsar 2.1.0-incubating-SNAPSHOT -> 2.2.0
==> Upgrading libpulsar --with-python3
==> Downloading <https://www.apache.org/dyn/mirrors/mirrors.cgi?action=download&filename=pulsar/pulsar-2.2>
######################################################################## 100.0%
==> cmake . -DBUILD_TESTS=OFF -DLINK_STATIC=ON -DPYTHON_INCLUDE_DIR=/usr/local/Frameworks/Python.framewor
==> make pulsarShared pulsarStatic
Last 15 lines from /Users/ruthb/Library/Logs/Homebrew/libpulsar/02.make:
"_nghttp2_submit_request", referenced from:
_http2_send in libcurl.a(libcurl_la-http2.o)
"_nghttp2_submit_rst_stream", referenced from:
_Curl_http2_done in libcurl.a(libcurl_la-http2.o)
_on_frame_recv in libcurl.a(libcurl_la-http2.o)
"_nghttp2_submit_settings", referenced from:
_Curl_http2_switched in libcurl.a(libcurl_la-http2.o)
"_nghttp2_version", referenced from:
_Curl_http2_ver in libcurl.a(libcurl_la-http2.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [lib/libpulsar.2.2.0.dylib] Error 1
make[2]: *** [lib/CMakeFiles/pulsarShared.dir/all] Error 2
make[1]: *** [lib/CMakeFiles/pulsarShared.dir/rule] Error 2
make: *** [pulsarShared] Error 2
Do not report this issue to Homebrew/brew or Homebrew/core!
These open issues may also help:
libpulsar 2.2.0 (new formula) <https://github.com/Homebrew/homebrew-core/pull/33584>
```
----
2018-11-30 17:51:24 UTC - Matteo Merli: @Byron Let me try again with newer brew. I was able to compile it without problems earlier.
Also, the homebrew CI the build goes through. I was trying to get it included in the official repo… I don’t know any maintainer to ping :slightly_smiling_face:
----
2018-11-30 17:52:41 UTC - Byron: Thanks.. yea I unfortunately don’t have any connections there. Hopefully it will be merged soon
----
2018-11-30 17:53:13 UTC - Matteo Merli: Actually, I had simplified the recipe there, based on the feedback
----
2018-11-30 17:53:36 UTC - Matteo Merli: Can you try to install <https://raw.githubusercontent.com/Homebrew/homebrew-core/31505b03c97755db2a11ad70d8ad605f7578b1a8/Formula/libpulsar.rb>
----
2018-11-30 17:55:49 UTC - Julien Plissonneau Duquène: Thanks for the support! I'm just beginning with Pulsar for now, but I will probably give a talk next year in my company, then in a local CNCF or DevOps meetup. A lot of IT people around heard about Pulsar and are interested to know more about it.
+1 : Guangzhong Yao
----
2018-11-30 17:59:03 UTC - Byron: Running..
----
2018-11-30 18:01:37 UTC - Byron: Same error
----
2018-11-30 18:02:23 UTC - Matteo Merli: Gotcha. Do you have any other `libcurl` installed?
----
2018-11-30 18:02:46 UTC - Matteo Merli: I think we generally pick the version that comes with macos
----
2018-11-30 18:03:44 UTC - Byron: I had a homebrew version. I purged and rerunning the pulsar install
----
2018-11-30 18:03:57 UTC - Matteo Merli: Also, can you use `--verbose` ?
----
2018-11-30 18:04:05 UTC - Matteo Merli: That should print the commands it’s using
----
2018-11-30 18:04:19 UTC - Byron: Yes
----
2018-11-30 18:10:20 UTC - Byron: The install worked
----
2018-11-30 18:10:41 UTC - Byron: So `brew install curl` may break the install
----
2018-11-30 18:11:49 UTC - Byron: Probably should add this to the docs or an FAQ (or figure out why it breaks). Thanks for the help
----
2018-11-30 18:12:01 UTC - Matteo Merli: :+1:
----
2018-11-30 20:12:39 UTC - Tobias: @Tobias has joined the channel
----
2018-11-30 22:38:24 UTC - Matteo Merli: @Christophe Bornet (Sorry I forgot about this thread).
> I don’t see how the consumer priorityLevel can bring the feature. To my understanding, at one point in time one broker is the partition owner and all reads go to that broker. Reads from consumers of the other DC will result in cross-dc bandwidth consumption. What would be needed is that these consumer read from a broker on their dc (so not the partition owner but another read-only one). (e
In any case, with replication the messages have to be sent to the other DC1. If you’re consuming 1 copy of the data, the network bandwidth will be the same.
> What would be nice would be to have region aware election of partition owners. So that we could configure for a given topic or partition on which DC the owner broker shall be chosen preferably.
That’s not easily achievable in an async-replication mode. You want to isolate the clusters (as much as possible) to prevent outages to spread to different regions.
----
2018-11-30 23:13:14 UTC - Christophe Bornet: My question is for sync replication (only one cluster spanning over multiple regions). It's in this setup that we have to consider cross-dc consumption
----
2018-11-30 23:17:10 UTC - Christophe Bornet: I can setup Bookies so that they are placed region aware. Then from what I understand, the brokers will do speculative reads on the closest bookie. But currently it's not possible to set region affinity to a partition.
----
2018-11-30 23:28:26 UTC - Matteo Merli: That’s correct. In that case it’s a single logical cluster, where the data is configured to be stored in the different regions. Under normal conditions though, the brokers don’t read the messages from BK, but rather dispatch directly from cache
----
2018-12-01 07:48:06 UTC - Yifan: One quick question, is it true that using MessageId.earliest will seek to the earliest message including acknowledged messages?
----
2018-12-01 08:19:33 UTC - Sijie Guo: @Yifan in what context / method?
----
2018-12-01 08:23:41 UTC - Yifan: I am using consumer, I am trying to understand earliest, latest and the default cursor
----
2018-12-01 08:26:52 UTC - Sijie Guo: yes. when you call consumer#seek() it is basically reset the subscription to the earliest message (including acked messages)
----
2018-12-01 08:29:21 UTC - Pavel Vetokhin: Hi! I am trying to deploy Pulsar to the local microk8s kube node.
But `pulsar-cluster-metadata-init` pod getting the error `[main-SendThread(zookeeper:2181)] WARN org.apache.zookeeper.ClientCnxn - Client session timed out, have not heard from server in 31029ms for sessionid 0x0`.
Is anyone having the same issue or is it just my kube misconfiguration?
----
2018-12-01 08:30:36 UTC - Yifan: make sense @Sijie Guo thank you
----
2018-12-01 08:32:33 UTC - Pavel Vetokhin: I use latest specs from <https://github.com/apache/pulsar/tree/master/deployment/kubernetes/generic>
And follow the guide <https://pulsar.apache.org/docs/latest/deployment/Kubernetes/#pulsar-on-a-custom-kubernetes-cluster>
----