You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2018/05/16 09:11:04 UTC
Slack digest for #general - 2018-05-16
2018-05-15 09:18:43 UTC - Matej Grašič: @Matej Grašič has joined the channel
----
2018-05-15 14:51:07 UTC - Vasily Yanov: Hi! when i perform load stress tests all consumer connections got closed and in pulsar logs I see the following exception:
```
2018-05-15 14:31:51,786 - ERROR [pulsar-web-61-28:PulsarWebResource@388] - [null] Failed to validate namespace bundle server-ali-t2-1526392322520/prod-pulsar-cluster-1/session_queue/0x6555554f_0x6aaaaaa4
java.lang.IllegalArgumentException: Invalid upper boundary for bundle
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:122)
at org.apache.pulsar.common.naming.NamespaceBundles.validateBundle(NamespaceBundles.java:110)
at org.apache.pulsar.broker.web.PulsarWebResource.validateNamespaceBundleRange(PulsarWebResource.java:385)
at org.apache.pulsar.broker.web.PulsarWebResource.validateNamespaceBundleOwnership(PulsarWebResource.java:411)
at org.apache.pulsar.broker.admin.Namespaces.splitNamespaceBundle(Namespaces.java:1007)
at sun.reflect.GeneratedMethodAccessor141.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
......
```
is it normal? is it possible to avoid this situation?
----
2018-05-15 15:56:41 UTC - Sijie Guo: @Vasily Yanov
are you using 1.22?
this exception is okay. it doesn’t impact anything, it happens when conflicts on splitting bundles. There was a change in 2.0 to get rid of this logging, since it is a bit annoying.
----
2018-05-15 16:10:43 UTC - Matteo Merli: @Vasily Yanov for an explanation on how the load distribution works, and how you can how you can pre-plan for it, take a look at <http://pulsar.apache.org/docs/latest/admin/LoadDistribution/>
----
2018-05-15 16:36:26 UTC - Vasily Yanov: @Sijie Guo @Matteo Merli thank you for explanation but why connections to pulsar got closed?
----
2018-05-15 16:38:56 UTC - Matteo Merli: the load manager realizes the load on the individual bundles (shards of the namespace) is higher than the threshold, therefore it will “split” the bundle and “unload” to have the topic from each half of the bundle to be redistributed across brokers
----
2018-05-15 16:40:08 UTC - Matteo Merli: when the unload happen, the broker will initiate a graceful close of the topic, by signaling the clients that the producer/consumer that was registered is about to be transferred somewhere else
----
2018-05-15 16:40:52 UTC - Matteo Merli: The client logic is to internally retry to re-establish a producer/consumer session, by redoing service lookup and retry
----
2018-05-15 16:41:24 UTC - Matteo Merli: Typically this won’t involve physical TCP connections being closed
----
2018-05-15 17:36:55 UTC - Igor Zubchenok: Hello guys! A quick question:
What is the default value used for backlog_quota_map of a namespace? (BacklogQuotaType.destination_storage)
----
2018-05-15 17:40:27 UTC - Matteo Merli: default is 10GB. It’s defined in `broker.conf` and it’s applied if the namespace quota config is empty
----
2018-05-15 17:42:05 UTC - Igor Zubchenok: Is this value per topic in namespace or per whole namespace?
----
2018-05-15 17:42:16 UTC - Matteo Merli: per topic
----
2018-05-15 17:42:43 UTC - Matteo Merli: it’s tricky to enforce per-namespace since topics are spread across multiple brokers
+1 : Igor Zubchenok, Vasily Yanov
----
2018-05-15 17:43:20 UTC - Igor Zubchenok: It's ok. Thanks a lot!
----
2018-05-15 19:47:26 UTC - Ryan Rose: @Ryan Rose has joined the channel
----
2018-05-15 20:33:49 UTC - Mads Grønfeldt: @Mads Grønfeldt has joined the channel
----
2018-05-16 01:26:38 UTC - Byron: @Matteo Merli hopefully a quick question.. trying to build one of the Go examples but I am getting this
----
2018-05-16 01:26:44 UTC - Byron: @Byron uploaded a file: <https://apache-pulsar.slack.com/files/UACD54WB1/FAQ8LTXT6/-.pl|Untitled>
----
2018-05-16 01:27:21 UTC - Matteo Merli: yup, it’s missing the libpulsar C++ library client
----
2018-05-16 01:27:43 UTC - Matteo Merli: I’m preparing homebrew and RPM/Deb packages to make it easy to install
----
2018-05-16 01:27:45 UTC - Byron: arent the relative paths pointing to it?
----
2018-05-16 01:28:23 UTC - Byron: @Byron uploaded a file: <https://apache-pulsar.slack.com/files/UACD54WB1/FARC500KY/-.txt|Untitled>
----
2018-05-16 01:28:26 UTC - Byron: this bit
----
2018-05-16 01:32:30 UTC - Matteo Merli: Yes, I’m updating the PR as well. Can you try to install the client lib with instructions: <https://gist.github.com/merlimat/1b4925e01bc7fcfb04b9a6193026a020>
----
2018-05-16 01:32:48 UTC - Matteo Merli: assuming you’re on mac..
----
2018-05-16 01:34:32 UTC - Matteo Merli: I fixed the included paths in last commit just pushed to the PR: <https://github.com/apache/incubator-pulsar/pull/1764/commits/d1b2ea3e04a6c4d55f935c3aef1aac831aceab8d>
----
2018-05-16 01:36:25 UTC - Byron: @Byron uploaded a file: <https://apache-pulsar.slack.com/files/UACD54WB1/FAPQAF4SC/-.pl|Untitled>
----
2018-05-16 01:36:31 UTC - Byron: i am running 10.11
----
2018-05-16 01:36:35 UTC - Byron: if that is relevant
----
2018-05-16 01:37:07 UTC - Ali Ahmed: 10.11 is pretty old latest osx is 10.13
----
2018-05-16 01:37:57 UTC - Byron: yea i am aware.. work machine
----
2018-05-16 01:38:00 UTC - Matteo Merli: :slightly_smiling_face:
----
2018-05-16 01:38:12 UTC - Ali Ahmed: ```brew install boost-python```
----
2018-05-16 01:39:10 UTC - Matteo Merli: I think in my setup brew it’s a bit cheating because I need to find the exact set of dependencies to add to the `depens_on` list
----
2018-05-16 01:39:11 UTC - Byron: i already have it installed. i think the problem is the python 3.x boost installation
----
2018-05-16 01:39:21 UTC - Byron: on 10.11
----
2018-05-16 01:39:41 UTC - Byron: which is why i had the issue of installing the py3 client libraries if i recall
----
2018-05-16 01:40:11 UTC - Ali Ahmed: try
```brew install boost-python3```
----
2018-05-16 01:40:39 UTC - Byron: shows it is installed..
----
2018-05-16 01:40:51 UTC - Byron: 1.67.0
----
2018-05-16 01:41:16 UTC - Matteo Merli: I have a 10.11 vm with vagrant. I’ll check in a few hours
----
2018-05-16 01:41:26 UTC - Byron: great thanks
----
2018-05-16 01:41:50 UTC - Byron: going to bed.. but if you figure something out post it and i will try it out in the morning
+1 : Matteo Merli
----
2018-05-16 01:42:28 UTC - Matteo Merli: Yep, I’m still working on the brew formula
----
2018-05-16 02:00:04 UTC - Ali Ahmed: ```
cmake -DENABLE_PYTHON3=ON -DPYTHON_EXECUTABLE:FILEPATH=/usr/local/bin/python3 -DBoost_PYTHON3_LIBRARY_RELEASE=/usr/local/opt/boost-python3/lib/libboost_python36-mt.dylib -DOPENSSL_ROOT_DIR=/usr/local/Cellar/openssl/1.0.2o_1/
-- LINK_STATIC: OFF
-- CMAKE_BUILD_TYPE: RelWithDebInfo
-- PYTHON: 3.6.5
-- DETECTED Python 3
-- Boost version: 1.67.0
-- Found the following Boost libraries:
-- program_options
-- filesystem
-- regex
-- thread
-- system
-- python3
-- chrono
-- date_time
-- atomic
clang-tidy not found
clang-format not found
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
```
----
2018-05-16 06:20:34 UTC - Matteo Merli: @Byron Did a couple of fixes. It should work now on a bare brew install. Please update the brew formal from same gist and make sure to remove the old tgz to force brew to fetch it again: `rm ~/Library/Caches/Homebrew/libpulsar-2.1.0-incubating-SNAPSHOT.tar.gz`
----