You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2019/12/08 09:11:02 UTC
Slack digest for #general - 2019-12-08
2019-12-07 12:12:06 UTC - Roman Popenov: @Roman Popenov has joined the channel
----
2019-12-07 12:18:44 UTC - Roman Popenov: Hello everyone! I had a question about *`MaxMessageSize`* . I am aware that is now configurable, but that I shouldn’t set it to an arbitrary large value. Did anyone experiment with the message sizes? What is the possible largest size Pulsar can handle in theory? And what about practice? How large can the message get at 50k messages/s rate before seeing significant degradation in performance?
----
2019-12-07 12:28:33 UTC - Yong Zhang: As far as I know, the max message size is restricted by the Nettry max frame size.
----
2019-12-07 12:33:54 UTC - Roman Popenov: What about IO and resource efficiency?
----
2019-12-07 12:55:42 UTC - Jermaine: @Jermaine has joined the channel
----
2019-12-07 13:00:33 UTC - Yong Zhang: Sorry I have no idea about that ...
----
2019-12-07 13:02:38 UTC - Roman Popenov: Thanks for the fast reply
----
2019-12-07 15:38:05 UTC - Igor Zubchenok: My pulsat clients get disconnected too.
----
2019-12-07 15:38:51 UTC - Igor Zubchenok: And I noted also that non-persistent responpse contains persistent topics.
----
2019-12-07 17:29:36 UTC - Robin Barendregt: @Robin Barendregt has joined the channel
----
2019-12-08 02:02:01 UTC - Ali Ahmed: <https://github.com/kafkaesque-io/pulsar-beam>
----
2019-12-08 03:05:07 UTC - Roman Popenov: I am trying to create a topic in a namespace, but I don’t seem to be able to find the right steps in the documentation. I am running the following command:
```10:00 PM rp:~/fp/apache-pulsar-2.4.1/deployment/kubernetes/generic$>pulsar-admin persistent list test-tenant/cluster-1/test-ns
10:00 PM rp:~/fp/apache-pulsar-2.4.1/deployment/kubernetes/generic$>pulsar-admin topics create <persistent://test-tenant/test-ns/demo-topic>
03:02:31.125 [AsyncHttpClient-5-1] WARN org.apache.pulsar.client.admin.internal.BaseResource - [<http://proxy:8080/admin/v2/persistent/test-tenant/test-ns/demo-topic>] Failed to perform http put request: javax.ws.rs.ClientErrorException: HTTP 412 Precondition Failed
Namespace does not have any clusters configured
Reason: Namespace does not have any clusters configured
command terminated with exit code 1```
----
2019-12-08 03:09:02 UTC - David Kjerrumgaard: Try `/pulsar/bin/pulsar-admin tenants create test-tenant` to create the tenant first, then run `/pulsar/bin/pulsar-admin namespaces create test-ns` to create the namespace. Lastly, run `pulsar/bin/pulsar-admin topics create <persistent://test-tenant/test-ns/demo-topic>` to create the topic
----
2019-12-08 03:09:49 UTC - David Kjerrumgaard: FYI, be sure the add a subscriber to the topic right way ( < 30 seconds) because Pulsar automatically cleans up topics w/o any subscribers.
----
2019-12-08 03:10:53 UTC - Roman Popenov: These are the commands that I ran already:
`pulsar-admin tenants create test-tenant --admin-roles admin --allowed-clusters cluster-1,local` and
`pulsar-admin namespaces create test-tenant/cluster-1/test-ns`
----
2019-12-08 03:12:47 UTC - David Kjerrumgaard: what version of Pulsar are you running?
----
2019-12-08 03:13:21 UTC - David Kjerrumgaard: try removing the `cluster-1` from the second command.
----
2019-12-08 03:14:02 UTC - Roman Popenov: 2.4.1
----
2019-12-08 03:15:48 UTC - David Kjerrumgaard: <https://pulsar.apache.org/docs/en/pulsar-admin/#create-2> shows the syntax, which is `tenant/namespace` .
pray : Roman Popenov
----
2019-12-08 03:18:37 UTC - David Kjerrumgaard: any cluster info is specified by the -c switch
----
2019-12-08 03:18:58 UTC - Roman Popenov: That did seem to have worked!
----
2019-12-08 03:19:48 UTC - Roman Popenov: Thank you! Now I just have to add the subscription to it, and then I will try to connect using python client
----
2019-12-08 03:22:26 UTC - David Kjerrumgaard: awesome! Glad it worked
----
2019-12-08 03:29:38 UTC - Roman Popenov: Can you guide me to the subscription command?
----
2019-12-08 03:29:56 UTC - Roman Popenov: and do I need to add permission of consumer and produce on the topic or are those added automatically?
----
2019-12-08 03:30:57 UTC - David Kjerrumgaard: `/pulsar/bin/pulsar-client consume -n 0 -s my-sub persisten://`
----
2019-12-08 03:31:40 UTC - David Kjerrumgaard: you should be able to consume from the topic unless you configured security policies on the namepsace or topic
----
2019-12-08 03:34:56 UTC - Roman Popenov: Many, many, many thanks!
----
2019-12-08 03:36:20 UTC - David Kjerrumgaard: no problem
----
2019-12-08 03:53:17 UTC - Roman Popenov: Running the python sample code:
```import pulsar
TOPIC = '<persistent://test-tenant/test-ns/demo-topic>'
PULSAR_SERVICE_URL = '<pulsar://localhost:6650>'
SUBSCRIPTION = 'test-sub'
client = pulsar.Client(PULSAR_SERVICE_URL)
producer = client.create_producer(TOPIC)
for i in range(10):
producer.send('Hello-%d' % i)
client.close()```
----
2019-12-08 03:53:30 UTC - Roman Popenov: Produces:
```2019-12-07 22:52:38.169 INFO ConnectionPool:72 | Created connection for <pulsar://localhost:6650>
2019-12-07 22:52:38.172 ERROR ClientConnection:374 | [<none> -> <pulsar://localhost:6650>] Failed to establish connection: Connection refused
2019-12-07 22:52:38.172 INFO ClientConnection:1337 | [<none> -> <pulsar://localhost:6650>] Connection closed
2019-12-07 22:52:38.172 ERROR ClientImpl:182 | Error Checking/Getting Partition Metadata while creating producer on <persistent://test-tenant/test-ns/demo-topic> -- 5
2019-12-07 22:52:38.172 INFO ClientConnection:229 | [<none> -> <pulsar://localhost:6650>] Destroyed connection
Traceback (most recent call last):
File "/Users/roman.popenov/PycharmProjects/PulsarDemo/Producer.py", line 9, in <module>
producer = client.create_producer(TOPIC)
File "/Users/roman.popenov/PycharmProjects/PulsarDemo/venv/lib/python2.7/site-packages/pulsar/__init__.py", line 476, in create_producer
p._producer = self._client.create_producer(topic, conf)
Exception: Pulsar error: ConnectError```
----
2019-12-08 04:20:24 UTC - Roman Popenov: Yeah, something was not right with my the python client
----
2019-12-08 04:23:20 UTC - Roman Popenov: `error: [Errno 2] No such file or directory: '_pulsar.so'`
----
2019-12-08 06:23:34 UTC - Roman Popenov: Did anyone have issues building the C++ libraries on a Mac?
I keep getting:
```01:21 AM rp:~/fp/pulsar/pulsar-client-cpp$(master)>cmake .
-- BUILD_TESTS: ON
-- BUILD_PYTHON_WRAPPER: ON
-- LINK_STATIC: OFF
-- USE_LOG4CXX: OFF
-- CMAKE_BUILD_TYPE: RelWithDebInfo
-- Found Boost: /usr/local/include (found version "1.71.0") found components: program_options regex system
-- PYTHON: 2.7.16
-- Found Boost: /usr/local/include (found version "1.71.0") found components: python
-- HAS_ZSTD: 1
-- HAS_SNAPPY: 0
-- Using Boost Python libs: /usr/local/opt/python@2/Frameworks/Python.framework/Versions/2.7/lib;/usr/local/lib/libboost_python-mt.dylib;/usr/local/lib/libboost_python27-mt.dylib;Boost_PYTHON37-MT_LIBRARY_RELEASE-NOTFOUND
CMake Error at python/CMakeLists.txt:70 (MESSAGE):
Could not find Boost Python library
-- Configuring incomplete, errors occurred!
See also "/Users/rp/fp/pulsar/pulsar-client-cpp/CMakeFiles/CMakeOutput.log".```
I think I somehow need to change the symlinks somewhere for Python libs, but I am not sure what `cmake` needs exactly.
----