Slack digest for #general - 2018-10-30

[Apache Pulsar Slack <ap...@gmail.com>]

2018-10-29 10:16:44 UTC - Ivan Kelly: @dba I haven't checked the code, but unsubscribe likely kills the subscription (cursors and all this stuff in zk), while close consumer probably just closes it at the broker
+1 : dba
----
2018-10-29 10:31:02 UTC - sam.huang: @sam.huang has joined the channel
----
2018-10-29 11:25:57 UTC - Chris Bartholomew: @Chris Bartholomew has joined the channel
----
2018-10-29 11:56:44 UTC - Abhinay: @Abhinay has joined the channel
----
2018-10-29 14:56:15 UTC - Grant Wu: @sam.huang if you have questions please ask them here
----
2018-10-29 14:56:19 UTC - Grant Wu: Do not randomly PM people
----
2018-10-29 14:56:22 UTC - Grant Wu: i.e. me
----
2018-10-29 14:57:26 UTC - Rodrigo Malacarne: Hello all, I'm using the C++ client to send messages to several different topics in Pulsar. In this case, each producer will have it's own thread when sending messages to Pulsar?
----
2018-10-29 14:58:46 UTC - Rodrigo Malacarne: Or all producers share the same thread when sending messages to different topics?
----
2018-10-29 15:09:39 UTC - Sijie Guo: @Rodrigo Malacarne it is connection pool in a client, shared across producers. One connection or a pool of connections per broker. 
----
2018-10-29 16:09:56 UTC - Nicolas Ha: I moved from a single embedded pulsar broker in the same jvm to the “generic” kubernetes.

It seems to take much longer to reply though, so I am wondering:
- where should I look to debug response times?
- which settings should I look at (currently messages are persisted)?
----
2018-10-29 16:18:24 UTC - Rodrigo Malacarne: @Sijie Guo , thank you very much. So in this case a connection pool is defined as a thread pool, with a smaller thread number than the amount of topics, right? For instance, if I have, say, 10 topics and I’m using ```producer.send(message)``` it might be possible that the ```producer``` makes use of 2-3 different threads to send data to a pulsar broker?
----
2018-10-29 17:39:54 UTC - Sijie Guo: @Rodrigo Malacarne: threading wise, pulsar is using netty. so there is a pool of threads doing IOs, a connection is assigned to be handled by one thread. for a given producer, it will grab a connection from the pool, so most likely there will be one thread for handling its IO
+1 : Rodrigo Malacarne
----
2018-10-29 18:52:23 UTC - Matteo Merli: @Rodrigo Malacarne The C++ client instance has a thread pool and a set of “IO-Threads” (default is 1 thread). Each connection gets assigned to one of these threads.

All producer will be internally handled asynchronously sharing the IO-Thread
----
2018-10-29 18:53:50 UTC - Matteo Merli: > it might be possible that the `producer` makes use of 2-3 different threads to send data to a pulsar broker?

For a single producer it would be using the same thread internally
+1 : Rodrigo Malacarne
----
2018-10-29 18:59:45 UTC - Rodrigo Malacarne: @Matteo Merli, thank you very much. In a single producer case, if I want to speed up sending (sync'ed) data to Pulsar, do you think is it worth working with multiple threads?
----
2018-10-29 19:11:00 UTC - Rodrigo Malacarne: For instance, is it any useful to create, say, one thread for each topic?
----
2018-10-29 19:44:59 UTC - Matteo Merli: Not really, since it’s all async anyway.
You might want to use multiple IO threads in the client if you have many topics spread across multiple brokers.
+1 : Rodrigo Malacarne
----
2018-10-29 21:00:41 UTC - Sijie Guo: > I moved from a single embedded pulsar broker in the same jvm to the “generic” kubernetes.

you mean running broker and bookie in one same jvm?

> It seems to take much longer to reply though,

you mean publish time increase or what?

> - where should I look to debug response times?

ideally you should setup the grafana dashboard to collect all bookkeeper and pulsar metrics. that would give you an idea about the latency.
----
2018-10-30 06:08:25 UTC - Gowrishankar Sunder: @Gowrishankar Sunder has joined the channel
----
2018-10-30 07:01:08 UTC - rogear.huan: @rogear.huan has joined the channel
----
2018-10-30 08:45:54 UTC - dba: No one worried that the server accepted a message and then lost it?
----