Slack digest for #dev - 2020-04-02

[Apache Pulsar Slack <ap...@gmail.com>]

2020-04-01 20:46:27 UTC - Michael Scott: @Michael Scott has joined the channel
----
2020-04-01 20:46:45 UTC - Michael Scott: Hi, I'm using the cpp client to send messages over Pulsar. The problem is that I don't know what topics I'm going to publish to beforehand so I cannot create a `Producer`. Is it possible to publish a message without creating a producer that's tied to a specific topic? If not, how expensive is to create a new producer for each message I publish?
----
2020-04-01 21:00:44 UTC - Roman Popenov: You can use the pulsar-client to produce a message
```pulsar-client produce <persistent://public/default/some-topic> -f /pulsar/message_file.json ```
and I think you can specify the broker URL in there too
----
2020-04-01 21:01:24 UTC - Roman Popenov: <http://pulsar.apache.org/docs/en/2.2.1/standalone/>
----
2020-04-01 21:01:48 UTC - Roman Popenov: `bin/pulsar-client produce my-topic --messages "hello-pulsar"` is what is specified, but I think the pulsar-client has a help displayed too
----
2020-04-01 21:12:38 UTC - Vince Pergolizzi: Could you not just maintain a pool of them? So as each new distinct topic comes in, you create one for that just once?
----
2020-04-01 21:14:37 UTC - Michael Scott: I could but then I need to make sure to close them after some time passes to not accumulate too many open producers. This is my second choice but I'm just looking for easier solution without any maintenance needed.
----
2020-04-01 21:16:12 UTC - Michael Scott: This isn't ideal because I bet that `pulsar-client` also creates a producer in the background on every call, but additionalyl to that it also creates a new process so the performance is going to be terrible.
----
2020-04-01 21:20:33 UTC - Roman Popenov: So you basically would like to take usage of the bookkeeper api and push a message through the bookie directly?
----
2020-04-01 21:20:51 UTC - Roman Popenov: I don’t quite understand the use case maybe?
----
2020-04-01 21:37:38 UTC - Sijie Guo: It is relatively cheap to create a producer. Because producer is using multiplexed connections maintained by a client instance.
----
2020-04-01 22:20:02 UTC - Michael Scott: Great, that's exactly the answer I was looking for!
----
2020-04-01 23:49:50 UTC - Michael Scott: When using `Producer.sendAsync` c++ method, is the callback called concurrently? Should I make my datastructures that are modified thread-safe?
----
2020-04-01 23:50:24 UTC - Matteo Merli: Yes, the callback is being invoked from a background thread
----
2020-04-01 23:52:05 UTC - Michael Scott: Right, but is it a single background thread or can there be multiple?
----
2020-04-01 23:59:12 UTC - Matteo Merli: It depends.

The thread is an io-thread. By default there is 1 thread only, though you can configure more than 1.

For a given partition, all the acks, will be coming back from the same thread, though that will not be necessarily true across partitions.
----
2020-04-02 00:18:20 UTC - Michael Scott: Well, I guess I'm going to lock then. Better be safe than sorry. Thanks!
----