Slack digest for #general - 2019-08-01

[Apache Pulsar Slack <ap...@gmail.com>]

2019-07-31 09:15:28 UTC - Alexandre DUVAL: There is a way to list offloaded topics? Or at least offloaded topics parts or size per topic?
----
2019-07-31 09:16:53 UTC - Penghui Li: You can disable ack timeout, just use ack/negative ack. It means explicitly telling the broker that the process failed and then the broker redeliver this message, if message is in progress, no need to ack/negative ack.
----
2019-07-31 09:20:54 UTC - Yi Tang: @Guillaume Braibant oh, no apologize,  really thanks for your reply:grinning:
slightly_smiling_face : Guillaume Braibant
----
2019-07-31 09:39:32 UTC - Shubham Maheshwari: @Penghui Li But if my application crashes while it is processing the message, it'll never be able to ack/negative ack that message ever. This'll result in that message never being retried. This is the exact scenario why I cannot ditch the ackTimeout as well.
----
2019-07-31 09:42:41 UTC - Penghui Li: Broker already handle consumer close event, if consumer crashes, broker will redeliver messages which already distribute to the crashed consumer to active consumers
----
2019-07-31 09:44:42 UTC - Shubham Maheshwari: @Penghui Li ohkay .. is there any link to the documentation where this is mentioned .. I might've missed this.
----
2019-07-31 09:48:21 UTC - Penghui Li: It seems that there is no document describing this mechanism at present :joy:
----
2019-07-31 09:49:41 UTC - Shubham Maheshwari: he he he ...
in that case, let me try to create this scenario and verify. Thanks for the help
----
2019-07-31 09:49:42 UTC - Shubham Maheshwari: :slightly_smiling_face:
----
2019-07-31 09:50:11 UTC - Penghui Li: :+1:
----
2019-07-31 10:02:50 UTC - Sijie Guo: if you already have an avro serialized bytes, you can use AUTO_PRODUCE schema.

- first register your avro schema to pulsar using `pulsar-admin schema`.
- use AUTO_PRODUCE to produce the serialized avro bytes.
----
2019-07-31 10:09:41 UTC - Yi Tang: @Sijie Guo great, that is what i’m looking for, thanks:+1:
----
2019-07-31 10:16:07 UTC - Guillaume Braibant: Hello !

I have a question about message routing with the Producer API :

In my use case, an application receives requests from multiple instances of a front-end web service (one input topic per front-end web service instance). When the application recevies a request, it processes the request and must send back the response to the output topic of the front-end webservice instance that issued the request (one output topic per front-end webservice instance). The application receives the name of the output topic inside the input message properties.

I don't see a way to change the destination topic of a Producer at runtime. Is there one ? Or should I create a new producer each time the application must send a message to a new output topic (and making a cache for producers) ?
----
2019-07-31 10:20:50 UTC - Richard Sherman: Yes you can't change the destination of a producer, so yes a cache of producers would do the job. Personally for something like this I'd use apache camel which since 2.24 has support for Pulsar.
----
2019-07-31 11:14:51 UTC - Venugopal Doddi: thanks @Sijie Guo will check that
----
2019-07-31 12:23:19 UTC - Guillaume Braibant: Thank you for your answer. I will check this out :slightly_smiling_face:
----
2019-07-31 14:12:58 UTC - Matteo Merli: That would be just for the DLog storage which is mostly used for Pulsar Functions metadata and archive storage.

For Pulsar messages data, it would be:

```
# Number of bookies to use when creating a ledger
managedLedgerDefaultEnsembleSize=2

# Number of copies to store for each message
managedLedgerDefaultWriteQuorum=2

# Number of guaranteed copies (acks to wait before write is complete)
managedLedgerDefaultAckQuorum=2
```

Also you can override these values on a per-namespace basis with `pulsar-admin namespaces set-persistence ...`
----
2019-07-31 14:14:36 UTC - Richard Sherman: Yep sorry about that copied wrong part of the wrong file. The above is in broker configuration broker.conf
----
2019-07-31 14:52:30 UTC - Howard Zhang: @jia zhai Thank you so much! It works!
----
2019-07-31 17:42:00 UTC - Grant Wu: I feel like I’ve asked this before, but I’ve forgotten and I don’t think this is very well documented anywhere - what’s the story for backwards compatibility between brokers and clients?
----
2019-07-31 17:42:12 UTC - Grant Wu: I’m running into an issue between the 2.4.0 `pulsar-admin` tool and 2.2.1 brokers.
----
2019-08-01 08:32:42 UTC - xue: Hi：
    I looked at the pulsar documentation. As for dashboad, I saw that it only works in the docker environment, but I want to run it directly in the Linux environment. What should I do?
----
2019-08-01 08:39:14 UTC - Richard Sherman: It runs fine on any machine that has Python installed and can be deployed in a standard way for a django app. You'll also need a database to store the stats, it supports postgresql, mySql and sqlLite. It's just that the docker image is the easiest way to deploy it.
----