Slack digest for #general - 2019-01-30

[Apache Pulsar Slack <ap...@gmail.com>]

2019-01-29 09:32:32 UTC - jia zhai: right, if also need the config in bk and broker, it need a re-config and reboot
----
2019-01-29 11:01:20 UTC - Yuvaraj Loganathan: What is the right order to upgrade the cluster ? Rolling update Bookkeeper nodes  and rolling update broker and then zookeeper nodes? is there any doc available ?
----
2019-01-29 13:37:25 UTC - jia zhai: Seems there is no doc for upgrade currently.
The order of bk - > broker -> zk seems good
----
2019-01-29 17:34:55 UTC - Emma Pollum: I'm consistently seeing this message in my pulsar logs
INFO  org.apache.pulsar.broker.admin.v2.NonPersistentTopics - [null] Namespace bundle is not owned by any broker prod/prod/0x1a800000_0x1b000000
What is this referring to?
----
2019-01-29 21:18:59 UTC - Grant Wu: I’m getting some weird problems with Pulsar, seems related to my use of Pulsar functions:
----
2019-01-29 21:20:01 UTC - Grant Wu: 
----
2019-01-29 21:35:27 UTC - Grant Wu: Actually, scratch that, we had it happen once without Pulsar functions being involved
----
2019-01-29 21:38:35 UTC - Ali Ahmed: @Emma Pollum do you have more details
----
2019-01-29 21:42:20 UTC - Shalin: Is there a max redelivery configuration that can be set for python consumer client?
----
2019-01-29 21:45:14 UTC - Grant Wu: @Shalin does `acknowledge_cumulative` work?
----
2019-01-29 21:46:36 UTC - Shalin: Not quite. I am thinking in terms of dead letter topic, <https://github.com/apache/pulsar/pull/2400>
----
2019-01-29 22:59:49 UTC - Dilara Çakır: @Dilara Çakır has joined the channel
----
2019-01-29 23:01:24 UTC - Patrick Johnmeyer: @Patrick Johnmeyer has joined the channel
----
2019-01-29 23:01:28 UTC - Dilara Çakır: Hi everyone I'm Dilara
wave : Matteo Merli, Ali Ahmed, Karthik Ramasamy, Joe Francis, bossbaby, Nazrvl
----
2019-01-29 23:05:34 UTC - Grant Wu: :wave:
----
2019-01-29 23:55:59 UTC - Vincent Ngan: Does a message id carry information about its ordering? Can I compare two message ids to determine their ordering? For example, if (messageId1 &lt; messageId2) means message1 comes before message2.
----
2019-01-29 23:56:48 UTC - Matteo Merli: Yes, `MessageId` interface extends `Comparable&lt;MessageId&gt;`
----
2019-01-29 23:57:45 UTC - Matteo Merli: Even though it’s “opaque”, it can be compared. The “comes before” here would mean “was written earlier in the topic”
----
2019-01-30 01:10:59 UTC - Vincent Ngan: Thanks for your explanation. I have another question about the ordering of messages regarding the use of async call. If I send two messages in parallel using async. The ordering of these two messages is determined by (A) the ordering of making two async calls; (B) the ordering of the actual completion sequence of the async calls; or (C) the ordering information of the message ids returned when the async calls complete?
----
2019-01-30 01:22:51 UTC - Matteo Merli: All three A, B and C are guaranteed to be the same
----
2019-01-30 01:32:57 UTC - Vincent Ngan: Great! Thanks a lot.
----
2019-01-30 01:41:05 UTC - Vincent Ngan: So, that means the Pulsar client serialises certain part of its logic to make sure that, even for async calls, the ordering of the messages written to the topic is same as the calling sequence of the async calls.
----
2019-01-30 01:46:16 UTC - Joe Francis: @Matteo Merli how so?
----
2019-01-30 01:49:57 UTC - Matteo Merli: * Messages are enqueued based on the sequence of `sendAsync()` calls
 * Producer maintains items in queue until an ack is received
 * If there’s a failure, producer will re-send all the pending items in the queue (in same order)
 * Broker will send acks back in the same order. There’s validation in producer that a successful ack from broker will have to match the 1st item in the queue
 * Completable futures from send operations are therefore triggered in order
 * Message Id for these published messages will be in order too
----
2019-01-30 01:50:31 UTC - Matteo Merli: &gt; So, that means the Pulsar client serialises certain part of its logic to make sure that, even for async calls, the ordering of the messages written to the topic is same as the calling sequence of the async calls.

Yes, a portion of `sendAsync()` is protected by mutex (only for the duration of the enqueuing)
----
2019-01-30 01:51:50 UTC - Joe Francis: But parallel async calls - is there a guarantee between them?  I thought before entering the queue, there is no ordering among threads.
----
2019-01-30 01:55:05 UTC - Matteo Merli: If you have 2 threads calling sendAsync(), there’s an inherent race. The 2 calls will be serialized (in some way) though that’s non-deterministic.

What is deterministic is if you have one single thread doing:

```java
producer.sendAsync("Hello-1");
producer.sendAsync("Hello-2");
producer.flush();
```
----
2019-01-30 01:55:58 UTC - Matteo Merli: In this case, it’s guaranteed that “Hello-1” will always come before “Hello-2", even if there are other threads writing on the same producer instance
----
2019-01-30 01:57:15 UTC - Joe Francis: I think @Vincent Ngan qn was about parallel calls - and I dont know whether we return comparables when the async call returns to see who entered the queue first - Do we?
----
2019-01-30 02:01:35 UTC - Matteo Merli: &gt; If I send two messages in parallel using async. The ordering of these two messages is determined by (A) the ordering of making two async calls;

More that “in parallel” it would be “pipelining”.

&gt; and I dont know whether we return comparables when the async call returns to see who entered the queue first - Do we?

`sendAsync()` just returns the future, to get the actual message id one has to wait until the future is completed
----
2019-01-30 02:02:09 UTC - Matteo Merli: so, you cannot compare the futures directly, though they will be triggered in the same order
----
2019-01-30 06:56:33 UTC - bossbaby: Once zookeeper stops or dies, how to recover data from the bookie when connecting to another zookeeper?
----
2019-01-30 08:34:44 UTC - jia zhai: @bossbaby zookeeper contains the metadata of bookkeeper cluster. It would be troublesome if the data in zookeeper lost.
----