Pulsar Community Meeting Notes 2022/05/26, (8:30 AM PST)

[Michael Marshall <mm...@apache.org>]

Hi Pulsar Community,

Here are the meeting notes from last Thursday's community meeting.
Thanks to all who participated!

Disclaimer: If something is misattributed or misrepresented, please
send a correction to this list.

Source google doc:
https://docs.google.com/document/d/19dXkVXeU2q_nHmkG8zURjKnYlvD96TbKf5KjYyASsOE

Thanks,
Michael

2022/05/26, (8:30 AM PST)
-   Attendees:
-   Matteo Merli
-   Michael Marshall
-   Heesung Sohn
-   Dave Fisher
-   Andrey Yegorov
-   Chris Bartholomew
-   Hang Chen

-   Discussions

-   Matteo: the thread from Lari on the mailing list. One of the
points from Lari’s email is about full consistency, and that would
certainly be nice, but the reality is that it is pretty much
impossible to achieve that with a distributed cache because it is
based on the assumption that each broker can do its own caching.
Michael: I believe there is supposed to be an owner for each piece of
metadata, so there isn’t the same caching. Matteo: for example, create
a namespace on one broker and then call a get operation on another
broker, it might not know about the new namespace. This would only
work without caching by going back through zookeeper. Dave: I’ve seen
this behavior with OMB. What if you return right away but signal
eventual consistency? Matteo: either you wait for all brokers to
respond, or you don’t. If you do, you have an availability problem.
Dave: what if you have a guarantee of about a second? Matteo:
everything should be distributed within a second, but there are always
edge cases. Michael: I am not able to represent the full argument, so
we’ll need to defer to another meeting to discuss further with Lari.
Matteo: acknowledges that there is cost to the persistent watches, but
believes it is better in the new implementation than having many
smaller watches, though it’d be worth doing additional testing to
verify the assumption.

-   Matteo: Heesung is on the call and he will be looking at improving
the load manager. Heesung: I’m starting by documenting the current
flow, including sequence diagrams. Michael: that sounds great!

-   PIPs

-   Michael: Asks for review on PIP 167
https://github.com/apache/pulsar/issues/15597. Discussed with Matteo.
There was confusion about how the current subscription level
authorization actually works because it is not additive, but rather
negative. Matteo proposed changing the current implementation to make
subscription permission additive and add a configuration to allow for
users that rely on the current implementation to have backwards
compatibility. Michael: that would certainly work, too. The main goal
is to improve the subscription level auth, and it’d certainly be
easier to understand in an additive model. Michael: also note that the
current feature is essentially undocumented. Matteo: that likely means
that only a handful of users are actually using the feature, which
makes it even more reasonable to take the time to change it to the
“right” implementation. Matteo to review the PIP, discuss the
historical implementation with Rajan, and then respond on the mailing
list.