Topic readers and acked messages 12:44:07.456

[Jean Helou <je...@gmail.com>]

Hello everyone,

I contribute to implementing the apache james mailqueue contract on top of
pulsar.
I encountered a surprising situation and I would like to know if it's my
expectations or if I'm doing something wrong.

A bit of context :
I have a producer configured on a persistent topic with batching explicitly
disabled
I have a subscription on that same topic, the consumers of that
subscription are configured with a subscription type shared and an initial
position earliest.
I have a negative ack delay of one second and I acknowledge messages
individually.
I made sure that I only have a single shared subscription on the topic.
With this setup I expect to be able to do out of order acks.

And indeed message production and consumption works as expected and the
consumers act as a worker pool.

The issue:
I want to be able to list all the **unacked** messages that are in the
topic waiting to be consumed.
To do that, I create a reader on the topic which starts at the earliest and
iterate over the messages.
However this yields acked messages which I didn't expect. I have tried
adding an artificial delay between the ack of the consumed message and the
read to no avail

I built a junit integration test which uses test containers to start a
pulsar 2.9.1 (and we use the corresponding pulsar client) and connects the
mailqueue implementation to it.
I have uploaded the logs (which mix the log output of the pulsar container
and the logs of our implementation) at https://logpaste.com/8uahGvHm

The part we try to figure out starts at 12:44:07.426 which is a call to the
admin api to get the topic stats which shows among other things:
 msgBacklog=0, backlogSize=0, msgBacklogNoDelayed=0 and
lastAckedTimestamp=1642160642430,
lastMarkDeleteAdvancedTimestamp=1642160642442
yet at  12:44:07.456 the reader received a message

is there a way to filter out messages that have been acked when reading ?

Thanks
jean