You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "Samreay (via GitHub)" <gi...@apache.org> on 2024/01/11 01:54:20 UTC
[I] Reader set to MessageId.latest and inclusive start message does not work [pulsar-client-python]
Samreay opened a new issue, #193:
URL: https://github.com/apache/pulsar-client-python/issues/193
Hi team,
When using `start_message_id_inclusive`, we expect the message seeked to be returned. This is the case for the `pulsar.MessageId.earliest`, but does not work with `pulsar.MessageId.latest`.
# Reproduction
First, run a pulsar standalone instance:
```bash
docker run -it -p 6650:6650 -p 8080:8080 --tmpfs /pulsar/data apachepulsar/pulsar:3.1.0 bin/pulsar standalone
```
Then, run this code:
```python
import pulsar
host = "pulsar://localhost:6650"
topic = "example"
client = pulsar.Client(host)
producer = client.create_producer(topic)
producer.send(b"Hello world 1!")
producer.send(b"Hello world 2!")
earliest_reader = client.create_reader(topic, pulsar.MessageId.earliest, start_message_id_inclusive=True)
msg = earliest_reader.read_next()
print(msg.value())
# Prints Hello world 1!, as wanted
latest_reader = client.create_reader(topic, pulsar.MessageId.latest, start_message_id_inclusive=True)
msg = latest_reader.read_next(timeout_millis=5000)
print(msg.value())
# Times out
```
The behaviour of the earliest reader is as expected. But the latest reader should not time out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Reader set to MessageId.latest and inclusive start message does not work [pulsar-client-python]
Posted by "Samreay (via GitHub)" <gi...@apache.org>.
Samreay commented on issue #193:
URL: https://github.com/apache/pulsar-client-python/issues/193#issuecomment-1886817185
Yeah looks like the Java client also checks to see if its latest and inclusive, and then seeks explicitly to the last message: https://github.com/apache/pulsar/blob/176bdeacd309e8c1e49358987a1946abd30ba34a/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ConsumerImpl.java#L2362
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Reader set to MessageId.latest and inclusive start message does not work [pulsar-client-python]
Posted by "RobertIndie (via GitHub)" <gi...@apache.org>.
RobertIndie commented on issue #193:
URL: https://github.com/apache/pulsar-client-python/issues/193#issuecomment-1886887060
Thanks for all your information.
> That seems incredibly unintuitive. If I say to someone "Hey, get me the latest message" I'd expect them to give me the latest message, not the first message produced after I asked.
For the term of `latest`. It's documented that way: https://github.com/apache/pulsar/blob/82237d3684fe506bcb6426b3b23f413422e6e4fb/pulsar-client-api/src/main/java/org/apache/pulsar/client/api/MessageId.java#L83C1-L85
> I would expect the Python client to match that behaviour or at least document that the functionality isn't working yet.
That makes sense to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Reader set to MessageId.latest and inclusive start message does not work [pulsar-client-python]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat closed issue #193: Reader set to MessageId.latest and inclusive start message does not work
URL: https://github.com/apache/pulsar-client-python/issues/193
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Reader set to MessageId.latest and inclusive start message does not work [pulsar-client-python]
Posted by "RobertIndie (via GitHub)" <gi...@apache.org>.
RobertIndie commented on issue #193:
URL: https://github.com/apache/pulsar-client-python/issues/193#issuecomment-1886702845
The `latest` means that you are reading the messages that are produced after the reader is created. It shouldn't work with `start_message_id_inclusive`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Reader set to MessageId.latest and inclusive start message does not work [pulsar-client-python]
Posted by "treuherz (via GitHub)" <gi...@apache.org>.
treuherz commented on issue #193:
URL: https://github.com/apache/pulsar-client-python/issues/193#issuecomment-1886782074
@RobertIndie That doesn't seem right. The [Go client explicitly handles](https://github.com/apache/pulsar-client-go/blob/master/pulsar/consumer_partition.go#L383-L399) the case where `startMessageIDInclusive && startingMessageID == latestMessageID`. I would expect the Python client to match that behaviour or at least document that the functionality isn't working yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Reader set to MessageId.latest and inclusive start message does not work [pulsar-client-python]
Posted by "Samreay (via GitHub)" <gi...@apache.org>.
Samreay commented on issue #193:
URL: https://github.com/apache/pulsar-client-python/issues/193#issuecomment-1886706890
That seems incredibly unintuitive. If I say to someone "Hey, get me the latest message" I'd *expect* them to give me the latest message, not the first message produced after I asked.
Be that as it may, how should I go about consuming the last message in a topic with the python client then? Right now I'm seeking to a day in the past and just reading everything until the final message, which is incredibly wasteful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Reader set to MessageId.latest and inclusive start message does not work [pulsar-client-python]
Posted by "RobertIndie (via GitHub)" <gi...@apache.org>.
RobertIndie commented on issue #193:
URL: https://github.com/apache/pulsar-client-python/issues/193#issuecomment-1888823428
This issue is related to this C++ client issue: https://github.com/apache/pulsar-client-cpp/issues/385. I have pushed a PR to fix it: https://github.com/apache/pulsar-client-cpp/pull/386.
Hopefully it will be released in the next feature release of the Python client.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org