You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/02/24 10:29:00 UTC
[GitHub] [pulsar] codelipenghui opened a new pull request #14449: Penghui/compaction inclusive
codelipenghui opened a new pull request #14449:
URL: https://github.com/apache/pulsar/pull/14449
Fix can't read the latest message of the compacted topic
### Motivation
If the reader enabled read compacted and all the data of topic has been compacted
to the compacted ledger, the original topic does not have any data. In this case,
the reader is not able to read the latest message of the compacted topic.
```java
Reader<byte[]> reader = pulsarClient.newReader()
.topic(topic)
.startMessageId(MessageId.latest)
.startMessageIdInclusive()
.readCompacted(true)
.create();
```
The root cause is if the `startMessageIdInclusive` is true
and the `startMessageId` is `latest`, the reader will get the
last message ID from the broker and then seek to the last message.
But, the seek method did not consider if there are messages in the
compacted ledger, so not able to seek to last message of the compacted
ledger.
### Modification
Add force reset option for the managed cursor, if the seek
position < compaction horizon, we should force reset the cursor
to the given position, so that the reader is able to start reading from
the compacted ledger.
### Verification
A new test added for verifying the read the latest message from the
compacted ledger.
### Documentation
Check the box below or label this PR directly (if you have committer privilege).
Need to update docs?
- [ ] `doc-required`
(If you need help on updating docs, create a doc issue)
- [x] `no-need-doc`
(Please explain why)
- [ ] `doc`
(If this PR contains doc changes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui merged pull request #14449: Fix can't read the latest message of the compacted topic
Posted by GitBox <gi...@apache.org>.
codelipenghui merged pull request #14449:
URL: https://github.com/apache/pulsar/pull/14449
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Jason918 commented on a change in pull request #14449: Fix can't read the latest message of the compacted topic
Posted by GitBox <gi...@apache.org>.
Jason918 commented on a change in pull request #14449:
URL: https://github.com/apache/pulsar/pull/14449#discussion_r814423040
##########
File path: managed-ledger/src/main/java/org/apache/bookkeeper/mledger/ManagedCursor.java
##########
@@ -532,10 +532,14 @@ void asyncFindNewestMatching(FindPositionConstraint constraint, Predicate<Entry>
*
* @param position
* position to move the cursor to
+ * @param forceReset
+ * whether to force reset the position event if the position is no longer in the managed ledger,
Review comment:
```suggestion
* whether to force reset the position even if the position is no longer in the managed ledger,
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on pull request #14449: Fix can't read the latest message of the compacted topic
Posted by GitBox <gi...@apache.org>.
codelipenghui commented on pull request #14449:
URL: https://github.com/apache/pulsar/pull/14449#issuecomment-1054943034
@eolivelli For this case, if the broker using an old version or the client using an old version,
users not able to read the latest message from the compacted ledger, the fix will not introduce
This means if users only upgrade the broker, the broker can seek to latest message in the compacted
ledger, but the client still can't consume it. If only upgrade the client without the broker upgrading,
the client also can't consume latest message from the compacted ledger.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on pull request #14449: Fix can't read the latest message of the compacted topic
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #14449:
URL: https://github.com/apache/pulsar/pull/14449#issuecomment-1051145245
This change is both on the server and in the Java client.
What about compatibility with old servers and old clients?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org