You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "zhanglistar (via GitHub)" <gi...@apache.org> on 2026/03/08 10:47:35 UTC

[PR] [Test] Fix flaky test: ReaderSeekTest.testHasMessageAvailableAfterSeekToEnd [pulsar-client-cpp]

zhanglistar opened a new pull request, #549:
URL: https://github.com/apache/pulsar-client-cpp/pull/549

   <!--
   ### Contribution Checklist
     
     - PR title format should be *[type][component] summary*. For details, see *[Guideline - Pulsar PR Naming Convention](https://docs.google.com/document/d/1d8Pw6ZbWk-_pCKdOmdvx9rnhPiyuxwq60_TrD68d7BA/edit#heading=h.trs9rsex3xom)*. 
   
     - Fill out the template below to describe the changes contributed by the pull request. That will give reviewers the context they need to do the review.
     
     - Each pull request should address only one issue, not mix up code from multiple issues.
     
     - Each commit in the pull request has a meaningful commit message
   
     - Once all items of the checklist are addressed, remove the above text and this checklist, leaving only the filled out template below.
   -->
   
   Fixes #<xyz>
   
   ### Motivation
   
   When a reader seeks to `MessageId::latest()`, the broker may close the consumer and trigger a reconnect. In `seekAsyncInternal`, when the seek succeeds but the connection is already expired or `reconnectionPending_` is true, we only set `seekStatus_ = COMPLETED` and did not clear the local receive queue or update `startMessageId_`. As a result, `incomingMessages_` still held prefetched messages from before the seek until `connectionOpened()` later called `clearReceiveQueue()`. During that window, `hasMessageAvailable()` saw `!incomingMessages_.empty()` and returned true, causing the flaky test `ReaderSeekTest.testHasMessageAvailableAfterSeekToEnd`.
   
   ### Modifications
   
   - In `ConsumerImpl::seekAsyncInternal`, when seek result is OK and `getCnx().expired() || reconnectionPending_` (reconnection path), we now perform the same local state updates as the non-reconnection path:
     - `ackGroupingTrackerPtr_->flushAndClean()`
     - `incomingMessages_.clear()`
     - If `lastSeekArg_` holds a `MessageId`, set `startMessageId_` from it
   
   So after the seek is acknowledged, stale prefetched messages are cleared immediately and `hasMessageAvailable()` no longer returns true from old queue content before reconnect completes.
   
   ### Verifying this change
   
   - [ ] Make sure that the change passes the CI checks.
   
   This change is already covered by existing tests:
   
   - `ReaderSeekTest.testHasMessageAvailableAfterSeekToEnd` (parameterized) – asserts that after seek to latest, `hasMessageAvailable()` becomes false, then after producing a new message it becomes true and the message can be read; the fix removes the flakiness when the broker triggers reconnect on seek-to-end.
   
   ### Documentation
   
   - [x] `doc-not-needed`
   
   Bug fix in C++ client implementation only; no API or behavior contract change and no user-facing docs update required.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [fix][cpp-client] Pass seek error to callback when SEEK command fails [pulsar-client-cpp]

Posted by "zhanglistar (via GitHub)" <gi...@apache.org>.
zhanglistar commented on code in PR #549:
URL: https://github.com/apache/pulsar-client-cpp/pull/549#discussion_r2999360526


##########
lib/ConsumerImpl.cc:
##########
@@ -1844,9 +1846,8 @@ void ConsumerImpl::seekAsyncInternal(long requestId, const SharedBuffer& seek, c
                 LockGuard lock{mutex_};
                 seekStatus_ = SeekStatus::NOT_STARTED;
                 lastSeekArg_ = previousLastSeekArg;
-                executor_->postWork([self, callback{std::exchange(seekCallback_, std::nullopt).value()}]() {
-                    callback(ResultOk);
-                });
+                executor_->postWork([self, callback{std::exchange(seekCallback_, std::nullopt).value()},
+                                     result]() { callback(result); });

Review Comment:
   Yes, done.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Test] Fix flaky test: ReaderSeekTest.testHasMessageAvailableAfterSeekToEnd [pulsar-client-cpp]

Posted by "zhanglistar (via GitHub)" <gi...@apache.org>.
zhanglistar commented on code in PR #549:
URL: https://github.com/apache/pulsar-client-cpp/pull/549#discussion_r2999360526


##########
lib/ConsumerImpl.cc:
##########
@@ -1844,9 +1846,8 @@ void ConsumerImpl::seekAsyncInternal(long requestId, const SharedBuffer& seek, c
                 LockGuard lock{mutex_};
                 seekStatus_ = SeekStatus::NOT_STARTED;
                 lastSeekArg_ = previousLastSeekArg;
-                executor_->postWork([self, callback{std::exchange(seekCallback_, std::nullopt).value()}]() {
-                    callback(ResultOk);
-                });
+                executor_->postWork([self, callback{std::exchange(seekCallback_, std::nullopt).value()},
+                                     result]() { callback(result); });

Review Comment:
   yes.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [fix][cpp-client] Pass seek error to callback when SEEK command fails [pulsar-client-cpp]

Posted by "BewareMyPower (via GitHub)" <gi...@apache.org>.
BewareMyPower commented on PR #549:
URL: https://github.com/apache/pulsar-client-cpp/pull/549#issuecomment-4183318838

   Could you rebase or merge the latest master branch?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Test] Fix flaky test: ReaderSeekTest.testHasMessageAvailableAfterSeekToEnd [pulsar-client-cpp]

Posted by "BewareMyPower (via GitHub)" <gi...@apache.org>.
BewareMyPower commented on code in PR #549:
URL: https://github.com/apache/pulsar-client-cpp/pull/549#discussion_r2940086368


##########
lib/ConsumerImpl.cc:
##########
@@ -1819,7 +1819,13 @@ void ConsumerImpl::seekAsyncInternal(long requestId, const SharedBuffer& seek, c
             if (result == ResultOk) {
                 LockGuard lock(mutex_);
                 if (getCnx().expired() || reconnectionPending_) {
-                    // It's during reconnection, complete the seek future after connection is established
+                    // It's during reconnection, complete the seek future after connection is established.
+                    // Clear local state now so hasMessageAvailable() does not see stale prefetched messages.
+                    ackGroupingTrackerPtr_->flushAndClean();
+                    incomingMessages_.clear();
+                    if (lastSeekArg_.has_value() && std::holds_alternative<MessageId>(lastSeekArg_.value())) {
+                        startMessageId_ = std::get<MessageId>(lastSeekArg_.value());
+                    }

Review Comment:
   It makes the same changes in `clearReceiveQueue`. As you can see the seek callback is only executed after that, so when `hasMessageAvailable` is called after `seek`, the state is guaranteed to be cleared.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Test] Fix flaky test: ReaderSeekTest.testHasMessageAvailableAfterSeekToEnd [pulsar-client-cpp]

Posted by "zhanglistar (via GitHub)" <gi...@apache.org>.
zhanglistar commented on code in PR #549:
URL: https://github.com/apache/pulsar-client-cpp/pull/549#discussion_r2943871818


##########
lib/ConsumerImpl.cc:
##########
@@ -1819,7 +1819,13 @@ void ConsumerImpl::seekAsyncInternal(long requestId, const SharedBuffer& seek, c
             if (result == ResultOk) {
                 LockGuard lock(mutex_);
                 if (getCnx().expired() || reconnectionPending_) {
-                    // It's during reconnection, complete the seek future after connection is established
+                    // It's during reconnection, complete the seek future after connection is established.
+                    // Clear local state now so hasMessageAvailable() does not see stale prefetched messages.
+                    ackGroupingTrackerPtr_->flushAndClean();
+                    incomingMessages_.clear();
+                    if (lastSeekArg_.has_value() && std::holds_alternative<MessageId>(lastSeekArg_.value())) {
+                        startMessageId_ = std::get<MessageId>(lastSeekArg_.value());
+                    }

Review Comment:
   Moved the clear in handleCreateConsumer() inside the lock (before unlock()), so the queue is cleared while holding mutex_. That way the seek callback and hasMessageAvailable() always see the cleared state and the flaky test is fixed. No lock-order change, so no new deadlock risk; the only effect is a bit longer lock hold during reconnection.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Test] Fix flaky test: ReaderSeekTest.testHasMessageAvailableAfterSeekToEnd [pulsar-client-cpp]

Posted by "BewareMyPower (via GitHub)" <gi...@apache.org>.
BewareMyPower commented on code in PR #549:
URL: https://github.com/apache/pulsar-client-cpp/pull/549#discussion_r2946309629


##########
lib/ConsumerImpl.cc:
##########
@@ -334,11 +334,11 @@ Result ConsumerImpl::handleCreateConsumer(const ClientConnectionPtr& cnx, Result
                 return ResultAlreadyClosed;
             }
 
-            mutexLock.unlock();
-            LOG_INFO(getName() << "Created consumer on broker " << cnx->cnxString());
             incomingMessages_.clear();
             possibleSendToDeadLetterTopicMessages_.clear();
             backoff_.reset();
+            mutexLock.unlock();
+            LOG_INFO(getName() << "Created consumer on broker " << cnx->cnxString());

Review Comment:
   This change still does not make sense. `incomingMessages_.clear()` is called in `handleCreateConsumer`, which happens before completing the seek callback in https://github.com/apache/pulsar-client-cpp/blob/1a39e5323a2ec07a10cb8dc8b9d4b1c1aea8d324/lib/ConsumerImpl.cc#L266-L277 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [fix][cpp-client] Pass seek error to callback when SEEK command fails [pulsar-client-cpp]

Posted by "BewareMyPower (via GitHub)" <gi...@apache.org>.
BewareMyPower merged PR #549:
URL: https://github.com/apache/pulsar-client-cpp/pull/549


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [fix][cpp-client] Pass seek error to callback when SEEK command fails [pulsar-client-cpp]

Posted by "BewareMyPower (via GitHub)" <gi...@apache.org>.
BewareMyPower commented on PR #549:
URL: https://github.com/apache/pulsar-client-cpp/pull/549#issuecomment-4167095251

   You can merge the latest main branch to get CI running


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Test] Fix flaky test: ReaderSeekTest.testHasMessageAvailableAfterSeekToEnd [pulsar-client-cpp]

Posted by "BewareMyPower (via GitHub)" <gi...@apache.org>.
BewareMyPower commented on code in PR #549:
URL: https://github.com/apache/pulsar-client-cpp/pull/549#discussion_r2993781264


##########
lib/ConsumerImpl.cc:
##########
@@ -1844,9 +1846,8 @@ void ConsumerImpl::seekAsyncInternal(long requestId, const SharedBuffer& seek, c
                 LockGuard lock{mutex_};
                 seekStatus_ = SeekStatus::NOT_STARTED;
                 lastSeekArg_ = previousLastSeekArg;
-                executor_->postWork([self, callback{std::exchange(seekCallback_, std::nullopt).value()}]() {
-                    callback(ResultOk);
-                });
+                executor_->postWork([self, callback{std::exchange(seekCallback_, std::nullopt).value()},
+                                     result]() { callback(result); });

Review Comment:
   It seems that this PR is no longer fixing `testHasMessageAvailableAfterSeekToEnd` but just to propagate the correct error from seek response. Could you update the PR title?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org