You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "Denovo1998 (via GitHub)" <gi...@apache.org> on 2026/09/17 11:50:04 UTC
[PR] [fix][ml] Fix active cursor position tracking after untracking [pulsar]
Denovo1998 opened a new pull request, #26629:
URL: https://github.com/apache/pulsar/pull/26629
### Motivation
When `ActiveManagedCursorContainerImpl` flushes an `updateCursor(cursor, null)` or `add(cursor, null)` operation, it removes the cursor from position tracking but retains its old position. Subsequent queries can return stale ranks, and restoring a position can leave the cursor detached or incorrectly linked. Repeated null updates can also attempt to unlink an already-untracked node and corrupt the remaining list.
This fixes a pre-existing container correctness issue identified while reviewing #26511. Normal broker deactivation uses `removeCursor()`, so these tests do not establish a failure in that production path.
### Modifications
- Clear the stored position when incremental processing removes a node from position tracking.
- Clear the position and both list links when a rebuild excludes an untracked cursor, allowing subsequent incremental insertion.
- Only schedule removal for nodes with a tracked position, making null updates safe for already-untracked and never-tracked nodes.
- Add regression tests covering incremental and rebuild paths, repeated untracking, and restoration through both `add()` and real cursor seek callbacks.
### Verifying this change
- [x] Make sure that the change passes the CI checks.
The new tests start real ZooKeeper and BookKeeper services and create managed ledgers and cursors through `ManagedLedgerFactoryImpl`, without mocking cursors or containers. Null untracking is invoked through the container API; seek-based restoration exercises `ManagedCursorImpl` → `ManagedLedgerImpl` → the active cursor container.
All 14 new cases fail on the unpatched implementation with incorrect rank or slowest-position results. After the fix, all 73 cases across the three container test classes pass. `quickCheck` also passes locally.
```shell
./gradlew :managed-ledger:test \
--tests ActiveManagedCursorContainerUntrackingTest \
--tests ActiveManagedCursorContainerTest \
--tests ActiveManagedCursorContainerRetentionTest \
-PtestRetryCount=0 -PtestFailFast=false
./gradlew quickCheck
```
### Does this pull request potentially affect one of the following parts:
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
*If the box was checked, please highlight the changes*
- [ ] Dependencies (add or upgrade a dependency)
- [ ] The public API
- [ ] The schema
- [ ] The default values of configurations
- [ ] The threading model
- [ ] The binary protocol
- [ ] The REST endpoints
- [ ] The admin CLI options
- [ ] The metrics
- [ ] Anything that affects deployment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [fix][ml] Fix active cursor position tracking after untracking [pulsar]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on code in PR #26629:
URL: https://github.com/apache/pulsar/pull/26629#discussion_r4037784166
##########
managed-ledger/src/test/java/org/apache/bookkeeper/mledger/impl/ActiveManagedCursorContainerUntrackingTest.java:
##########
@@ -0,0 +1,189 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.bookkeeper.mledger.impl;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.UUID;
+import org.apache.bookkeeper.mledger.ManagedCursor;
+import org.apache.bookkeeper.mledger.ManagedLedgerConfig;
+import org.apache.bookkeeper.mledger.Position;
+import org.apache.bookkeeper.test.BookKeeperClusterTestCase;
+import org.apache.pulsar.common.api.proto.CommandSubscribe.InitialPosition;
+import org.testng.annotations.AfterMethod;
+import org.testng.annotations.BeforeMethod;
+import org.testng.annotations.DataProvider;
+import org.testng.annotations.Test;
+
+public class ActiveManagedCursorContainerUntrackingTest extends BookKeeperClusterTestCase {
Review Comment:
Could we make this regression matrix lightweight unit tests using the real `ActiveManagedCursorContainerImpl`, `PositionFactory` positions, and the existing [`MockManagedCursor` helper](https://github.com/apache/pulsar/blob/07586c3f3df8ecf4b59780d070282312510fc2f6/managed-ledger/src/test/java/org/apache/bookkeeper/mledger/impl/MockManagedCursor.java#L63-L69), as `ActiveManagedCursorContainerTest` already does? The null-untracking, rebuild, reinsertion, rank, and ordering checks exercise container state and do not require ZooKeeper, bookies, or persisted entries. A small separate integration test could retain the real cursor seek callback coverage if useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [fix][ml] Fix active cursor position tracking after untracking [pulsar]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari merged PR #26629:
URL: https://github.com/apache/pulsar/pull/26629
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [fix][ml] Fix active cursor position tracking after untracking [pulsar]
Posted by "Denovo1998 (via GitHub)" <gi...@apache.org>.
Denovo1998 commented on code in PR #26629:
URL: https://github.com/apache/pulsar/pull/26629#discussion_r4038111856
##########
managed-ledger/src/test/java/org/apache/bookkeeper/mledger/impl/ActiveManagedCursorContainerUntrackingTest.java:
##########
@@ -0,0 +1,189 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.bookkeeper.mledger.impl;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.UUID;
+import org.apache.bookkeeper.mledger.ManagedCursor;
+import org.apache.bookkeeper.mledger.ManagedLedgerConfig;
+import org.apache.bookkeeper.mledger.Position;
+import org.apache.bookkeeper.test.BookKeeperClusterTestCase;
+import org.apache.pulsar.common.api.proto.CommandSubscribe.InitialPosition;
+import org.testng.annotations.AfterMethod;
+import org.testng.annotations.BeforeMethod;
+import org.testng.annotations.DataProvider;
+import org.testng.annotations.Test;
+
+public class ActiveManagedCursorContainerUntrackingTest extends BookKeeperClusterTestCase {
Review Comment:
@lhotari Makes sense, already made the changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org