You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by "Caideyipi (via GitHub)" <gi...@apache.org> on 2026/10/12 07:46:03 UTC
[PR] Fix subscription initialization retries for transient WAL reads [iotdb]
Caideyipi opened a new pull request, #18829:
URL: https://github.com/apache/iotdb/pull/18829
## Description
A transient `NoSuchFileException` during the initial region-progress WAL lookup was treated as a locate failure and thrown from `poll`, producing `SUBSCRIPTION_POLL_ERROR` (1904). This can interrupt consumers while a WAL filename is changing during rolling.
Carry an explicit `RETRY_LATER` decision for deferred reads and leave prefetch initialization dormant until a subsequent poll completes the lookup. Recovery frontiers and the replay cursor are installed only after a successful lookup. Confirmed retained WAL corruption remains a lookup failure, including scans that also encounter a temporarily unreadable live file.
The regression tests reproduce a file disappearing between directory listing and opening, repeated unreadable polls followed by recovery, retained corruption, and corruption followed by a deferred live-file read. Recovery checks verify that committed records are skipped and the remaining records replay exactly once without WAL gap increments.
## Validation
- The new tests reproduce two initialization exceptions before the fix; all four pass with the fix.
- Related queue, seek, initialization, and WAL iterator tests: 75 tests, 0 failures, 0 errors, 1 optional performance test skipped.
- English and Chinese full-reactor test compilation passed: `mvn test-compile -DskipTests` and `mvn test-compile -DskipTests -P with-zh-locale`.
- The original 3C3D workload has not been rerun.
This PR has:
- [x] been self-reviewed.
- [x] added comments explaining the retry and corruption distinction.
- [x] added unit tests covering the new behavior.
##### Key changed/added classes
`ConsensusPrefetchingQueue`, `ConsensusPrefetchingQueueInitializationTest`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix subscription initialization retries for transient WAL reads [iotdb]
Posted by "jt2594838 (via GitHub)" <gi...@apache.org>.
jt2594838 commented on code in PR #18829:
URL: https://github.com/apache/iotdb/pull/18829#discussion_r4266342700
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/subscription/broker/consensus/ConsensusPrefetchingQueue.java:
##########
@@ -1245,9 +1259,13 @@ protected ReplayLocateDecision locateReplayStartForRegionProgress(
final ReplayLocateDecision decision =
scanReplayStartForRequests(iterator, regionProgress, seekAfter);
if (iterator.hasIncompleteScan()) {
- return ReplayLocateDecision.locateMiss(
- regionProgress,
- "replay lookup did not complete: " + iterator.getIncompleteScanDetail());
+ final String detail =
+ "replay lookup did not complete: " + iterator.getIncompleteScanDetail();
+ // Retry only a deferred read. Confirmed corruption encountered earlier in the scan must
+ // still surface as a lookup failure, even if a later file is temporarily unreadable.
+ return iterator.isWaitingForReadableWal() && !iterator.hasSkippedBrokenWalFiles()
+ ? ReplayLocateDecision.retryLater(regionProgress, detail)
+ : ReplayLocateDecision.locateMiss(regionProgress, detail);
Review Comment:
i18n
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix subscription initialization retries for transient WAL reads [iotdb]
Posted by "jt2594838 (via GitHub)" <gi...@apache.org>.
jt2594838 merged PR #18829:
URL: https://github.com/apache/iotdb/pull/18829
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org