[PR] PHOENIX-7938 Round-align the sync-replication replay consistency point [phoenix]

["Himanshu-g81 (via GitHub)" <gi...@apache.org>]

Himanshu-g81 opened a new pull request, #2570:
URL: https://github.com/apache/phoenix/pull/2570

   JIRA: https://issues.apache.org/jira/browse/PHOENIX-7938 (sub-task of PHOENIX-7562)
     
     **Problem:** In `SYNC` state, `ReplicationLogDiscoveryReplay.getConsistencyPoint()` used the raw minimum timestamp of files in the IN-PROGRESS directory. Because files within a round are moved IN →
     IN-PROGRESS in random order, a later-timestamp file can be in progress while an older sibling from the same round still waits in IN. The consistency point then advances past that unreplayed file, and since
     the compaction guard treats it as an exclusive bound (`consistencyPoint - 1`), it can drop delete markers for unreplayed data — a data-resurrection risk.
     
     **Fix:** Align the minimum IN-PROGRESS timestamp down to its round start before using it as the consistency point. Every file in round N has `ts >= roundStart(N)` and earlier rounds are fully replayed first,
     so `roundStart(N)` is a safe, conservative bound that never advances past unreplayed data — and it needs no listing of the IN directories. Only the SYNC/in-progress branch changes; the guard formula and
     shared min helpers are untouched. Honors the configurable round duration. 
     
     **Testing:** New unit tests (out-of-order, boundary, multi-round-span) and a new mini-cluster IT reproducing the ticket scenario end-to-end. The pre-fix code returns the raw minimum and fails these
     assertions.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] PHOENIX-7938 Round-align the sync-replication replay consistency point [phoenix]

["tkhurana (via GitHub)" <gi...@apache.org>]

tkhurana merged PR #2570:
URL: https://github.com/apache/phoenix/pull/2570


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] PHOENIX-7938 Round-align the sync-replication replay consistency point [phoenix]

["tkhurana (via GitHub)" <gi...@apache.org>]

tkhurana commented on code in PR #2570:
URL: https://github.com/apache/phoenix/pull/2570#discussion_r3582682657


##########
phoenix-core-server/src/main/java/org/apache/phoenix/replication/reader/ReplicationLogDiscoveryReplay.java:
##########
@@ -571,8 +579,12 @@ public long getConsistencyPoint() throws IOException {
         Optional<Long> optionalMinTimestampInProgressTimestamp =
           getMinTimestampFromInProgressFiles();
         if (optionalMinTimestampInProgressTimestamp.isPresent()) {
-          // Use minimum timestamp from in-progress files as consistency point
-          consistencyPoint = optionalMinTimestampInProgressTimestamp.get();
+          // Align the minimum in-progress timestamp down to the start of the round it belongs to,
+          // so the consistency point never advances past older, still-unreplayed files of the same
+          // round that are waiting in the IN directory (files are picked in random order).
+          long minTimestampInProgress = optionalMinTimestampInProgressTimestamp.get();
+          consistencyPoint = replicationLogTracker.getReplicationShardDirectoryManager()

Review Comment:
   Simpler to use `getNearestRoundStartTimestamp` ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org