You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@phoenix.apache.org by "Himanshu-g81 (via GitHub)" <gi...@apache.org> on 2026/05/01 06:49:40 UTC

Re: [PR] PHOENIX-7792 New file age check before processing during replay [phoenix]

Himanshu-g81 commented on code in PR #2423:
URL: https://github.com/apache/phoenix/pull/2423#discussion_r3172379006


##########
phoenix-core-server/src/main/java/org/apache/phoenix/replication/ReplicationLogDiscovery.java:
##########
@@ -283,19 +295,54 @@ protected boolean shouldProcessInProgressDirectory() {
 
   /**
    * Processes all new files for a specific replication round. Continuously processes files until no
-   * new files remain for the round.
+   * new files remain for the round. Files that are younger than the configured minimum age
+   * (roundTime + buffer percentage) are skipped to avoid processing files still being written by
+   * the source cluster. When all remaining files are too young, sleeps for the minimum time needed
+   * until the oldest young file becomes eligible.
    * @param replicationRound - The replication round for which to process new files
    * @throws IOException if there's an error during file processing
    */
   protected void processNewFilesForRound(ReplicationRound replicationRound) throws IOException {
     LOG.info("Starting new files processing for round: {} for haGroup: {}", replicationRound,
       haGroupName);
     long startTime = EnvironmentEdgeManager.currentTime();
-    List<Path> files = replicationLogTracker.getNewFilesForRound(replicationRound);
-    LOG.info("Number of new files for round {} is {}", replicationRound, files.size());
-    while (!files.isEmpty()) {
-      processOneRandomFile(files);
-      files = replicationLogTracker.getNewFilesForRound(replicationRound);
+    List<Path> allFiles = replicationLogTracker.getNewFilesForRound(replicationRound);
+    LOG.info("Number of new files for round {} is {}", replicationRound, allFiles.size());
+    long minAgeMs = getNewFileMinAgeMs();
+    while (!allFiles.isEmpty()) {

Review Comment:
   Sure, added in this PR itself - https://github.com/apache/phoenix/pull/2432/changes.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org