[GitHub] [ozone] sodonnel commented on pull request #2963: HDDS-5267 Merge ICR and FCR for containers if part of the same HB

[GitBox <gi...@apache.org>]

sodonnel commented on pull request #2963:
URL: https://github.com/apache/ozone/pull/2963#issuecomment-1047961461


   Looking at the current code, are things still as they were when I last reviewed? Ie, we check for the presence of any ICRs when sending the full report, and if there are any ICRs, we have to generate the FCR again? I still believe this is going to cause the full report to often be generated twice, as it is quite likely there will be an ICR after the FCR gets generated, but before it gets sent. In fact, as we have many endpoints, it could be regenerated "endPoint" times I think.
   
   In OzoneContainer, we can see that when a new container is constructed, it triggers a heartbeat immediately:
   
   ```
       IncrementalReportSender<Container> icrSender = container -> {
         synchronized (containerSet) {
           ContainerReplicaProto containerReport = container.getContainerReport();
   
           IncrementalContainerReportProto icr = IncrementalContainerReportProto
               .newBuilder()
               .addReport(containerReport)
               .build();
   // Heartbeat is triggered here - sending a full report if needed or just pending ICRs.
           context.addIncrementalReport(icr);  
           context.getParent().triggerHeartbeat();
         }
       };
   ```
   This means that there is up to a 30 second window (heartbeart interval) from when a FCR gets generated until it gets sent, but the arrival of any ICR in that window will cause it to be sent immediately but also refreshed as there will be a new ICR in the queue.
   
   If we change things, so that when we are generating a FCR, we purge all pending ICRs and block new ones getting generated, we avoid this double refresh problem. The heartbeat trigger will first send the FCR, and then any newer ICRs, which must have been generated after it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org