[GitHub] [ozone] ChenSammi commented on pull request #2360: HDDS-5188. Replace GRPC based closed-container replication with Netty based streaming

[GitBox <gi...@apache.org>]

ChenSammi commented on pull request #2360:
URL: https://github.com/apache/ozone/pull/2360#issuecomment-895128615


   Hey @elek , thanks for improving the memory usage of DN replication.  
   
   I generally support the idea, which will makes DN more stable.  After gone through the code,  one question is what's the expected state of metadata and data consistency and integrity if the file under container directory is transferred one by one. 
   
   We have observed several container replication issues in our cluster, such as need more metrics, two much grpc thread, no flow control on SCM side,  downloaded .tar.gz failed because of missing container file or chunks directory,  temp files left by DN crash is not clean, etc,  and are trying to mitigate them one by one.  
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org