[GitHub] [ozone] GlenGeng commented on pull request #1533: Datanode State Machine Thread should keep alive during the whole lifetime of Datanode

[GitBox <gi...@apache.org>]

GlenGeng commented on pull request #1533:
URL: https://github.com/apache/ozone/pull/1533#issuecomment-719366692


   @mukul1987 I consider the previous description for this PR/Jira is not properly, I've updated them.
   
   First of all, the key point here is not about OOM. The key point is Datanode State Machine Thread should keep alive during the whole lifetime of Datanode. Without this thread, DN is just a zombie.
   
   Let's talk about the OOM. The root cause for OOM is quite straightforward: as explained in HDDS-4404, we have a dead Recon, the report for Recon is blocked in StateContext, after running for more than 1 week, the DN finally suffers from continually OOM.
   
   <img width="1065" alt="截屏2020-10-30 下午3 44 17" src="https://user-images.githubusercontent.com/8412672/97673933-92e18780-1ac7-11eb-9f63-15fdd1ab6dec.png">
   <img width="1240" alt="截屏2020-10-30 下午3 49 07" src="https://user-images.githubusercontent.com/8412672/97673941-96750e80-1ac7-11eb-8894-e1ef4b0dce41.png">
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org