[GitHub] [ozone] sodonnel commented on pull request #2338: HDDS-5341. Container report processing is single threaded

[GitBox <gi...@apache.org>]

sodonnel commented on pull request #2338:
URL: https://github.com/apache/ozone/pull/2338#issuecomment-866174814


   I've got a couple of questions on this topic:
   
   1. Does it make sense to thread pool the Incremental Reports too? I know they are much smaller, but they are also much more frequent. I have not seen any evidence they are backing up or not, but its worth considering / checking if we can.
   2. I believe the DNs send a FCR every 60 - 90 seconds. Is that frequency really needed? Unknown bugs aside, is there any reason to send a FCR after startup and first registration? ON HDFS datanodes only send full block reports every 6 hours by default. If ICRs carry all the required information for SCM, perhaps we should increase the FCR interval to an hour or more?
   3. Have we been able to capture any profiles (eg flame charts) of processing a large FCR to see if there is anything to be optimised in that flow?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org