[GitHub] [hadoop-ozone] timmylicheng edited a comment on issue #393: HDDS-1590 Aggregate bytes write metrics from container to datanode and to SCM

[GitBox <gi...@apache.org>]

timmylicheng edited a comment on issue #393: HDDS-1590 Aggregate bytes write metrics from container to datanode and to SCM
URL: https://github.com/apache/hadoop-ozone/pull/393#issuecomment-588299820
 
 
   > > Sure. This patch is to summarize the bytes in for every pipeline, which requires the sum of each container that is allocated to the pipeline. The bytes in stats for each container shall be collected on datanode to be near real-time. So that I collect them up by datanode reports and aggregate by pipeline on SCM.
   > 
   > Thanks to explain it.
   > 
   > I understand that it's good to calculate the number. I don't understand why is it added to the `StorageContainerDatanodeProtocol.proto` and sent back to the SCM. It seems to be a metric which can be fetched directly from the datanode. We don't need to send it to the SCM.
   > 
   > Unless the SCM needs this data for any **decision** (to create new pipelines, replica containers, etc..). I might missed it but as I see this is not the case (but please let me now if you have such a plan...).
   
   Yea I think you already have the picture. We plan to expose pipeline heavy load in terms of bytes written to make pipeline lifecycle more balanced. @elek 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org