You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Xiaobing Zhou (JIRA)" <ji...@apache.org> on 2017/01/10 01:38:58 UTC
[jira] [Comment Edited] (HDFS-11194) Maintain aggregated peer
performance metrics on NameNode
[ https://issues.apache.org/jira/browse/HDFS-11194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15813493#comment-15813493 ]
Xiaobing Zhou edited comment on HDFS-11194 at 1/10/17 1:38 AM:
---------------------------------------------------------------
Thank you [~arpitagarwal] for the patch.
I've some comments.
# RollingAverages#getStats is using non-rolling mean states, it should use rolling ones instead. See RollingAverages#snapshot for calculation of rolling averages.
# DFS_METRICS_ROLLING_AVERAGES_WINDOW_SIZE_DEFAULT should be changed to the same naming with DFS_METRICS_ROLLING_AVERAGE_WINDOW_LENGTH_KEY
# These parameters can be changed to be configurable.
SlowNodeDetector#minOutlierDetectionPeers
DataNodePeerMetrics#LOW_THRESHOLD_MS
SlowPeerTracker#MAX_NODES_TO_REPORT
# In BlockReceiver#receivePacket, after trackSendPacketToLastNodeInPipeline(duration); It may need to change a bit, e.g. if (duration > DataNodePeerMetrics#LOW_THRESHOLD_MS). or remove the warning msg at all.
was (Author: xiaobingo):
Thank you [~arpitagarwal] for the patch.
I've some comments.
# RollingAverages#getStats is using non-rolling mean states, it should use rolling ones instead. See RollingAverages#snapshot for calculation of rolling averages.
# DFS_METRICS_ROLLING_AVERAGES_WINDOW_SIZE_DEFAULT should be changed to the same naming with DFS_METRICS_ROLLING_AVERAGE_WINDOW_LENGTH_KEY
# These parameters can be changed to be configurable.
SlowNodeDetector#minOutlierDetectionPeers
DataNodePeerMetrics#LOW_THRESHOLD_MS
SlowPeerTracker#MAX_NODES_TO_REPORT
# In BlockReceiver#receivePacket, after trackSendPacketToLastNodeInPipeline(duration); It may need to change a bit, e.g. if (duration > DataNodePeerMetrics#LOW_THRESHOLD_MS). or remove the warning msg at all.
> Maintain aggregated peer performance metrics on NameNode
> --------------------------------------------------------
>
> Key: HDFS-11194
> URL: https://issues.apache.org/jira/browse/HDFS-11194
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.8.0
> Reporter: Xiaobing Zhou
> Assignee: Arpit Agarwal
> Attachments: HDFS-11194-03-04.delta, HDFS-11194.01.patch, HDFS-11194.02.patch, HDFS-11194.03.patch, HDFS-11194.04.patch
>
>
> The metrics collected in HDFS-10917 should be reported to and aggregated on NameNode as part of heart beat messages. This will make is easy to expose it through JMX to users who are interested in them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org