You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Dmytro Sen <ds...@hortonworks.com> on 2016/03/28 20:19:16 UTC
Review Request 45395: Ambari API does not return HDFS RPC metrics
based on ports
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45395/
-----------------------------------------------------------
Review request for Ambari and Sid Wagle.
Bugs: AMBARI-15606
https://issues.apache.org/jira/browse/AMBARI-15606
Repository: ambari
Description
-------
ISSUE: Ambari API does not return HDFS RPC metrics based on ports (Example 8020 and 8040)
When creating Ambari Widget we don't see an option to monitor by port. Hence the monitoring of Random port or aggregate doesn't give correct information
After the patch, ambari automatically replaces port number with the corresponding suffix
client for the port specified in hdfs-site/dfs.namenode.rpc-address
datanode for the port specified in hdfs-site/dfs.namenode.servicerpc-address
healthcheck for the port specified in hdfs-site/dfs.namenode.lifeline.rpc-address
Request/response example
{
"href" : "http://10.2.2.22:1081/api/v1/clusters/c1/services/HDFS/components/NAMENODE?fields=host_components/metrics/rpc/*,host_components/metrics/runtime/*",
"ServiceComponentInfo" : {
"cluster_name" : "c1",
"component_name" : "NAMENODE",
"service_name" : "HDFS"
},
"host_components" : [
{
"href" : "http://10.2.2.22:1081/api/v1/clusters/c1/hosts/c6401.ambari.apache.org/host_components/NAMENODE",
"HostRoles" : {
"cluster_name" : "c1",
"component_name" : "NAMENODE",
"host_name" : "c6401.ambari.apache.org"
},
"metrics" : {
"rpc" : {
"client" : {
"NumOpenConnections" : 0,
"ReceivedBytes" : 2508,
"RpcProcessingTime_avg_time" : 0.0,
"RpcProcessingTime_num_ops" : 12,
"RpcQueueTime_avg_time" : 0.0,
"RpcQueueTime_num_ops" : 12,
"SentBytes" : 408,
"callQueueLen" : 0,
"rpcAuthenticationFailures" : 0,
"rpcAuthenticationSuccesses" : 0,
"rpcAuthorizationFailures" : 0,
"rpcAuthorizationSuccesses" : 12
},
"datanode" : {
"NumOpenConnections" : 2,
"ReceivedBytes" : 3269926,
"RpcProcessingTime_avg_time" : 0.0,
"RpcProcessingTime_num_ops" : 6029,
"RpcQueueTime_avg_time" : 0.0,
"RpcQueueTime_num_ops" : 6029,
"SentBytes" : 248169,
"callQueueLen" : 0,
"rpcAuthenticationFailures" : 0,
"rpcAuthenticationSuccesses" : 0,
"rpcAuthorizationFailures" : 0,
"rpcAuthorizationSuccesses" : 288
}
},
"runtime" : {
"StartTime" : 1459170322047
}
}
}
]
}
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java 4b15b80
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java 7747753
ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXHostProvider.java 15d44b2
ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXPropertyProvider.java b883d2b
ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/PropertyHelper.java 41da279
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py b0e2e7a
ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/JMXPropertyProviderTest.java 35ba605
ambari-server/src/test/java/org/apache/ambari/server/controller/utilities/PropertyHelperTest.java 2beb462
Diff: https://reviews.apache.org/r/45395/diff/
Testing
-------
Unit tests passed
Thanks,
Dmytro Sen
Re: Review Request 45395: Ambari API does not return HDFS RPC metrics
based on ports
Posted by Dmytro Sen <ds...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45395/
-----------------------------------------------------------
(Updated Март 29, 2016, 10:26 д.п.)
Review request for Ambari and Sid Wagle.
Bugs: AMBARI-15606
https://issues.apache.org/jira/browse/AMBARI-15606
Repository: ambari
Description
-------
ISSUE: Ambari API does not return HDFS RPC metrics based on ports (Example 8020 and 8040)
When creating Ambari Widget we don't see an option to monitor by port. Hence the monitoring of Random port or aggregate doesn't give correct information
After the patch, ambari automatically replaces port number with the corresponding suffix
client for the port specified in hdfs-site/dfs.namenode.rpc-address
datanode for the port specified in hdfs-site/dfs.namenode.servicerpc-address
healthcheck for the port specified in hdfs-site/dfs.namenode.lifeline.rpc-address
Request/response example
{
"href" : "http://10.2.2.22:1081/api/v1/clusters/c1/services/HDFS/components/NAMENODE?fields=host_components/metrics/rpc/*,host_components/metrics/runtime/*",
"ServiceComponentInfo" : {
"cluster_name" : "c1",
"component_name" : "NAMENODE",
"service_name" : "HDFS"
},
"host_components" : [
{
"href" : "http://10.2.2.22:1081/api/v1/clusters/c1/hosts/c6401.ambari.apache.org/host_components/NAMENODE",
"HostRoles" : {
"cluster_name" : "c1",
"component_name" : "NAMENODE",
"host_name" : "c6401.ambari.apache.org"
},
"metrics" : {
"rpc" : {
"client" : {
"NumOpenConnections" : 0,
"ReceivedBytes" : 2508,
"RpcProcessingTime_avg_time" : 0.0,
"RpcProcessingTime_num_ops" : 12,
"RpcQueueTime_avg_time" : 0.0,
"RpcQueueTime_num_ops" : 12,
"SentBytes" : 408,
"callQueueLen" : 0,
"rpcAuthenticationFailures" : 0,
"rpcAuthenticationSuccesses" : 0,
"rpcAuthorizationFailures" : 0,
"rpcAuthorizationSuccesses" : 12
},
"datanode" : {
"NumOpenConnections" : 2,
"ReceivedBytes" : 3269926,
"RpcProcessingTime_avg_time" : 0.0,
"RpcProcessingTime_num_ops" : 6029,
"RpcQueueTime_avg_time" : 0.0,
"RpcQueueTime_num_ops" : 6029,
"SentBytes" : 248169,
"callQueueLen" : 0,
"rpcAuthenticationFailures" : 0,
"rpcAuthenticationSuccesses" : 0,
"rpcAuthorizationFailures" : 0,
"rpcAuthorizationSuccesses" : 288
}
},
"runtime" : {
"StartTime" : 1459170322047
}
}
}
]
}
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java 434aa96
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java 30e5b9c
ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXHostProvider.java 15d44b2
ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXPropertyProvider.java 1e7f4af
ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/PropertyHelper.java 41da279
ambari-server/src/main/java/org/apache/ambari/server/utils/AmbariPath.java 4648cbc
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackDefinedPropertyProviderTest.java 6f5fdff
ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/JMXPropertyProviderTest.java 6dfced1
ambari-server/src/test/java/org/apache/ambari/server/controller/utilities/PropertyHelperTest.java 2beb462
Diff: https://reviews.apache.org/r/45395/diff/
Testing
-------
Unit tests passed
Thanks,
Dmytro Sen
Re: Review Request 45395: Ambari API does not return HDFS RPC metrics
based on ports
Posted by Sid Wagle <sw...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45395/#review125784
-----------------------------------------------------------
Fix it, then Ship it!
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java (line 102)
<https://reviews.apache.org/r/45395/#comment188659>
Doesn't this need to be ConcurrentHashMap?
ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXHostProvider.java (line 81)
<https://reviews.apache.org/r/45395/#comment188660>
Decription doesn't match function.
- Sid Wagle
On March 28, 2016, 6:19 p.m., Dmytro Sen wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45395/
> -----------------------------------------------------------
>
> (Updated March 28, 2016, 6:19 p.m.)
>
>
> Review request for Ambari and Sid Wagle.
>
>
> Bugs: AMBARI-15606
> https://issues.apache.org/jira/browse/AMBARI-15606
>
>
> Repository: ambari
>
>
> Description
> -------
>
> ISSUE: Ambari API does not return HDFS RPC metrics based on ports (Example 8020 and 8040)
> When creating Ambari Widget we don't see an option to monitor by port. Hence the monitoring of Random port or aggregate doesn't give correct information
>
> After the patch, ambari automatically replaces port number with the corresponding suffix
> client for the port specified in hdfs-site/dfs.namenode.rpc-address
> datanode for the port specified in hdfs-site/dfs.namenode.servicerpc-address
> healthcheck for the port specified in hdfs-site/dfs.namenode.lifeline.rpc-address
>
> Request/response example
>
> {
> "href" : "http://10.2.2.22:1081/api/v1/clusters/c1/services/HDFS/components/NAMENODE?fields=host_components/metrics/rpc/*,host_components/metrics/runtime/*",
> "ServiceComponentInfo" : {
> "cluster_name" : "c1",
> "component_name" : "NAMENODE",
> "service_name" : "HDFS"
> },
> "host_components" : [
> {
> "href" : "http://10.2.2.22:1081/api/v1/clusters/c1/hosts/c6401.ambari.apache.org/host_components/NAMENODE",
> "HostRoles" : {
> "cluster_name" : "c1",
> "component_name" : "NAMENODE",
> "host_name" : "c6401.ambari.apache.org"
> },
> "metrics" : {
> "rpc" : {
> "client" : {
> "NumOpenConnections" : 0,
> "ReceivedBytes" : 2508,
> "RpcProcessingTime_avg_time" : 0.0,
> "RpcProcessingTime_num_ops" : 12,
> "RpcQueueTime_avg_time" : 0.0,
> "RpcQueueTime_num_ops" : 12,
> "SentBytes" : 408,
> "callQueueLen" : 0,
> "rpcAuthenticationFailures" : 0,
> "rpcAuthenticationSuccesses" : 0,
> "rpcAuthorizationFailures" : 0,
> "rpcAuthorizationSuccesses" : 12
> },
> "datanode" : {
> "NumOpenConnections" : 2,
> "ReceivedBytes" : 3269926,
> "RpcProcessingTime_avg_time" : 0.0,
> "RpcProcessingTime_num_ops" : 6029,
> "RpcQueueTime_avg_time" : 0.0,
> "RpcQueueTime_num_ops" : 6029,
> "SentBytes" : 248169,
> "callQueueLen" : 0,
> "rpcAuthenticationFailures" : 0,
> "rpcAuthenticationSuccesses" : 0,
> "rpcAuthorizationFailures" : 0,
> "rpcAuthorizationSuccesses" : 288
> }
> },
> "runtime" : {
> "StartTime" : 1459170322047
> }
> }
> }
> ]
> }
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java 4b15b80
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java 7747753
> ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXHostProvider.java 15d44b2
> ambari-server/src/main/java/org/apache/ambari/server/controller/jmx/JMXPropertyProvider.java b883d2b
> ambari-server/src/main/java/org/apache/ambari/server/controller/utilities/PropertyHelper.java 41da279
> ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py b0e2e7a
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/JMXPropertyProviderTest.java 35ba605
> ambari-server/src/test/java/org/apache/ambari/server/controller/utilities/PropertyHelperTest.java 2beb462
>
> Diff: https://reviews.apache.org/r/45395/diff/
>
>
> Testing
> -------
>
> Unit tests passed
>
>
> Thanks,
>
> Dmytro Sen
>
>