You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by "Oleksandr Diachenko (JIRA)" <ji...@apache.org> on 2013/10/16 18:47:42 UTC
[jira] [Updated] (AMBARI-3085) API latency time increases linearly
to number of hosts managed
[ https://issues.apache.org/jira/browse/AMBARI-3085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Oleksandr Diachenko updated AMBARI-3085:
----------------------------------------
Attachment: AMBARI-3085.patch
During profiling server processing mentioned requests, I've found two methods with huge number of invocations. - PropertyHelper.getPropertyCategory and PropertyHelper.containsArguments. These methods performs string operations and returns results depending on propertyId. Therefore I've added caching in them, which will cost us some additional memory(two HashMaps up to 10000 elements), but we will avoid same operations on string.
> API latency time increases linearly to number of hosts managed
> --------------------------------------------------------------
>
> Key: AMBARI-3085
> URL: https://issues.apache.org/jira/browse/AMBARI-3085
> Project: Ambari
> Issue Type: Task
> Components: controller
> Affects Versions: 1.4.0
> Reporter: Srimanth Gunturi
> Labels: performance
> Attachments: AMBARI-3085.patch, Screen Shot 2013-09-03 at 4.56.47 PM.png
>
>
> Doing performance measurements on 10, 50 and 100 nodes, it was noticed that a variety of API invocations have latency time increase linearly to number of hosts managed.
> The specific API calls made were:
> /requests:
> {noformat}
> /api/v1/clusters/${cluster}/requests?to=end&page_size=10&fields=*,tasks/Tasks/*
> {noformat}
> /services:
> {noformat}
> /api/v1/clusters/${cluster}/services?fields=components/ServiceComponentInfo,components/host_components,components/host_components/HostRoles,components/host_components/metrics/jvm/memHeapUsedM,components/host_components/metrics/jvm/memHeapCommittedM,components/host_components/metrics/mapred/jobtracker/trackers_decommissioned,components/host_components/metrics/cpu/cpu_wio,components/host_components/metrics/rpc/RpcQueueTime_avg_time,components/host_components/metrics/flume/flume,components/host_components/metrics/yarn/Queue
> {noformat}
> /hosts:
> {noformat}
> /api/v1/clusters/${cluster}/hosts?fields=Hosts/host_name,Hosts/public_host_name,Hosts/disk_info,Hosts/cpu_count,Hosts/total_mem,Hosts/host_status,Hosts/last_heartbeat_time,Hosts/os_arch,Hosts/os_type,Hosts/ip,host_components,metrics/disk,metrics/load/load_one
> {noformat}
> Times of the latency involved in the calls in milliseconds is below:
> {noformat}
> /requests 10 hosts 50 hosts 100 hosts
> Total Time 117.93 482.1904762 903.5070423
> Latency Time 45.63 135.952381 239.6478873
> Transfer Time 72.3 346.2380952 663.8591549
> {noformat}
> {noformat}
> /services 10 hosts 50 hosts 100 hosts
> Total Time 627.1717172 1568.710145 2757.891892
> Latency Time 614.4848485 1514.202899 2670.594595
> Transfer Time 12.68686869 54.50724638 87.2972973
> {noformat}
> {noformat}
> /hosts 10 hosts 50 hosts 100 hosts
> Total Time 125.3505155 394.9402985 711.943662
> Latency Time 123.2886598 379.9701493 683.4366197
> Transfer Time 2.06185567 14.97014925 28.47540984
> {noformat}
> Graphs are attached.
--
This message was sent by Atlassian JIRA
(v6.1#6144)