[jira] [Commented] (AMBARI-18055) service config page load takes long time on cluster with large number of service config versions

["Hadoop QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AMBARI-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15414690#comment-15414690 ] 

Hadoop QA commented on AMBARI-18055:
------------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12822897/AMBARI-18055.2.patch
  against trunk revision .

    {color:red}-1 patch{color}.  The patch command could not apply the patch.

Console output: https://builds.apache.org/job/Ambari-trunk-test-patch/8351//console

This message is automatically generated.

> service config page load takes long time on cluster with large number of service config versions
> ------------------------------------------------------------------------------------------------
>
>                 Key: AMBARI-18055
>                 URL: https://issues.apache.org/jira/browse/AMBARI-18055
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 2.4.0
>            Reporter: Jaimin D Jetly
>            Assignee: Jaimin D Jetly
>            Priority: Critical
>             Fix For: 2.4.0
>
>         Attachments: AMBARI-18055.2.patch, AMBARI-18055.patch, AMBARI-18055.trunk.patch
>
>
> It was observed that service config page load time took very long on clusters with 1000 and 4000 config versions. This was mainly because of slowness of below two APIs that are called everytime when user navigates to any service config page
> *1. Get call to get the current service config version for a service along with other dependent services:*
> {code}
> curl --user admin:admin -i  -H "X-Requested-By: ambari" -X GET "http://localhost:8080/api/v1/clusters/c1/configurations/service_config_versions?service_name.in(HIVE,ATLAS,YARN,TEZ,ZOOKEEPER,RANGER,HDFS)&is_current=true&fields=*"
> {code}
> Above API takes 7-8 seconds when any one of the service in the API has 1000 service config versions. As a fix to this issue, Patch uses another namedQuery that precisely queries for only current service config versions of the service rather than all service config versions and then filtering the active ones in the code. With the patch it takes around 100-300ms for the API to complete
> *2. Get call to get metadata for all service config version of a service:*
> {code}
> http://localhost:8080/api/v1/clusters/c1/configurations/service_config_versions?service_name=HDFS&fields=service_config_version,user,hosts,group_id,group_name,is_current,createtime,service_name,service_config_version_note,stack_id,is_cluster_compatible&minimal_response=true
> {code}
> It was analyzed that responses of all APIs are sorted by ambari-server either by the sorting parameter specified by user in the API and if not explicitly specifed by the user then default comparator is used for sorting. When default comparator is used, sorting result set takes much longer time. Sorting 1000 service config version took around 6 seconds at [code| https://github.com/apache/ambari/blob/branch-2.4/ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterControllerImpl.java#L204]. As a workaround in the patch, I have changed this API to use sorting by appending "&sortBy=createtime.desc" to the API. After this the API time came down from 6.5 seconds to 500ms for 1000 service config versions. I have created AMBARI-18059 to investigate and work further on this issue. I have also verified that any hosts or service config version API being used by ambari-web that can return large result set uses sortBy parameter in the API. 
> Also created AMBARI-18058 for ambari-web to use pagination for this API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)