You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by "Robert Nettleton (JIRA)" <ji...@apache.org> on 2015/06/10 21:46:00 UTC
[jira] [Updated] (AMBARI-11850) Blueprint export process is very
slow in multi-node clusters
[ https://issues.apache.org/jira/browse/AMBARI-11850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Nettleton updated AMBARI-11850:
--------------------------------------
Description:
The Blueprint export process (using the REST API to create a Blueprint based on a running cluster) has been found to have some performance issues in multi-node clusters.
Steps to reproduce:
1. Setup a 3-node cluster.
2. Using the Ambari UI, start a cluster that includes HDFS, Yarn, MapReduce, Zookeeper, and Ambari Metrics. Accept all the defaults in configuration and host assignment.
3. When the cluster deployment succeeds, attempt to export a Blueprint with the following REST API URL:
http://host:port/api/v1/clusters/clusterone?format=blueprint
The Blueprint export will eventually succeed, but takes quite a while to complete. With some basic testing, I've determined that for a 3-node cluster, the Blueprint export process can take between 7-16 seconds on average. I've attempted the export multiple times to make sure no initial startup issues were involved.
The workaround for this issue is to turn off the Ambari Metrics services prior to exporting the Blueprint. When Metrics is turned off, the export is much faster.
The Blueprint export process needs to be updated, such that the Metrics and Alerting services are not called by the REST framework during a Blueprint export. These services add quite a bit to the overall time it takes to export the Blueprint, but without any real benefit, since the Blueprint does not make use of these services' data.
I'm working on a fix for this, and will be submitting a patch shortly.
was:
The Blueprint export process (using the REST API to create a Blueprint based on a running cluster) has been found to have some performance issues in multi-node clusters.
Steps to reproduce:
1. Setup a 3-node cluster.
2. Using the Ambari UI, start a cluster that includes HDFS, Yarn, MapReduce, Zookeeper, and Ambari Metrics. Accept all the defaults in configuration and host assignment.
3. When the cluster deployment succeeds, attempt to export a Blueprint with the following REST API URL:
http://host:port/api/v1/clusters/clusterone?format=blueprint
The Blueprint export will eventually succeed, but takes quite a while to complete. With some basic testing, I've determined that for a 3-node cluster, the Blueprint export process can take between 7-16 seconds on average. I've attempted the export multiple times to make sure no initial startup issues were involved.
The workaround for this issue is to turn off the Ambari Metrics services prior to exporting the Blueprint. When Metrics is turned off, the export is much faster.
The Blueprint export process needs to be updated, such that the Metrics and Alerting services are not called by the REST framework during a Blueprint export. These services add quite a bit to the overall time it takes to export the Blueprint, but without any real benefit, since the Blueprint does not make use of these servcies' data.
I'm working on a fix for this, and will be submitting a patch shortly.
> Blueprint export process is very slow in multi-node clusters
> ------------------------------------------------------------
>
> Key: AMBARI-11850
> URL: https://issues.apache.org/jira/browse/AMBARI-11850
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Affects Versions: 2.0.0
> Reporter: Robert Nettleton
> Assignee: Robert Nettleton
> Priority: Critical
> Fix For: 2.1.0
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> The Blueprint export process (using the REST API to create a Blueprint based on a running cluster) has been found to have some performance issues in multi-node clusters.
> Steps to reproduce:
> 1. Setup a 3-node cluster.
> 2. Using the Ambari UI, start a cluster that includes HDFS, Yarn, MapReduce, Zookeeper, and Ambari Metrics. Accept all the defaults in configuration and host assignment.
> 3. When the cluster deployment succeeds, attempt to export a Blueprint with the following REST API URL:
> http://host:port/api/v1/clusters/clusterone?format=blueprint
> The Blueprint export will eventually succeed, but takes quite a while to complete. With some basic testing, I've determined that for a 3-node cluster, the Blueprint export process can take between 7-16 seconds on average. I've attempted the export multiple times to make sure no initial startup issues were involved.
> The workaround for this issue is to turn off the Ambari Metrics services prior to exporting the Blueprint. When Metrics is turned off, the export is much faster.
> The Blueprint export process needs to be updated, such that the Metrics and Alerting services are not called by the REST framework during a Blueprint export. These services add quite a bit to the overall time it takes to export the Blueprint, but without any real benefit, since the Blueprint does not make use of these services' data.
> I'm working on a fix for this, and will be submitting a patch shortly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)