[jira] [Created] (AMBARI-20490) Optimizations for EU on large-scale clusters

["Alejandro Fernandez (JIRA)" <ji...@apache.org>]

Alejandro Fernandez created AMBARI-20490:
--------------------------------------------

             Summary: Optimizations for EU on large-scale clusters
                 Key: AMBARI-20490
                 URL: https://issues.apache.org/jira/browse/AMBARI-20490
             Project: Ambari
          Issue Type: Bug
          Components: ambari-server
    Affects Versions: 2.4.0
            Reporter: Alejandro Fernandez
            Assignee: Alejandro Fernandez
             Fix For: 3.0.0, 2.5.0


On large-scale clusters over 1000 nodes, performance during Express Upgrade can be very slow and it takes more time to fix potential errors.
We can remedy these with the following fixes:

* "hdp-select set all" is currently ran in the ClusterGroup with a single stage instead of batching. This can cause Ambari Server to be the bottleneck when processing 1000+ requests. Allow the execute-stage inside a ClusterGroup to use batching.

* If some hosts have symlinks or dirs that don't belong in /usr/hdp/ then it will prevent the hosts from running "hdp-select versions". We can find these problematic hosts with an alert.

* There is currently no upgrade path from HDP 2.2 directly to 2.5. Doing a 2-step upgrade is very time consuming.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)