[jira] [Created] (AMBARI-20712) Parallel Requests With Intersecting Hosts Don't Block Correctly

["Jonathan Hurley (JIRA)" <ji...@apache.org>]

Jonathan Hurley created AMBARI-20712:
----------------------------------------

             Summary: Parallel Requests With Intersecting Hosts Don't Block Correctly
                 Key: AMBARI-20712
                 URL: https://issues.apache.org/jira/browse/AMBARI-20712
             Project: Ambari
          Issue Type: Bug
    Affects Versions: 2.5.1
            Reporter: Jonathan Hurley
            Assignee: Jonathan Hurley
            Priority: Blocker
             Fix For: 2.5.1


AMBARI-20646 introduced a regression which can be seen when deploying a cluster via blueprint with Kerberization. The problem is caused by the change which was made to only pull in the most recent stage in progress per request.

There was logic which prevented concurrently executing request stages to be scheduled if a prior request had an un-executed stage with a conflicting host. For example:

- Request 1
-- Stage 1 (IN_PROGRESS)
--- Host 1
--- Host 2
--- Host 3
-- Stage 2 (PENDING)
--- Host 4

- Request 2
-- Stage 1
--- Host 4

In the above scenario, the scheduler was tricked into thinking that Request 2 can run even though Host-4 has a conflict. This is because it was only looking at the stage in progress.

We can't simply look at all stages in progress since this would cause the performance bug to manifest again. So, the solution is to have the database tell us which hosts are blocking from prior requests. This SQL should be very fast and will only run in the specific cases where there are multiple concurrent requests (which is typically at blueprint time).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)