You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Jonathan Hurley <jh...@hortonworks.com> on 2015/02/22 03:04:27 UTC
Review Request 31273: Ambari Server Deadlocks During Cluster
Provisioning
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31273/
-----------------------------------------------------------
Review request for Ambari, Mahadev Konar, Nate Cole, and Tom Beerbower.
Bugs: AMBARI-9738
https://issues.apache.org/jira/browse/AMBARI-9738
Repository: ambari
Description
-------
During provisioning of a cluster, a deadlock scenario can occur between {{ClusterImpl}} and {{ClustersImpl}}.
{noformat}
Thread-1
Clusters.mapHostsToCluster (writeLock)
Cluster.refresh (writeLock)
Thread-2
Cluster.convertToResponse (readLock)
Clusters.getHost (readLock)
Thread-1 Clusters.mapHostsToCluster (writeLock)
Thread-2 Cluster.convertToResponse (readLock)
Thread-1 Cluster.refresh (writeLock) (BLOCKED by Thread-2)
Thread-2 Clusters.getHost (readLock) (BLOCKED by Thread-1)
{noformat}
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 8589182
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java 97ac2be
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClustersDeadlockTest.java PRE-CREATION
Diff: https://reviews.apache.org/r/31273/diff/
Testing
-------
Unit tests written before code fix to verify the deadlock could be reproduced. Deployed a clean cluster.
Tests run: 2724, Failures: 0, Errors: 0, Skipped: 15
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13:55 min
[INFO] Finished at: 2015-02-21T17:45:45-05:00
[INFO] Final Memory: 32M/371M
[INFO] ------------------------------------------------------------------------
Thanks,
Jonathan Hurley
Re: Review Request 31273: Ambari Server Deadlocks During Cluster
Provisioning
Posted by Tom Beerbower <tb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31273/#review73565
-----------------------------------------------------------
Ship it!
Ship It!
- Tom Beerbower
On Feb. 22, 2015, 2:04 a.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31273/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2015, 2:04 a.m.)
>
>
> Review request for Ambari, Mahadev Konar, Nate Cole, and Tom Beerbower.
>
>
> Bugs: AMBARI-9738
> https://issues.apache.org/jira/browse/AMBARI-9738
>
>
> Repository: ambari
>
>
> Description
> -------
>
> During provisioning of a cluster, a deadlock scenario can occur between {{ClusterImpl}} and {{ClustersImpl}}.
>
> {noformat}
> Thread-1
> Clusters.mapHostsToCluster (writeLock)
> Cluster.refresh (writeLock)
>
> Thread-2
> Cluster.convertToResponse (readLock)
> Clusters.getHost (readLock)
>
> Thread-1 Clusters.mapHostsToCluster (writeLock)
> Thread-2 Cluster.convertToResponse (readLock)
> Thread-1 Cluster.refresh (writeLock) (BLOCKED by Thread-2)
> Thread-2 Clusters.getHost (readLock) (BLOCKED by Thread-1)
> {noformat}
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 8589182
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java 97ac2be
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClustersDeadlockTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/31273/diff/
>
>
> Testing
> -------
>
> Unit tests written before code fix to verify the deadlock could be reproduced. Deployed a clean cluster.
>
> Tests run: 2724, Failures: 0, Errors: 0, Skipped: 15
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 13:55 min
> [INFO] Finished at: 2015-02-21T17:45:45-05:00
> [INFO] Final Memory: 32M/371M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 31273: Ambari Server Deadlocks During Cluster
Provisioning
Posted by Mahadev Konar <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31273/#review73434
-----------------------------------------------------------
Ship it!
Ship It!
- Mahadev Konar
On Feb. 22, 2015, 2:04 a.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31273/
> -----------------------------------------------------------
>
> (Updated Feb. 22, 2015, 2:04 a.m.)
>
>
> Review request for Ambari, Mahadev Konar, Nate Cole, and Tom Beerbower.
>
>
> Bugs: AMBARI-9738
> https://issues.apache.org/jira/browse/AMBARI-9738
>
>
> Repository: ambari
>
>
> Description
> -------
>
> During provisioning of a cluster, a deadlock scenario can occur between {{ClusterImpl}} and {{ClustersImpl}}.
>
> {noformat}
> Thread-1
> Clusters.mapHostsToCluster (writeLock)
> Cluster.refresh (writeLock)
>
> Thread-2
> Cluster.convertToResponse (readLock)
> Clusters.getHost (readLock)
>
> Thread-1 Clusters.mapHostsToCluster (writeLock)
> Thread-2 Cluster.convertToResponse (readLock)
> Thread-1 Cluster.refresh (writeLock) (BLOCKED by Thread-2)
> Thread-2 Clusters.getHost (readLock) (BLOCKED by Thread-1)
> {noformat}
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java 8589182
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java 97ac2be
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClustersDeadlockTest.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/31273/diff/
>
>
> Testing
> -------
>
> Unit tests written before code fix to verify the deadlock could be reproduced. Deployed a clean cluster.
>
> Tests run: 2724, Failures: 0, Errors: 0, Skipped: 15
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 13:55 min
> [INFO] Finished at: 2015-02-21T17:45:45-05:00
> [INFO] Final Memory: 32M/371M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Jonathan Hurley
>
>