You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@geode.apache.org by "kirklund (GitHub)" <gi...@apache.org> on 2019/01/18 01:26:03 UTC
[GitHub] [geode] kirklund opened pull request #3096: GEODE-6232:
Disable JMX in persistent PersistentPartitionedRegionRegressionTest
The cause of the hang in GEODE-6232 is lock ordering that involves JMX ManagementListener. Since PersistentPartitionedRegionRegressionTest does not need JMX, this change will disable JMX in the test.
Requires PR #3095 to be merged first. The commit in that PR is included in this PR's branch. Please just the 2nd commit which changes PersistentPartitionedRegionRegressionTest to use disable-jmx.
[ Full content available at: https://github.com/apache/geode/pull/3096 ]
This message was relayed via gitbox.apache.org for notifications@geode.apache.org
[GitHub] [geode] jinmeiliao commented on issue #3096: GEODE-6232:
Disable JMX in PersistentPartitionedRegionRegressionTest
Posted by "jinmeiliao (GitHub)" <gi...@apache.org>.
Are we adding this config property just to fix a test? Is this config property required for user to get a particular behavior when using PersistentPartitionedRegion? If not, I would say we either fix the test or get rid of the test.
[ Full content available at: https://github.com/apache/geode/pull/3096 ]
This message was relayed via gitbox.apache.org for notifications@geode.apache.org
[GitHub] [geode] kirklund commented on issue #3096: GEODE-6232:
Disable JMX in PersistentPartitionedRegionRegressionTest
Posted by "kirklund (GitHub)" <gi...@apache.org>.
@jinmeiliao @upthewaterspout The underlying deadlock is filed as [GEODE-6255: ManagementListener may deadlock with Cache close](https://issues.apache.org/jira/browse/GEODE-6255). The only code path I can find that deadlocks is:
* Thread-1 is invoking Cache.close()
* Thread-2 is invoking Region creation for a persistent Region that will use the default Disk Store which has not yet been created
Creating the Disk Store MBean then deadlocks with Cache.close() because the Regions map in GemFireCacheImpl has been acquired in the wrong order with the ReadWriteLock in ManagementListener.
[ Full content available at: https://github.com/apache/geode/pull/3096 ]
This message was relayed via gitbox.apache.org for notifications@geode.apache.org
[GitHub] [geode] kirklund commented on issue #3096: GEODE-6232:
Disable JMX in PersistentPartitionedRegionRegressionTest
Posted by "kirklund (GitHub)" <gi...@apache.org>.
@jinmeiliao @upthewaterspout The underlying deadlock is filed as [GEODE-6255: ManagementListener may deadlock with Cache close](https://issues.apache.org/jira/browse/GEODE-6255). The only code path I can find that deadlocks is:
* Thread-1 is invoking Cache.close()
* Thread-2 is invoking Region creation for a persistent Region that will use the default Disk Store which has not yet been created
Creating the Disk Store (and its MBean) then deadlocks with Cache.close() because the Regions map in GemFireCacheImpl has been acquired in the wrong order with the ReadWriteLock in ManagementListener.
[ Full content available at: https://github.com/apache/geode/pull/3096 ]
This message was relayed via gitbox.apache.org for notifications@geode.apache.org
[GitHub] [geode] jinmeiliao commented on issue #3096: GEODE-6232:
Disable JMX in PersistentPartitionedRegionRegressionTest
Posted by "jinmeiliao (GitHub)" <gi...@apache.org>.
@kirklund I wish I did not comment on this PR then :-). Just ignore my comment.
[ Full content available at: https://github.com/apache/geode/pull/3096 ]
This message was relayed via gitbox.apache.org for notifications@geode.apache.org
[GitHub] [geode] kirklund commented on issue #3096: GEODE-6232:
Disable JMX in PersistentPartitionedRegionRegressionTest
Posted by "kirklund (GitHub)" <gi...@apache.org>.
Prior to disabling JMX in the test, StressNewTest would fail (hanging on the deadlock) 100% in PRs that changed PersistentPartitionedRegionRegressionTest.
[ Full content available at: https://github.com/apache/geode/pull/3096 ]
This message was relayed via gitbox.apache.org for notifications@geode.apache.org
[GitHub] [geode] kirklund commented on issue #3096: GEODE-6232:
Disable JMX in PersistentPartitionedRegionRegressionTest
Posted by "kirklund (GitHub)" <gi...@apache.org>.
@jinmeiliao It's a huge can of worms you don't want to go near. Trust me lol
[ Full content available at: https://github.com/apache/geode/pull/3096 ]
This message was relayed via gitbox.apache.org for notifications@geode.apache.org
[GitHub] [geode] kirklund commented on issue #3096: GEODE-6232:
Disable JMX in PersistentPartitionedRegionRegressionTest
Posted by "kirklund (GitHub)" <gi...@apache.org>.
@jinmeiliao In my discussion with @upthewaterspout yesterday, I noted that fixing the underlying deadlock is going to take a large effort which I don't have time to do. The next option to fix PersistentPartitionedRegionRegressionTest is to bounce the DUnit VM instead of invoking Cache.close() within the callback where it's being called. Dan and I then decided it was best to disable JMX MBeans for the test because test doesn't need MBeans and this will prevent the test from hanging. The underlying deadlock would still need to be fixed but at least precheckin would not hang. I do not have time to do any further work on this. So my proposal is this: I'll close these PRs and reassign the Jira tickets to you. Should I reassign these tickets to you?
[ Full content available at: https://github.com/apache/geode/pull/3096 ]
This message was relayed via gitbox.apache.org for notifications@geode.apache.org
[GitHub] [geode] kirklund closed pull request #3096: GEODE-6232:
Disable JMX in PersistentPartitionedRegionRegressionTest
Posted by "kirklund (GitHub)" <gi...@apache.org>.
[ pull request closed by kirklund ]
[ Full content available at: https://github.com/apache/geode/pull/3096 ]
This message was relayed via gitbox.apache.org for notifications@geode.apache.org
[GitHub] [geode] kirklund commented on issue #3096: GEODE-6232:
Disable JMX in PersistentPartitionedRegionRegressionTest
Posted by "kirklund (GitHub)" <gi...@apache.org>.
@jinmeiliao @upthewaterspout The underlying deadlock is filed as [GEODE-6255: ManagementListener may deadlock with Cache close](https://issues.apache.org/jira/browse/GEODE-6255). The only code path I can find that deadlocks is:
* Thread-1 is invoking Cache.close()
* Thread-2 is invoking Region creation for a persistent Region that will use the default Disk Store which has not yet been created
[ Full content available at: https://github.com/apache/geode/pull/3096 ]
This message was relayed via gitbox.apache.org for notifications@geode.apache.org