[GitHub] [hadoop-ozone] xiaoyuyao commented on a change in pull request #627: HDDS-3118. Possible deadlock in LockManager.

[GitBox <gi...@apache.org>]

xiaoyuyao commented on a change in pull request #627: HDDS-3118. Possible deadlock in LockManager.
URL: https://github.com/apache/hadoop-ozone/pull/627#discussion_r387331775
 
 

 ##########
 File path: hadoop-hdds/common/src/test/java/org/apache/hadoop/ozone/lock/TestLockManager.java
 ##########
 @@ -170,4 +175,32 @@ public void testMultiReadWriteLockWithSameResource() throws Exception {
     Assert.assertTrue(gotLock.get());
   }
 
+  @Test
+  public void testConcurrentWriteLockWithDifferentResource() throws Exception {
+    OzoneConfiguration conf = new OzoneConfiguration();
+    final int count = 100;
+    conf.setInt(HDDS_LOCK_MAX_CONCURRENCY, count/4);
 
 Review comment:
   Thanks @bharatviswa504 for the patch. The PR LGTM. I understand we are trying to avoid wait forever while holding the lock of the pool map. However, it does not completely solve the lock resource exhausting problem. I have a cluster with this patch but create key still can live lock by the limited number of locks (100). 
   
   My question is why we are restricting # of locks here via hdds.lock.max.concurrency.  In my opinion, if we change the default from 100 to -1, this deadlock will not happen. This also match the default max pool size (-1) of GenericObjectPool, especially lock is a cheaper resource compared with threads or tcp connections. To allow handling 4K+ request per second, we should consider removing this key that can easily cause deadlock, especially when handler counter is several times larger than the lock pool size. 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org