You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "swamirishi (via GitHub)" <gi...@apache.org> on 2025/11/27 03:34:42 UTC

[PR] HDDS-14020. Fix Deadlock in Background service shutdown [ozone]

swamirishi opened a new pull request, #9382:
URL: https://github.com/apache/ozone/pull/9382

   ## What changes were proposed in this pull request?
   Currently both background task and shutdown acquire a synchronized lock which can lead to a deadlock on shutdown which means the threadPool shutdown takes time unnecessarily. The synchronized block on task was taken to ensure there are no parallel tasks that run together. Introducing a semaphore with 1 would solve this issue.
   
   ## What is the link to the Apache JIRA
   https://issues.apache.org/jira/browse/HDDS-14020
   
   ## How was this patch tested?
   Unit test in Defrag service finishes faster


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14020. Fix Deadlock in Background service shutdown [ozone]

Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl closed pull request #9382: HDDS-14020. Fix Deadlock in Background service shutdown
URL: https://github.com/apache/ozone/pull/9382


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14020. Fix Deadlock in Background service shutdown [ozone]

Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on code in PR #9382:
URL: https://github.com/apache/ozone/pull/9382#discussion_r2568482554


##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/BackgroundService.java:
##########
@@ -162,7 +165,9 @@ public void run() {
       if (LOG.isDebugEnabled()) {
         LOG.debug("Number of background tasks to execute : {}", tasks.size());
       }
-      synchronized (BackgroundService.this) {
+
+      try {
+        semaphore.acquire();

Review Comment:
   No this would be a problem because the shutdown thread would wait for the threads to finish for the tasks already present in the queue of the threadPoolExecutor. Those have to finish first. Right now the shutdown function would always end up doing a force shutdown if the interval is small enough. 
   This wait would happen unnecessarily everytime.
   https://github.com/apache/ozone/blob/c42fc3aae470e4ac4ac29298a5c03c824d18bae7/hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/BackgroundService.java#L198-L200
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14020. Fix Deadlock in Background service shutdown [ozone]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9382:
URL: https://github.com/apache/ozone/pull/9382#issuecomment-3672807679

   This PR has been marked as stale due to 21 days of inactivity. Please comment or remove the stale label to keep it open. Otherwise, it will be automatically closed in 7 days.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14020. Fix Deadlock in Background service shutdown [ozone]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9382:
URL: https://github.com/apache/ozone/pull/9382#issuecomment-3747087500

   This PR has been marked as stale due to 21 days of inactivity. Please comment or remove the stale label to keep it open. Otherwise, it will be automatically closed in 7 days.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14020. Fix Deadlock in Background service shutdown [ozone]

Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on PR #9382:
URL: https://github.com/apache/ozone/pull/9382#issuecomment-3748528196

   Closing this one in favor of https://github.com/apache/ozone/pull/9390


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org


Re: [PR] HDDS-14020. Fix Deadlock in Background service shutdown [ozone]

Posted by "sumitagrawl (via GitHub)" <gi...@apache.org>.
sumitagrawl commented on code in PR #9382:
URL: https://github.com/apache/ozone/pull/9382#discussion_r2567942095


##########
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/BackgroundService.java:
##########
@@ -162,7 +165,9 @@ public void run() {
       if (LOG.isDebugEnabled()) {
         LOG.debug("Number of background tasks to execute : {}", tasks.size());
       }
-      synchronized (BackgroundService.this) {
+
+      try {
+        semaphore.acquire();

Review Comment:
   background task are added to async executor, and comes out of loop. it will be fraction of second to be in this block, and task count will be max 10s in number. So this is not a problem.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org