[jira] [Commented] (YARN-11835) DockerContainerDeletionTask did not clear tasks in leveldb

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/YARN-11835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18070766#comment-18070766 ] 

ASF GitHub Bot commented on YARN-11835:
---------------------------------------

Khrol opened a new pull request, #8405:
URL: https://github.com/apache/hadoop/pull/8405

   ## YARN-11835
   
   ### Problem
   
   `DockerContainerDeletionTask.run()` never calls `deletionTaskFinished()`, so every Docker container deletion task written to the NM recovery store (RocksDB/LevelDB) accumulates indefinitely and is replayed on every NodeManager restart.
   
   `FileDeletionTask.run()` has always had this call; it was simply missing from `DockerContainerDeletionTask`.
   
   ### Fix
   
   Added the missing `deletionTaskFinished()` call at the end of `DockerContainerDeletionTask.run()`:
   
   ```java
   public void run() {
       LinuxContainerExecutor exec = ((LinuxContainerExecutor)
           getDeletionService().getContainerExecutor());
       exec.removeDockerContainer(containerId);
       deletionTaskFinished(); // was missing — tasks accumulated in recovery store forever
   }
   ```
   
   ### Note on error handling
   
   The reviewer on a related patch asked whether we should wrap `removeDockerContainer` in try/catch and set `setSuccess(false)` on failure, mirroring the pattern in `FileDeletionTask.run()`.
   
   `FileDeletionTask` uses that pattern because `deleteAsUser` throws checked exceptions (`IOException | InterruptedException`) that must be handled. `removeDockerContainer`, however, already catches `ContainerExecutionException` internally and never propagates any exception — adding a try/catch here would be dead code that can never trigger. The simpler form is therefore correct.
   
   ### Testing
   
   Added `testRunCallsDeletionTaskFinished()` to `TestDockerContainerDeletionTask`:
   - Verifies `removeDockerContainer(containerId)` is called.
   - Verifies `stateStore.removeDeletionTask(taskId)` is called — the observable effect of `deletionTaskFinished()` — confirming the task removes itself from the recovery store.




> DockerContainerDeletionTask did not clear tasks in leveldb
> ----------------------------------------------------------
>
>                 Key: YARN-11835
>                 URL: https://issues.apache.org/jira/browse/YARN-11835
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: container
>    Affects Versions: 3.4.1
>            Reporter: gaozhan ding
>            Priority: Major
>              Labels: pull-request-available
>
> DockerContainerDeletionTask did not clear tasks in leveldb, which can result in tasks that were expected to be cleared remaining in leveldb. Each time the nodemanager is restarted, the corresponding deletion task is triggered



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org