You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@yunikorn.apache.org by "Peter Bacsko (Jira)" <ji...@apache.org> on 2024/03/03 10:10:00 UTC

[jira] [Created] (YUNIKORN-2465) Remove Task objects from the shim upon pod completion

Peter Bacsko created YUNIKORN-2465:
--------------------------------------

             Summary: Remove Task objects from the shim upon pod completion
                 Key: YUNIKORN-2465
                 URL: https://issues.apache.org/jira/browse/YUNIKORN-2465
             Project: Apache YuniKorn
          Issue Type: Task
          Components: shim - kubernetes
            Reporter: Peter Bacsko
            Assignee: Peter Bacsko


We don't remove Task objects from the shim when the pod completes. This has consequences for long running workloads which keeps generating new pods with the same applicationID such as Spark Streaming. The ever increasing memory usage eventually results in an OOM and the termination of Yunikorn.

Restart fixes the situation because completed pods are not restored and added to the Context/Application. We should remove the tasks during the lifetime of the application unless there's a good reason not to.





--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: issues-help@yunikorn.apache.org