You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@yunikorn.apache.org by "Brandon Grams (Jira)" <ji...@apache.org> on 2023/02/03 03:32:00 UTC

[jira] [Created] (YUNIKORN-1555) Completed Spark applications are recovered/remain as New

Brandon Grams created YUNIKORN-1555:
---------------------------------------

             Summary: Completed Spark applications are recovered/remain as New
                 Key: YUNIKORN-1555
                 URL: https://issues.apache.org/jira/browse/YUNIKORN-1555
             Project: Apache YuniKorn
          Issue Type: Bug
          Components: shim - kubernetes
    Affects Versions: 1.1.0, 1.2.0
            Reporter: Brandon Grams
         Attachments: CEEB85D7-D0D5-4D3D-8C0A-730CBA48B150.jpeg

The k8s Spark plugin is [documented|https://github.com/apache/yunikorn-k8shim/blob/master/pkg/appmgmt/sparkoperator/spark.go#L33] as implementing the Recoverable interface, however it does not. This leads to the following recovery outcome when driver pods in a terminal state remain in the cluster pending garbage collection:

!CEEB85D7-D0D5-4D3D-8C0A-730CBA48B150.jpeg|width=1063,height=157!

Instead, we should implement the Recoverable interface so that completed applications have their proper state propagated in such a scenario.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org