You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@yunikorn.apache.org by "Peter Bacsko (Jira)" <ji...@apache.org> on 2023/05/23 10:42:00 UTC
[jira] [Reopened] (YUNIKORN-1555) Completed Spark applications are recovered/remain as New
[ https://issues.apache.org/jira/browse/YUNIKORN-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Peter Bacsko reopened YUNIKORN-1555:
------------------------------------
> Completed Spark applications are recovered/remain as New
> --------------------------------------------------------
>
> Key: YUNIKORN-1555
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1555
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: shim - kubernetes
> Affects Versions: 1.1.0, 1.2.0
> Reporter: Brandon Grams
> Assignee: Brandon Grams
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.3.0
>
> Attachments: CEEB85D7-D0D5-4D3D-8C0A-730CBA48B150.jpeg
>
>
> The k8s Spark plugin is [documented|https://github.com/apache/yunikorn-k8shim/blob/master/pkg/appmgmt/sparkoperator/spark.go#L33] as implementing the Recoverable interface, however it does not. This leads to the following recovery outcome when driver pods in a terminal state remain in the cluster pending garbage collection:
> !CEEB85D7-D0D5-4D3D-8C0A-730CBA48B150.jpeg|width=1063,height=157!
> Instead, we should implement the Recoverable interface so that completed applications have their proper state propagated in such a scenario.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org