You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@yunikorn.apache.org by "Craig Condit (Jira)" <ji...@apache.org> on 2023/03/31 19:25:00 UTC
[jira] [Resolved] (YUNIKORN-1597) Gang scheduling: application might not transition to Running after recovery
[ https://issues.apache.org/jira/browse/YUNIKORN-1597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Craig Condit resolved YUNIKORN-1597.
------------------------------------
Fix Version/s: 1.3.0
Target Version: 1.3.0
Resolution: Fixed
Merged e2e tests to master.
> Gang scheduling: application might not transition to Running after recovery
> ---------------------------------------------------------------------------
>
> Key: YUNIKORN-1597
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1597
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: shim - kubernetes
> Reporter: Peter Bacsko
> Assignee: Peter Bacsko
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.3.0
>
>
> Pods get suck in a certain recovery scenario which involves gang scheduling.
> High level overview:
> 1. All placeholders are running and allocated
> 2. The real pod is in Pending state
> 3. Yunikorn crashes and recovers
> In this case, the real pod will not transition to Running. It's because:
> 1. Upon recovery, the state of recovered tasks will be set to "Allocated", not "Bound".
> 2. If placeholder tasks are already running and allocated, there will be no call to {{postTaskBound()}}.
> A possible fix:
> 1. In {{Task.initialize()}}, set the state to {{Bound}} if it's a placeholder task.
> 2. In {{Application.onReserving()}}, check if we have placeholders. If we do, that means we're after recovery, so send an "UpdateReservation" event.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org