[jira] [Resolved] (YUNIKORN-3427) flushReleaseableTasks removes from the context map and reads task state under the application lock only

["TingYao Huang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/YUNIKORN-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

TingYao Huang resolved YUNIKORN-3427.
-------------------------------------
     Fix Version/s: 1.11.0
    Target Version: 1.11.0
        Resolution: Fixed

> flushReleaseableTasks removes from the context map and reads task state under the application lock only
> -------------------------------------------------------------------------------------------------------
>
>                 Key: YUNIKORN-3427
>                 URL: https://issues.apache.org/jira/browse/YUNIKORN-3427
>             Project: Apache YuniKorn
>          Issue Type: Sub-task
>          Components: shim - kubernetes
>            Reporter: Dale Richardson
>            Assignee: Dian-Xuan Yang
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.11.0
>
>
> {{Application.flushReleaseableTasks}} runs as the enter-Accepted callback under {{app.handle}}'s write lock and nothing else. On the all-terminated path it calls {{app.context.removeApplication}}, the internal variant that deletes from {{ctx.applications}} with no context lock, while {{GetAllApplications}} reads that map under {{ctx.RLock()}} on every scheduling tick. On the other path it calls {{task.releaseAllocation(true)}} for each deferred task, reading {{terminationType}}, {{allocationKey}} and friends that {{SetTaskPod}} and the rollback paths write under {{task.lock}}.
> The context-map delete needs an application whose deferred tasks are all terminated by the time the core accepts it (exactly the YUNIKORN-3089 scenario) colliding with the ticker's read: a concurrent map read and write, i.e. fatal. Traced, not reproduced. The fix for the linked {{shouldAppRelease}} deadlock deliberately leaves both of these out of scope.
> Fix: take {{task.lock}} inside the loop (application before task is the established order), and route the removal through {{Context.RemoveApplication}} from outside the application lock, or defer it to the caller.
> Marker: two {{+checklocksignore}} sites in {{flushReleaseableTasks}} carry this JIRA; the fix removes them.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: issues-help@yunikorn.apache.org