You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@yunikorn.apache.org by "Peter Bacsko (Jira)" <ji...@apache.org> on 2023/07/18 20:26:00 UTC
[jira] [Created] (YUNIKORN-1874) Data race: unlocked access in Context.updatePodCondition()
Peter Bacsko created YUNIKORN-1874:
--------------------------------------
Summary: Data race: unlocked access in Context.updatePodCondition()
Key: YUNIKORN-1874
URL: https://issues.apache.org/jira/browse/YUNIKORN-1874
Project: Apache YuniKorn
Issue Type: Bug
Components: shim - kubernetes
Reporter: Peter Bacsko
Assignee: Peter Bacsko
Running some performance tests locally triggered data races:
{noformat}
WARNING: DATA RACE
Read at 0x00c008351538 by goroutine 50598:
k8s.io/api/core/v1.(*Pod).DeepCopyInto()
/home/bacskop/go/pkg/mod/k8s.io/api@v0.27.3/core/v1/zz_generated.deepcopy.go:3369 +0x44
k8s.io/api/core/v1.(*Pod).DeepCopy()
/home/bacskop/go/pkg/mod/k8s.io/api@v0.27.3/core/v1/zz_generated.deepcopy.go:3383 +0x7d4
github.com/apache/yunikorn-k8shim/pkg/cache.(*Task).postTaskAllocated.func1()
/home/bacskop/repos/incubator-yunikorn-k8shim/pkg/cache/task.go:358 +0x760Previous write at 0x00c008351538 by goroutine 66:
k8s.io/kubernetes/pkg/api/v1/pod.UpdatePodCondition()
/home/bacskop/go/pkg/mod/k8s.io/kubernetes@v1.27.3/pkg/api/v1/pod/util.go:380 +0x63d
github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).updatePodCondition()
/home/bacskop/repos/incubator-yunikorn-k8shim/pkg/cache/context.go:1023 +0x65d
github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).HandleContainerStateUpdate()
/home/bacskop/repos/incubator-yunikorn-k8shim/pkg/cache/context.go:1063 +0x30e
github.com/apache/yunikorn-k8shim/pkg/callback.(*AsyncRMCallback).UpdateContainerSchedulingState()
/home/bacskop/repos/incubator-yunikorn-k8shim/pkg/callback/scheduler_callback.go:213 +0x45
github.com/apache/yunikorn-core/pkg/scheduler.(*Scheduler).inspectOutstandingRequests()
/home/bacskop/repos/yunikorn-core/pkg/scheduler/scheduler.go:169 +0x581
github.com/apache/yunikorn-core/pkg/scheduler.(*Scheduler).inspectOutstandingRequests()
/home/bacskop/repos/yunikorn-core/pkg/scheduler/scheduler.go:169 +0x581
github.com/apache/yunikorn-core/pkg/scheduler.(*Scheduler).inspectOutstandingRequests()
/home/bacskop/repos/yunikorn-core/pkg/scheduler/scheduler.go:169 +0x581
github.com/apache/yunikorn-core/pkg/scheduler.(*Scheduler).internalInspectOutstandingRequests()
/home/bacskop/repos/yunikorn-core/pkg/scheduler/scheduler.go:84 +0x38
github.com/apache/yunikorn-core/pkg/scheduler.(*Scheduler).StartService.func3()
/home/bacskop/repos/yunikorn-core/pkg/scheduler/scheduler.go:67 +0x39
{noformat}
The problem is that inside {{Context.updatePodCondition()}}, we modify the {{PodStatus}} field of the pod object, which we use as a source of copies.
It might be better to store the condition separately and protect it with the task lock.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: issues-help@yunikorn.apache.org