You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@yunikorn.apache.org by "Peter Bacsko (Jira)" <ji...@apache.org> on 2024/02/05 22:09:00 UTC

[jira] [Updated] (YUNIKORN-2392) Concurrent map write in SchedulerCache

     [ https://issues.apache.org/jira/browse/YUNIKORN-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Peter Bacsko updated YUNIKORN-2392:
-----------------------------------
    Description: 
While running the simple performance test with MockScheduler, the following error was detected:
{noformat}
fatal error: concurrent map writes

goroutine 16 [running]:
github.com/apache/yunikorn-k8shim/pkg/cache/external.(*SchedulerCache).removeSchedulingTask(0xc0004261e0, {0xc00d6be600?, 0xc00d6be600?})
	/home/bacskop/repos/yunikorn-k8shim/pkg/cache/external/scheduler_cache.go:593 +0x45
github.com/apache/yunikorn-k8shim/pkg/cache/external.(*SchedulerCache).updatePod(0xc0004261e0, 0xc034a96900)
	/home/bacskop/repos/yunikorn-k8shim/pkg/cache/external/scheduler_cache.go:511 +0x8d1
github.com/apache/yunikorn-k8shim/pkg/cache/external.(*SchedulerCache).UpdatePod(0xc0004261e0, 0x1a?)
	/home/bacskop/repos/yunikorn-k8shim/pkg/cache/external/scheduler_cache.go:468 +0xba
github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).updateYuniKornPod(0xc00012a090, 0xc034a96900)
	/home/bacskop/repos/yunikorn-k8shim/pkg/cache/context.go:306 +0x285
github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).UpdatePod(0xc00012a090, {0xc0001da060?, 0xc0001da120?}, {0x1f92940?, 0xc034a96900?})
	/home/bacskop/repos/yunikorn-k8shim/pkg/cache/context.go:288 +0x2b9
k8s.io/client-go/tools/cache.ResourceEventHandlerFuncs.OnUpdate(...)
	/home/bacskop/go/pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/controller.go:246
github.com/apache/yunikorn-k8shim/pkg/client.(*MockedAPIProvider).RunEventHandler.func1()
	/home/bacskop/repos/yunikorn-k8shim/pkg/client/apifactory_mock.go:284 +0x5da
created by github.com/apache/yunikorn-k8shim/pkg/client.(*MockedAPIProvider).RunEventHandler in goroutine 13
	/home/bacskop/repos/yunikorn-k8shim/pkg/client/apifactory_mock.go:240 +0xda
{noformat}
We need locking inside {{{}SchedulerCache.addSchedulingTask(){}}}.

  was:
While running the simple performance test with MockScheduler, the following error was error was detected:

{noformat}
fatal error: concurrent map writes

goroutine 16 [running]:
github.com/apache/yunikorn-k8shim/pkg/cache/external.(*SchedulerCache).removeSchedulingTask(0xc0004261e0, {0xc00d6be600?, 0xc00d6be600?})
	/home/bacskop/repos/yunikorn-k8shim/pkg/cache/external/scheduler_cache.go:593 +0x45
github.com/apache/yunikorn-k8shim/pkg/cache/external.(*SchedulerCache).updatePod(0xc0004261e0, 0xc034a96900)
	/home/bacskop/repos/yunikorn-k8shim/pkg/cache/external/scheduler_cache.go:511 +0x8d1
github.com/apache/yunikorn-k8shim/pkg/cache/external.(*SchedulerCache).UpdatePod(0xc0004261e0, 0x1a?)
	/home/bacskop/repos/yunikorn-k8shim/pkg/cache/external/scheduler_cache.go:468 +0xba
github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).updateYuniKornPod(0xc00012a090, 0xc034a96900)
	/home/bacskop/repos/yunikorn-k8shim/pkg/cache/context.go:306 +0x285
github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).UpdatePod(0xc00012a090, {0xc0001da060?, 0xc0001da120?}, {0x1f92940?, 0xc034a96900?})
	/home/bacskop/repos/yunikorn-k8shim/pkg/cache/context.go:288 +0x2b9
k8s.io/client-go/tools/cache.ResourceEventHandlerFuncs.OnUpdate(...)
	/home/bacskop/go/pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/controller.go:246
github.com/apache/yunikorn-k8shim/pkg/client.(*MockedAPIProvider).RunEventHandler.func1()
	/home/bacskop/repos/yunikorn-k8shim/pkg/client/apifactory_mock.go:284 +0x5da
created by github.com/apache/yunikorn-k8shim/pkg/client.(*MockedAPIProvider).RunEventHandler in goroutine 13
	/home/bacskop/repos/yunikorn-k8shim/pkg/client/apifactory_mock.go:240 +0xda
{noformat}

We need locking inside {{SchedulerCache.addSchedulingTask()}}.


> Concurrent map write in SchedulerCache
> --------------------------------------
>
>                 Key: YUNIKORN-2392
>                 URL: https://issues.apache.org/jira/browse/YUNIKORN-2392
>             Project: Apache YuniKorn
>          Issue Type: Bug
>          Components: shim - kubernetes
>            Reporter: Peter Bacsko
>            Assignee: Craig Condit
>            Priority: Critical
>
> While running the simple performance test with MockScheduler, the following error was detected:
> {noformat}
> fatal error: concurrent map writes
> goroutine 16 [running]:
> github.com/apache/yunikorn-k8shim/pkg/cache/external.(*SchedulerCache).removeSchedulingTask(0xc0004261e0, {0xc00d6be600?, 0xc00d6be600?})
> 	/home/bacskop/repos/yunikorn-k8shim/pkg/cache/external/scheduler_cache.go:593 +0x45
> github.com/apache/yunikorn-k8shim/pkg/cache/external.(*SchedulerCache).updatePod(0xc0004261e0, 0xc034a96900)
> 	/home/bacskop/repos/yunikorn-k8shim/pkg/cache/external/scheduler_cache.go:511 +0x8d1
> github.com/apache/yunikorn-k8shim/pkg/cache/external.(*SchedulerCache).UpdatePod(0xc0004261e0, 0x1a?)
> 	/home/bacskop/repos/yunikorn-k8shim/pkg/cache/external/scheduler_cache.go:468 +0xba
> github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).updateYuniKornPod(0xc00012a090, 0xc034a96900)
> 	/home/bacskop/repos/yunikorn-k8shim/pkg/cache/context.go:306 +0x285
> github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).UpdatePod(0xc00012a090, {0xc0001da060?, 0xc0001da120?}, {0x1f92940?, 0xc034a96900?})
> 	/home/bacskop/repos/yunikorn-k8shim/pkg/cache/context.go:288 +0x2b9
> k8s.io/client-go/tools/cache.ResourceEventHandlerFuncs.OnUpdate(...)
> 	/home/bacskop/go/pkg/mod/k8s.io/client-go@v0.29.1/tools/cache/controller.go:246
> github.com/apache/yunikorn-k8shim/pkg/client.(*MockedAPIProvider).RunEventHandler.func1()
> 	/home/bacskop/repos/yunikorn-k8shim/pkg/client/apifactory_mock.go:284 +0x5da
> created by github.com/apache/yunikorn-k8shim/pkg/client.(*MockedAPIProvider).RunEventHandler in goroutine 13
> 	/home/bacskop/repos/yunikorn-k8shim/pkg/client/apifactory_mock.go:240 +0xda
> {noformat}
> We need locking inside {{{}SchedulerCache.addSchedulingTask(){}}}.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: issues-help@yunikorn.apache.org