You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@yunikorn.apache.org by "Ashwin Shroff (Jira)" <ji...@apache.org> on 2026/03/13 05:47:00 UTC
[jira] [Created] (YUNIKORN-3239) Lag is processing register node event in yunikorn core
Ashwin Shroff created YUNIKORN-3239:
---------------------------------------
Summary: Lag is processing register node event in yunikorn core
Key: YUNIKORN-3239
URL: https://issues.apache.org/jira/browse/YUNIKORN-3239
Project: Apache YuniKorn
Issue Type: Bug
Components: core - scheduler
Reporter: Ashwin Shroff
Attachments: stack-trace.json
We are running yunikorn-core + yunikorn k8s shim. We observe long delays in yunikorn processing node update events.
As shown in this stack trace, shim is waiting for more than 53 min to get an ack from core for a node update event. Also have attached the stack trace collected
goroutine 2976 [sync.WaitGroup.Wait, 53 minutes]: sync.runtime_SemacquireWaitGroup(0xc000056a20?) runtime/sema.go:110 +0x25 sync.(*WaitGroup).Wait(0xc000056a20?) sync/waitgroup.go:118 +0x48 github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).registerNodesInternal(0xc00086f830, \{0xc00039ce48, 0x1, 0x1}, 0xc09be3bd40) github.com/apache/yunikorn-k8shim/pkg/cache/context.go:1605 +0x58c github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).registerNodes(0xc00086f830, \{0xc082d6bad0, 0x1, 0xc000280690?}) github.com/apache/yunikorn-k8shim/pkg/cache/context.go:1538 +0x4c5 github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).registerNode(0xc000280690?, 0xc006e99b08) github.com/apache/yunikorn-k8shim/pkg/cache/context.go:1499 +0x2e github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).updateNodeInternal(0xc00086f830, 0xc006e99b08, 0x1) github.com/apache/yunikorn-k8shim/pkg/cache/context.go:187 +0x79 github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).updateNode(0xc00086f830, \{0xc007aea000?, 0xc016770c08?}, \{0x2509680, 0xc006e99b08}) github.com/apache/yunikorn-k8shim/pkg/cache/context.go:176 +0x291 github.com/apache/yunikorn-k8shim/pkg/cache.(*Context).addNode(...) github.com/apache/yunikorn-k8shim/pkg/cache/context.go:165 k8s.io/client-go/tools/cache.ResourceEventHandlerFuncs.OnAdd(...) k8s.io/client-go@v0.31.0/tools/cache/controller.go:246 k8s.io/client-go/tools/cache.(*processorListener).run.func1() k8s.io/client-go@v0.31.0/tools/cache/shared_informer.go:978 +0xa9 k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0xc000a80008?) k8s.io/apimachinery@v0.31.0/pkg/util/wait/backoff.go:226 +0x33 k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc082d6bf70, \{0x28a6800, 0xc0409f8120}, 0x1, 0xc01434b340) k8s.io/apimachinery@v0.31.0/pkg/util/wait/backoff.go:227 +0xaf k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc010b79770, 0x3b9aca00, 0x0, 0x1, 0xc01434b340) k8s.io/apimachinery@v0.31.0/pkg/util/wait/backoff.go:204 +0x7f k8s.io/apimachinery/pkg/util/wait.Until(...) k8s.io/apimachinery@v0.31.0/pkg/util/wait/backoff.go:161 k8s.io/client-go/tools/cache.(*processorListener).run(0xc01b59afc0) k8s.io/client-go@v0.31.0/tools/cache/shared_informer.go:972 +0x5a k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1() k8s.io/apimachinery@v0.31.0/pkg/util/wait/wait.go:72 +0x4c created by k8s.io/apimachinery/pkg/util/wait.(*Group).Start in goroutine 1 k8s.io/apimachinery@v0.31.0/pkg/util/wait/wait.go:70 +0x73
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: dev-help@yunikorn.apache.org