You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@yunikorn.apache.org by "Rainie Li (Jira)" <ji...@apache.org> on 2025/01/09 00:20:00 UTC
[jira] [Created] (YUNIKORN-3007) Improve YuniKorn reservation logic
Rainie Li created YUNIKORN-3007:
-----------------------------------
Summary: Improve YuniKorn reservation logic
Key: YUNIKORN-3007
URL: https://issues.apache.org/jira/browse/YUNIKORN-3007
Project: Apache YuniKorn
Issue Type: Improvement
Components: core - scheduler
Reporter: Rainie Li
Assignee: Rainie Li
Attachments: queue.yaml, test-job1.yaml, test-job2.yaml, test-job3.yaml
*Issue and Investigation:*
We’ve observed spark job slowness issues on our prod cluster, especially when large jobs are running on the cluster. This performance degradation impacts user experience.
When High cluster utilization with numerous pending pods, could cause large jobs that arrive first to reserve resources on nodes. This reservation mechanism prevents new jobs from getting necessary resources, which agains preemption.
*Test case:*
Pls refer to attached files.
# Submit test-job1 to queue-one
# Once test-job1 is running, Submit test-job2 to queue-two
# Once test-job2 is running and pending memory reaches to more than 40TB, Submit test-job3 to queue-three
*Proposal:*
YuniKorn incorporates multiple scenarios for making reservations. To address the current issue, we propose retaining only the preemption-related reservations, as preemption relies on reservations to ensure that resources can be reallocated later.
The rationale for removing other reservation scenarios is as follows:
# If a queue's usage exceeds its guaranteed resources, it should not maintain reservations.
# Conversely, if a queue's usage falls below its guaranteed resources, it should be able to secure resources through preemption.
*Our fix:*
We applied the fix internally to remove allocation case here [https://github.com/apache/yunikorn-core/blob/master/pkg/scheduler/objects/application.go#L1532]
Seems reservation [https://yunikorn.apache.org/release-announce/0.8.0/#resource-reservation] is by design, but in our case it agains preemption
I would like to open this ticket to have a follow up discussion with the community to see what will be better solution to address this issue. cc [~wilfreds]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: issues-help@yunikorn.apache.org