[jira] [Created] (YUNIKORN-3474) Quota preemption E2E can fail on 5-second pod readiness timeout

["Yen-Hua Chen (Jira)" <ji...@apache.org>]

Yen-Hua Chen created YUNIKORN-3474:
--------------------------------------

             Summary: Quota preemption E2E can fail on 5-second pod readiness timeout
                 Key: YUNIKORN-3474
                 URL: https://issues.apache.org/jira/browse/YUNIKORN-3474
             Project: Apache YuniKorn
          Issue Type: Task
          Components: test - e2e
            Reporter: Yen-Hua Chen
            Assignee: Yen-Hua Chen
         Attachments: logs_97238036669.zip

The Quota_Preemption_Delay_Timer_Reset_On_Quota_Re_Reduction E2E test can fail during its initial pod-readiness setup before the quota-preemption behavior under test is exercised.

Observed failure

In a Kubernetes v1.24.17 CI run, the test reached the initial pod-readiness wait at:

STEP: Applying initial config: quota preemption enabled, max=300Mi @ 09/24/26 04:26:27.448
STEP: Creating deployment in namespace dev-6rgv3 @ 09/24/26 04:26:27.453
STEP: Waiting for all 3 deployment pods to be running @ 09/24/26 04:26:27.458

It then failed at:

[FAILED] Unexpected error:

<context.deadlineExceededError>:
context deadline exceeded

occurred

In [It] at:
/home/runner/work/yunikorn-k8shim/yunikorn-k8shim/test/e2e/quota-preemption/quota_preemption_test.go:513
@ 09/24/26 04:26:32.648

The readiness wait therefore started at 04:26:27.458 and failed at 04:26:32.648, approximately 5.19 seconds later.

Source mapping

The failing assertion is immediately after this call:

err = kClient.WaitForNPodsBySelectorRunning(
    dev, "app=app-a", 3, 5*time.Second)

Ω(err).NotTo(gomega.HaveOccurred())

The shared helper returns context deadline exceeded when the timeout supplied by the caller expires.

In this case, the quota-preemption test itself supplies the 5-second timeout. The following assertion then fails because that wait returned an error.

The failure occurs during initial setup while waiting for the three app-a pods to become Running. The first quota reduction and the timer-reset behavior that this spec is intended to test have not yet been exercised.

Why change it to 30 seconds

Before the count-aware pod-running wait was introduced for this test, this setup used a 30-second pod-running allowance.

Comparable E2E readiness waits in the repository also use 30 seconds or more.

Using 30 seconds therefore restores an existing readiness allowance rather than introducing a new arbitrary timeout.

Proposed change

- err = kClient.WaitForNPodsBySelectorRunning(dev, "app=app-a", 3, 5*time.Second)
+ err = kClient.WaitForNPodsBySelectorRunning(dev, "app=app-a", 3, 30*time.Second)

Scope

This only changes the maximum amount of time allowed for the initial pod-readiness setup.

It does not change:
- the requirement for exactly three matching pods
- the requirement that those pods reach Running
- the workload
- the shared WaitForNPodsBySelectorRunning helper
- the quota-preemption behavior
- any quota-preemption delay or timer-reset assertion

Validation

Validated locally on Kubernetes v1.24.17 after the change.

Focused Quota_Preemption_Delay_Timer_Reset_On_Quota_Re_Reduction spec:
1 passed, 0 failed

Complete quota-preemption suite:
9 passed, 0 failed, 1 pre-existing pending

No production code was changed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@yunikorn.apache.org
For additional commands, e-mail: issues-help@yunikorn.apache.org