[jira] [Created] (YARN-12023) TestCapacityOverTimePolicy fails when run between 23:00 and 24:00 UTC

["Jose Luis López (Jira)" <ji...@apache.org>]

Jose Luis López created YARN-12023:
--------------------------------------

             Summary: TestCapacityOverTimePolicy fails when run between 23:00 and 24:00 UTC
                 Key: YARN-12023
                 URL: https://issues.apache.org/jira/browse/YARN-12023
             Project: Hadoop YARN
          Issue Type: Test
          Components: test, reservation system
            Reporter: Jose Luis López


{{TestCapacityOverTimePolicy#testAllocation[21]}} ({{\{25 * ONEHOUR, 0.25, 1, ONEDAYPERIOD, PlanningQuotaException.class\}}}) fails, also on every rerun, when it runs between 23:00 and 24:00 UTC:
{noformat}
org.opentest4j.AssertionFailedError
  at org.junit.jupiter.api.Assertions.fail(Assertions.java:119)
  at org.apache.hadoop.yarn.server.resourcemanager.reservation.BaseSharingPolicyTest.runTest(BaseSharingPolicyTest.java:138)
  at org.apache.hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy.testAllocation(TestCapacityOverTimePolicy.java:148)
{noformat}

h3. Root cause
{{BaseSharingPolicyTest}} sets {{initTime = System.currentTimeMillis()}} and {{generateRLEAlloc}} folds the reservation onto its period starting at {{initTime % period}}, which for a 1 day period is the UTC time of day. When the remainder of the duration wraps past the period end, the wrap-around arithmetic loses a full period: the full periods are added with {{Resources.multiply(alloc, duration / period - 1)}} (x0 for 25h on 24h) and the wrapped piece collapses to {{[0, rStart - 23h)}}. Between 23:00 and 24:00 UTC a 25h reservation becomes ~1h, fits the 25% average quota, and the expected {{PlanningQuotaException}} is not thrown.

h3. Time windows (sweep of every minute of the day, both periods)
* [21] {{\{25h, 0.25, 1, 1 day\}}}: fails 23:00-23:59 UTC.
* [24] {{\{25h, 0.05, 5, 1 day\}}}: fails only at 23:00 UTC. From 23:01 it gets the same truncated allocation but is rejected anyway, from the 3rd submission, because of YARN-12022 (instead of on the 5th submission because of the quota).
* [14] {{\{1min, 0.25, 3, 2h\}}} and [15] {{\{1min, 0.25, 3, 1 day\}}}: fail when {{initTime % period}} is 59999 or 60000 ms, also because of YARN-12022.
* {{TestNoOverCommitPolicy}} cases do not depend on the time of day.

h3. Fix
* Make {{generateRLEAlloc}} add {{duration / period}} full periods plus the remainder starting at {{initTime % period}}, wrapped once if needed, so the generated allocation always amounts to {{duration}}.
* Use a fixed {{initTime}} (12:30 UTC, clear of the period boundaries), so that every case is decided by the check it targets rather than by YARN-12022. This can be revisited once YARN-12022 is fixed.

Test-only change: https://github.com/joseluisll/hadoop/tree/YARN-TestCapacityOverTimePolicy-time-of-day



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org