You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@juneau.apache.org by ja...@apache.org on 2026/10/23 18:11:37 UTC
(juneau) branch master updated: Add the juneau-run live view header, handback env and Overall row; speed up the petstore console browser tests (parallel forks, cached Playwright install, per-host groom/species beans); inject a Clock into InMemoryReplayCache, Uuid7, McpMrtrConfig, RateLimitGuard and ThrownStore so tests pin time; add a .md doc next to every script, with argparse-first --help handling in create-mvn-site, revert-unstaged and revert-staged; and stop log env vars leaking into the script tests.
This is an automated email from the ASF dual-hosted git repository.
jamesbognar pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/juneau.git
The following commit(s) were added to refs/heads/master by this push:
new 2d21583140 Add the juneau-run live view header, handback env and Overall row; speed up the petstore console browser tests (parallel forks, cached Playwright install, per-host groom/species beans); inject a Clock into InMemoryReplayCache, Uuid7, McpMrtrConfig, RateLimitGuard and ThrownStore so tests pin time; add a .md doc next to every script, with argparse-first --help handling in create-mvn-site, revert-unstaged and revert-staged; and stop log env vars leaking into the script tests.
2d21583140 is described below
commit 2d21583140499ba3b5a69d9b9c2054e7d6ada58d
Author: James Bognar <ja...@apache.org>
AuthorDate: Fri Oct 23 14:11:30 2026 -0400
Add the juneau-run live view header, handback env and Overall row; speed up the petstore console browser tests (parallel forks, cached Playwright install, per-host groom/species beans); inject a Clock into InMemoryReplayCache, Uuid7, McpMrtrConfig, RateLimitGuard and ThrownStore so tests pin time; add a .md doc next to every script, with argparse-first --help handling in create-mvn-site, revert-unstaged and revert-staged; and stop log env vars leaking into the script tests.
---
.github/workflows/maven.yml | 1 +
.../commons/concurrent/InMemoryReplayCache.java | 22 +-
.../org/apache/juneau/commons/utils/Uuid7.java | 19 +-
.../concurrent/InMemoryReplayCache_Test.java | 51 +-
.../apache/juneau/commons/utils/Uuid7_Test.java | 11 +
.../server/guard/RateLimitGuard_Snapshot_Test.java | 20 +-
juneau-petstore/juneau-petstore-core/pom.xml | 17 +-
.../juneau/petstore/console/ops/JobsRest.java | 73 +-
.../browser/PetstoreAsyncJob_BrowserTest.java | 16 +-
.../petstore/console/browser/PetstoreBrowser.java | 30 +-
.../console/browser/PetstoreBrowser_Test.java | 42 +
.../browser/PetstoreBulkDiscard_BrowserTest.java | 2 +-
.../browser/PetstoreConsoleWrites_BrowserTest.java | 6 +-
.../console/browser/PetstoreCoverage_Test.java | 2 +-
.../browser/PetstoreHarness_BrowserTest.java | 7 +-
.../console/browser/PetstoreNav_BrowserTest.java | 6 +-
.../browser/PetstorePetDetail_BrowserTest.java | 4 +-
.../browser/PetstoreRowJobCancel_BrowserTest.java | 106 +++
.../browser/PetstoreRowJob_BrowserTest.java | 65 +-
.../browser/PetstoreSecure_BrowserTest.java | 2 +-
.../browser/PetstoreTerminal_BrowserTest.java | 6 +-
.../console/browser/PetstoreTestServer.java | 83 +-
.../browser/PetstoreThemes_BrowserTest.java | 2 +-
.../browser/PetstoreTypedConfirm_BrowserTest.java | 2 +-
.../browser/PetstoreVisibility_BrowserTest.java | 2 +-
.../juneau/petstore/console/ops/JobsRest_Test.java | 91 +++
.../src/test/js/petstore-browser.cjs | 15 +-
.../rest/server/mcp/v20260728/McpMrtrConfig.java | 36 +-
.../rest/server/mcp/v20260728/McpRevision.java | 4 +-
.../server/mcp/v20260728/McpMrtrConfig_Test.java | 16 +
.../server/mcp/v20260728/McpMrtrDispatch_Test.java | 42 +
.../juneau-rest-server-terminal-bus/pom.xml | 8 +-
juneau-rest/juneau-rest-server-terminal/pom.xml | 8 +-
juneau-rest/juneau-rest-server-views/pom.xml | 15 +-
.../juneau/rest/server/guard/RateLimitGuard.java | 41 +-
.../juneau/rest/server/stats/ThrownStats.java | 24 +-
.../juneau/rest/server/stats/ThrownStore.java | 20 +
.../juneau/rest/server/stats/ThrownStore_Test.java | 43 +
juneau-run/src/main/python/juneau_run.py | 790 ++++++++++++++----
juneau-run/src/main/python/juneau_scripts/run.py | 7 +-
juneau-run/src/test/python/support.py | 63 +-
juneau-run/src/test/python/test_estimate.py | 72 ++
juneau-run/src/test/python/test_render.py | 891 +++++++++++++++++++--
juneau-run/src/test/python/test_runner.py | 5 +-
juneau-run/src/test/python/test_scripts_session.py | 12 +
juneau-run/src/test/python/test_session_plan.py | 19 +
scripts/README_SONARQUBE.md | 220 +----
scripts/apply-eclipse-prefs.md | 52 ++
scripts/apply-eclipse-prefs.py | 27 +-
scripts/check-bom-completeness.md | 45 ++
scripts/check-bom-completeness.py | 27 +-
scripts/check-container-tags.md | 46 ++
scripts/check-container-tags.py | 5 +-
scripts/check-fluent-setter-overrides.md | 44 +
scripts/check-fluent-setter-overrides.py | 14 +-
scripts/check-topic-links.md | 44 +
scripts/check-topic-links.py | 8 +-
scripts/cleanup-whitespace.md | 47 ++
scripts/cleanup-whitespace.py | 9 +-
scripts/coverage.md | 59 ++
scripts/coverage.py | 12 +-
scripts/create-mvn-site.md | 54 ++
scripts/create-mvn-site.py | 26 +-
scripts/current-release.md | 45 ++
scripts/current-release.py | 11 +-
scripts/eclipse-warnings.md | 56 ++
scripts/eclipse-warnings.py | 8 +-
scripts/ensure-playwright.cjs | 105 +++
scripts/gen-symbols-key.md | 49 ++
scripts/gen-symbols-key.py | 27 +-
scripts/maven-version.md | 44 +
scripts/maven-version.py | 10 +-
scripts/microbench.md | 65 ++
scripts/microbench.py | 8 +-
scripts/normalize-generic-commas.md | 53 ++
scripts/normalize-generic-commas.py | 8 +-
scripts/perf-required.md | 56 ++
scripts/prompt-pgp-passphrase.md | 47 ++
scripts/prompt-pgp-passphrase.py | 8 +-
scripts/push-timings.md | 56 ++
scripts/push-timings.py | 17 +-
scripts/push.md | 71 ++
scripts/push.py | 50 +-
scripts/release.md | 76 ++
scripts/release.py | 8 +-
scripts/repin-consumers.md | 59 ++
scripts/repin-consumers.py | 12 +-
scripts/revert-staged.md | 53 ++
scripts/revert-staged.py | 80 +-
scripts/revert-unstaged.md | 54 ++
scripts/revert-unstaged.py | 76 +-
scripts/script-docs-exclude.txt | 16 +
scripts/slack-notify.md | 53 ++
scripts/slack-notify.py | 74 +-
scripts/sonarqube.md | 72 ++
scripts/sonarqube.py | 12 +-
scripts/start-petstore-jetty.md | 55 ++
scripts/start-petstore-jetty.py | 18 +-
scripts/start-petstore-springboot.md | 48 ++
scripts/start-petstore-springboot.py | 8 +-
scripts/surefire-rollup.md | 54 ++
scripts/surefire-rollup.py | 4 +-
scripts/test.md | 83 ++
scripts/test.py | 70 +-
scripts/tests/README.md | 17 +-
scripts/tests/conftest.py | 17 +-
scripts/tests/test_create_mvn_site_cli.py | 84 ++
scripts/tests/test_env_isolation.py | 43 +
scripts/tests/test_js_tests_flags.py | 45 +-
scripts/tests/test_push.py | 18 +
scripts/tests/test_release_run.py | 2 +-
scripts/tests/test_revert_staged.py | 108 +++
scripts/tests/test_revert_unstaged.py | 112 +++
scripts/tests/test_run_adoption.py | 6 +-
scripts/tests/test_script_docs.py | 598 ++++++++++++++
scripts/tests/test_slack_notify_cli.py | 45 ++
scripts/tests/test_test_py_console.py | 78 +-
scripts/view-sonar-category.md | 50 ++
scripts/view-sonar-category.py | 5 +-
119 files changed, 5580 insertions(+), 985 deletions(-)
diff --git a/.github/workflows/maven.yml b/.github/workflows/maven.yml
index bf808c1d72..24eb3c68a7 100644
--- a/.github/workflows/maven.yml
+++ b/.github/workflows/maven.yml
@@ -151,6 +151,7 @@ jobs:
-pl juneau-petstore/juneau-petstore-core -am
test
-Dtest='org.apache.juneau.petstore.**.*_BrowserTest' -Dsurefire.failIfNoSpecifiedTests=false
+ -Djuneau.jsTests.forks=2
-Djuneau.jsTests.browsers="${HOME}/.cache/ms-playwright"
# The terminal module's browser tests (Terminal_BrowserTest; TerminalSmoke_BrowserTest also matches but
# skips itself unless its own opt-in property is set): same pinned Playwright and
diff --git a/juneau-core/juneau-commons/src/main/java/org/apache/juneau/commons/concurrent/InMemoryReplayCache.java b/juneau-core/juneau-commons/src/main/java/org/apache/juneau/commons/concurrent/InMemoryReplayCache.java
index a5b1b89edb..ca288be4f4 100644
--- a/juneau-core/juneau-commons/src/main/java/org/apache/juneau/commons/concurrent/InMemoryReplayCache.java
+++ b/juneau-core/juneau-commons/src/main/java/org/apache/juneau/commons/concurrent/InMemoryReplayCache.java
@@ -16,10 +16,11 @@
*/
package org.apache.juneau.commons.concurrent;
+import static org.apache.juneau.commons.utils.Shorts.*;
+import java.time.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.*;
-import static org.apache.juneau.commons.utils.Shorts.*;
/**
* Built-in default {@link ReplayCache}: a per-process, {@link ConcurrentHashMap}-backed seen-identifier set with
@@ -61,6 +62,7 @@ public class InMemoryReplayCache implements ReplayCache {
public static final long DEFAULT_SWEEP_INTERVAL_MS = 1000L;
private final ConcurrentHashMap<String,Long> seen = new ConcurrentHashMap<>();
+ private final Clock clock;
private final long sweepIntervalMs;
private final AtomicLong nextSweepAtMs = new AtomicLong();
@@ -78,8 +80,24 @@ public class InMemoryReplayCache implements ReplayCache {
* {@code 0} sweeps on every call (useful in tests that assert eviction behavior). Must be {@code >= 0}.
*/
InMemoryReplayCache(long sweepIntervalMs) {
+ this(Clock.systemUTC(), sweepIntervalMs);
+ }
+
+ /**
+ * Constructor allowing the wall-clock time source to be overridden.
+ *
+ * <p>
+ * Tests that need deterministic expiry pass a {@link Clock#fixed fixed} or {@link Clock#offset offset} clock
+ * here instead of racing real time.
+ *
+ * @param clock The clock read for the current time when deciding which records have expired. Must not be <jk>null</jk>.
+ * @param sweepIntervalMs The minimum interval, in milliseconds, between opportunistic expired-record sweeps.
+ * {@code 0} sweeps on every call. Must be {@code >= 0}.
+ */
+ InMemoryReplayCache(Clock clock, long sweepIntervalMs) {
if (sweepIntervalMs < 0)
throw new IllegalArgumentException("sweepIntervalMs " + sweepIntervalMs + " must be >= 0");
+ this.clock = reqnn("clock", clock);
this.sweepIntervalMs = sweepIntervalMs;
}
@@ -96,7 +114,7 @@ public class InMemoryReplayCache implements ReplayCache {
* simply skips the sweep (its own {@code putIfAbsent} still runs, so no identifier is ever missed).
*/
private void maybeEvictExpired() {
- var now = System.currentTimeMillis();
+ var now = clock.millis();
var next = nextSweepAtMs.get();
if (now < next)
return;
diff --git a/juneau-core/juneau-commons/src/main/java/org/apache/juneau/commons/utils/Uuid7.java b/juneau-core/juneau-commons/src/main/java/org/apache/juneau/commons/utils/Uuid7.java
index 323859b557..c3acb65a5a 100644
--- a/juneau-core/juneau-commons/src/main/java/org/apache/juneau/commons/utils/Uuid7.java
+++ b/juneau-core/juneau-commons/src/main/java/org/apache/juneau/commons/utils/Uuid7.java
@@ -17,6 +17,7 @@
package org.apache.juneau.commons.utils;
import java.security.*;
+import java.time.*;
import java.util.*;
/**
@@ -43,6 +44,8 @@ public final class Uuid7 {
})
private static final ThreadLocal<SecureRandom> RANDOM = ThreadLocal.withInitial(SecureRandom::new);
+ private static final Clock SYSTEM_CLOCK = Clock.systemUTC();
+
private static final long VERSION_7 = 0x7000L; // version nibble in the low 16 bits of the MSB long
private static final long RAND_A_MASK = 0x0FFFL; // 12 random bits alongside the version nibble
private static final long VARIANT_IETF = 0x2L << 62;// binary 10 in the top 2 bits of the LSB long
@@ -56,8 +59,22 @@ public final class Uuid7 {
* @return A new v7 UUID. Never <jk>null</jk>.
*/
public static UUID create() {
+ return create(SYSTEM_CLOCK);
+ }
+
+ /**
+ * Generates a new version-7 UUID whose timestamp is read from the specified clock.
+ *
+ * <p>
+ * Lets tests pin the encoded timestamp with {@link Clock#fixed(Instant, ZoneId)}. The random bits are still
+ * {@link SecureRandom}-filled, so two UUIDs minted against the same fixed clock differ.
+ *
+ * @param clock The clock supplying the Unix-milliseconds timestamp. Must not be <jk>null</jk>.
+ * @return A new v7 UUID. Never <jk>null</jk>.
+ */
+ public static UUID create(Clock clock) {
var random = RANDOM.get();
- var ts = System.currentTimeMillis() & 0xFFFFFFFFFFFFL;
+ var ts = clock.millis() & 0xFFFFFFFFFFFFL;
var randA = random.nextInt() & RAND_A_MASK;
var msb = (ts << 16) | VERSION_7 | randA;
var randB = random.nextLong() & RAND_B_MASK;
diff --git a/juneau-core/juneau-commons/src/test/java/org/apache/juneau/commons/concurrent/InMemoryReplayCache_Test.java b/juneau-core/juneau-commons/src/test/java/org/apache/juneau/commons/concurrent/InMemoryReplayCache_Test.java
index 3710b45d5d..8e6d781a75 100644
--- a/juneau-core/juneau-commons/src/test/java/org/apache/juneau/commons/concurrent/InMemoryReplayCache_Test.java
+++ b/juneau-core/juneau-commons/src/test/java/org/apache/juneau/commons/concurrent/InMemoryReplayCache_Test.java
@@ -18,6 +18,7 @@ package org.apache.juneau.commons.concurrent;
import static org.junit.jupiter.api.Assertions.*;
+import java.time.*;
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.*;
@@ -31,6 +32,18 @@ import org.junit.jupiter.api.*;
*/
class InMemoryReplayCache_Test extends TestBase {
+ private static final Instant T0 = Instant.parse("2026-01-01T00:00:00Z");
+
+ /** A test clock the test can advance manually. */
+ static final class MutableClock extends Clock {
+ private Instant now;
+ MutableClock(Instant start) { now = start; }
+ void advance(Duration d) { now = now.plus(d); }
+ @Override public ZoneId getZone() { return ZoneOffset.UTC; }
+ @Override public Clock withZone(ZoneId zone) { return this; }
+ @Override public Instant instant() { return now; }
+ }
+
@Test void a01_firstSeenIdReturnsTrue() {
var cache = new InMemoryReplayCache();
assertTrue(cache.checkAndRecord("id-1", System.currentTimeMillis() + 60_000L));
@@ -83,8 +96,8 @@ class InMemoryReplayCache_Test extends TestBase {
// A record's own expiresAtMs bounds its retention: once expired it is swept out by a later call, and an
// id resubmitted after that sweep is observed as first-seen again rather than as a replay. Uses a
// sweep-every-call cache (interval 0) so the eviction is deterministic in-test.
- var cache = new InMemoryReplayCache(0);
- var now = System.currentTimeMillis();
+ var cache = new InMemoryReplayCache(Clock.fixed(T0, ZoneOffset.UTC), 0);
+ var now = T0.toEpochMilli();
assertTrue(cache.checkAndRecord("id-1", now - 1_000L)); // recorded already-expired
// With interval 0 every later call sweeps expired records before recording its own.
assertTrue(cache.checkAndRecord("id-2", now + 60_000L));
@@ -103,8 +116,8 @@ class InMemoryReplayCache_Test extends TestBase {
// The sweep is throttled: with a large interval, an expired record is NOT swept on the immediately
// following call, so a still-recorded (though expired) id is observed as a replay until the window
// elapses. This documents the throttle honestly -- eviction bounds memory opportunistically.
- var cache = new InMemoryReplayCache(600_000L); // 10-minute sweep window: no sweep will fire during this test
- var now = System.currentTimeMillis();
+ var cache = new InMemoryReplayCache(Clock.fixed(T0, ZoneOffset.UTC), 600_000L); // time never reaches the next sweep
+ var now = T0.toEpochMilli();
assertTrue(cache.checkAndRecord("id-1", now - 1_000L)); // first call arms the window (sweeps an empty map)
assertFalse(cache.checkAndRecord("id-1", now - 1_000L)); // still present: throttle suppressed the sweep
}
@@ -112,4 +125,34 @@ class InMemoryReplayCache_Test extends TestBase {
@Test void a08_negativeSweepIntervalRejected() {
assertThrows(IllegalArgumentException.class, () -> new InMemoryReplayCache(-1L));
}
+
+ @Test void a09_nullClockRejected() {
+ assertThrows(IllegalArgumentException.class, () -> new InMemoryReplayCache(null, 0L));
+ }
+
+ @Test void a10_recordIsEvictedOnceClockPassesItsExpiry() {
+ // Clock-driven: an unexpired record is a replay until the clock reaches its expiresAtMs, after which the next
+ // call sweeps it and the same id is first-seen again. No sleeping, so the outcome never depends on machine speed.
+ var clock = new MutableClock(T0);
+ var cache = new InMemoryReplayCache(clock, 0);
+ var expiresAtMs = T0.plusSeconds(60).toEpochMilli();
+ assertTrue(cache.checkAndRecord("id-1", expiresAtMs));
+ clock.advance(Duration.ofSeconds(59));
+ assertFalse(cache.checkAndRecord("id-1", expiresAtMs)); // still unexpired: replay
+ clock.advance(Duration.ofSeconds(1));
+ assertTrue(cache.checkAndRecord("id-1", expiresAtMs)); // expiresAtMs <= now: swept, so first-seen again
+ }
+
+ @Test void a11_throttledSweepRunsOnceTheIntervalElapses() {
+ // The throttle window is measured on the injected clock: an expired record survives calls made inside the
+ // window and is swept by the first call at or after it.
+ var clock = new MutableClock(T0);
+ var cache = new InMemoryReplayCache(clock, 1_000L);
+ var expiredAtMs = T0.toEpochMilli() - 1L;
+ assertTrue(cache.checkAndRecord("id-1", expiredAtMs)); // arms the window at T0 + 1s
+ clock.advance(Duration.ofMillis(999));
+ assertFalse(cache.checkAndRecord("id-1", expiredAtMs)); // inside the window: no sweep yet
+ clock.advance(Duration.ofMillis(1));
+ assertTrue(cache.checkAndRecord("id-1", expiredAtMs)); // window elapsed: swept, so first-seen again
+ }
}
diff --git a/juneau-core/juneau-commons/src/test/java/org/apache/juneau/commons/utils/Uuid7_Test.java b/juneau-core/juneau-commons/src/test/java/org/apache/juneau/commons/utils/Uuid7_Test.java
index 14555213bb..d237eed550 100644
--- a/juneau-core/juneau-commons/src/test/java/org/apache/juneau/commons/utils/Uuid7_Test.java
+++ b/juneau-core/juneau-commons/src/test/java/org/apache/juneau/commons/utils/Uuid7_Test.java
@@ -18,6 +18,7 @@ package org.apache.juneau.commons.utils;
import static org.junit.jupiter.api.Assertions.*;
+import java.time.*;
import java.util.*;
import org.apache.juneau.commons.*;
@@ -63,6 +64,16 @@ class Uuid7_Test extends TestBase {
assertTrue(t >= before && t <= after, "encoded timestamp must be the current wall-clock ms");
}
+ @Test void a06_timestampReadFromInjectedClock() {
+ var clock = Clock.fixed(Instant.parse("2026-01-01T00:00:00Z"), ZoneOffset.UTC);
+ var u1 = Uuid7.create(clock);
+ var u2 = Uuid7.create(clock);
+ assertEquals(clock.millis(), Uuid7.timestamp(u1));
+ assertEquals(clock.millis(), Uuid7.timestamp(u2));
+ assertNotEquals(u1, u2, "random bits must still differ under a fixed clock");
+ assertEquals(7, u1.version());
+ }
+
//====================================================================================================
// SecureRandom is reused per thread (no fresh instantiation per call)
//====================================================================================================
diff --git a/juneau-integration-tests/src/test/java/org/apache/juneau/rest/server/guard/RateLimitGuard_Snapshot_Test.java b/juneau-integration-tests/src/test/java/org/apache/juneau/rest/server/guard/RateLimitGuard_Snapshot_Test.java
index 6df9df5685..9cfce45e3f 100644
--- a/juneau-integration-tests/src/test/java/org/apache/juneau/rest/server/guard/RateLimitGuard_Snapshot_Test.java
+++ b/juneau-integration-tests/src/test/java/org/apache/juneau/rest/server/guard/RateLimitGuard_Snapshot_Test.java
@@ -42,7 +42,10 @@ class RateLimitGuard_Snapshot_Test extends TestBase {
//------------------------------------------------------------------------------------------------------------------
@Test void a01_inMemorySnapshotReportsTokensAndRemainingAfterPartialDrain() {
- var s = RateLimitGuard.Storage.inMemory();
+ // Both time sources pinned: a frozen nanosecond source means no refill between calls, and a fixed wall clock
+ // makes lastRequest an exact instant rather than a "close to now" tolerance check.
+ var now = Instant.parse("2026-01-01T00:00:00Z");
+ var s = new RateLimitGuard.InMemoryStorage(100_000, () -> 0L, Clock.fixed(now, ZoneOffset.UTC));
s.tryAcquire("k", 5, 1.0);
s.tryAcquire("k", 5, 1.0);
s.tryAcquire("k", 5, 1.0);
@@ -51,16 +54,11 @@ class RateLimitGuard_Snapshot_Test extends TestBase {
var b = snapshot.get("k");
assertNotNull(b);
assertEquals("k", b.key());
- // Started full (5), three tokens consumed → at-most ~2 left, refill barely measurable in the test
- // window so the actual fraction sits in (2.0, 2.0 + ε].
- assertTrue(b.tokens() >= 2.0 && b.tokens() < 3.0,
- "expected tokens in [2.0, 3.0), got " + b.tokens());
- assertEquals((int) Math.floor(b.tokens()), b.remaining());
- assertFalse(b.throttled(), "bucket should not be throttled with 2+ tokens left");
- assertNotNull(b.lastRequest());
- // Wall-clock instant should be within a few seconds of now.
- var skewMillis = Math.abs(System.currentTimeMillis() - b.lastRequest().toEpochMilli());
- assertTrue(skewMillis < 5_000L, "lastRequest drift too large: " + skewMillis + "ms");
+ // Started full (5), three tokens consumed, no time elapsed on the nanosecond source.
+ assertEquals(2.0, b.tokens());
+ assertEquals(2, b.remaining());
+ assertFalse(b.throttled(), "bucket should not be throttled with 2 tokens left");
+ assertEquals(now, b.lastRequest());
}
//------------------------------------------------------------------------------------------------------------------
diff --git a/juneau-petstore/juneau-petstore-core/pom.xml b/juneau-petstore/juneau-petstore-core/pom.xml
index 903d152e17..e84bd9ab88 100644
--- a/juneau-petstore/juneau-petstore-core/pom.xml
+++ b/juneau-petstore/juneau-petstore-core/pom.xml
@@ -174,7 +174,8 @@
mvn -Pjs-tests -pl juneau-petstore/juneau-petstore-core -am test -Dtest='org.apache.juneau.petstore.**.*_BrowserTest' -Dsurefire.failIfNoSpecifiedTests=false
A copy of juneau-rest-server-views' js-tests profile (see the notes there on target/js, the browser cache
- and the -Djuneau.jsTests.* overrides), with only the harness path and the npm packages changed: the
+ and the -Djuneau.jsTests.* overrides; -Djuneau.jsTests.forks=N sets how many classes run at once, 3 by
+ default), with only the harness path and the npm packages changed: the
petstore pages load jQuery and DataTables from the WebJars the console serves (WebJarsMixin).
-->
<profile>
@@ -186,6 +187,8 @@
<juneau.jsTests.npm>npm</juneau.jsTests.npm>
<js.tests.dir>${project.build.directory}/js</js.tests.dir>
<juneau.jsTests.browsers>${js.tests.dir}/browsers</juneau.jsTests.browsers>
+ <!-- Surefire forks the *_BrowserTest classes run in at once; override with -Djuneau.jsTests.forks=N (1 is serial). -->
+ <juneau.jsTests.forks>3</juneau.jsTests.forks>
</properties>
<build>
<plugins>
@@ -201,13 +204,11 @@
<goal>exec</goal>
</goals>
<configuration>
- <executable>${juneau.jsTests.npm}</executable>
+ <executable>${juneau.jsTests.node}</executable>
<arguments>
- <argument>install</argument>
- <argument>--no-fund</argument>
- <argument>--no-audit</argument>
- <argument>--prefix</argument>
+ <argument>${maven.multiModuleProjectDirectory}/scripts/ensure-playwright.cjs</argument>
<argument>${js.tests.dir}</argument>
+ <argument>${juneau.jsTests.npm}</argument>
<argument>playwright@${playwright.version}</argument>
</arguments>
</configuration>
@@ -238,6 +239,10 @@
<configuration>
<!-- Run the *_BrowserTest classes the default build excludes. -->
<excludes combine.self="override"/>
+ <!-- Each *_BrowserTest class owns its server, port, store and target/js files (suite names are unique),
+ so classes can run side by side. Override with -Djuneau.jsTests.forks=N. -->
+ <forkCount>${juneau.jsTests.forks}</forkCount>
+ <reuseForks>true</reuseForks>
<systemPropertyVariables>
<juneau.jsTests>true</juneau.jsTests>
<juneau.jsTests.dir>${js.tests.dir}</juneau.jsTests.dir>
diff --git a/juneau-petstore/juneau-petstore-core/src/main/java/org/apache/juneau/petstore/console/ops/JobsRest.java b/juneau-petstore/juneau-petstore-core/src/main/java/org/apache/juneau/petstore/console/ops/JobsRest.java
index c601ffaa1e..691d33d132 100644
--- a/juneau-petstore/juneau-petstore-core/src/main/java/org/apache/juneau/petstore/console/ops/JobsRest.java
+++ b/juneau-petstore/juneau-petstore-core/src/main/java/org/apache/juneau/petstore/console/ops/JobsRest.java
@@ -53,7 +53,12 @@ import org.apache.juneau.rest.server.widgets.*;
* with {@code slowMs} waits that long before it touches the store (cancel it then and nothing changes), and every
* restock holds for {@code holdMs} (a second by default) after it has added its pets (cancel it then and the result
* says the change was made).
- * The registry's hard timeout is 30 s, or whatever a {@link HardTimeout} bean on the host says.
+ * The registry's hard timeout is 30 s, or whatever a {@link HardTimeout} bean on the host says; a {@link GroomPeriod}
+ * bean sets the groom step and a {@link SpeciesHold} bean the species-row hold.
+ *
+ * <p>
+ * The seeded groom run is created on the first groom request rather than at construction, so a {@link GroomPeriod} bean
+ * (injected after construction) can set its pace; every groom endpoint goes through it, so the page sees the same data.
*
* <p>
* The page also hosts the terminal demo: a "groom pet" job ({@link GroomRuns}) whose output is served by
@@ -113,8 +118,8 @@ public class JobsRest extends PetstoreConsolePage implements AsyncJobsMixin, Ter
/** When each species was last restocked from its row, for the row's detail panel. */
private final transient Map<Species,Instant> lastRestock = new ConcurrentHashMap<>();
- /** The groom-pet demo runs; one verbose run is seeded so the page always has a log. */
- private final transient GroomRuns groom = GroomRuns.seeded(Long.getLong(GroomRuns.PERIOD_PROPERTY, GroomRuns.DEFAULT_PERIOD_MS));
+ /** The groom-pet demo runs, created on first use; one verbose run is seeded so the page always has a log. */
+ private final transient AtomicReference<GroomRuns> groomRef = new AtomicReference<>();
/** The default hard timeout of a restock job. */
public static final Duration DEFAULT_HARD_TIMEOUT = Duration.ofSeconds(30);
@@ -159,12 +164,39 @@ public class JobsRest extends PetstoreConsolePage implements AsyncJobsMixin, Ter
@Bean
private transient HardTimeout hardTimeout;
+ /**
+ * A per-host groom step period. Declare it as a {@code @Bean} on the host, as {@link HardTimeout}.
+ *
+ * @param value Time between steps, at least 1 ms.
+ */
+ public record GroomPeriod(Duration value) {}
+
+ /**
+ * A per-host species-row hold: how long a species row's restock holds after its effect. Declare it as a
+ * {@code @Bean} on the host, as {@link HardTimeout}.
+ *
+ * @param value The post-effect hold, 0 to {@value #MAX_HOLD_MS} ms.
+ */
+ public record SpeciesHold(Duration value) {}
+
@SuppressWarnings({
"java:S2226" // the field is set by the framework's @Bean injection after construction, so it cannot be final.
})
@Bean
private transient KeyClock keyClock;
+ @SuppressWarnings({
+ "java:S2226" // the field is set by the framework's @Bean injection after construction, so it cannot be final.
+ })
+ @Bean
+ private transient GroomPeriod groomPeriod;
+
+ @SuppressWarnings({
+ "java:S2226" // the field is set by the framework's @Bean injection after construction, so it cannot be final.
+ })
+ @Bean
+ private transient SpeciesHold speciesHold;
+
private final transient AtomicReference<AsyncJobRegistry> jobs = new AtomicReference<>();
private final transient ExecutorService pool = Executors.newCachedThreadPool(r -> {
var t = new Thread(r, "petstore-restock");
@@ -248,6 +280,21 @@ public class JobsRest extends PetstoreConsolePage implements AsyncJobsMixin, Ter
}
}
+ /** @return The groom runs, created (with their seeded run) on first use. */
+ private GroomRuns groomRuns() {
+ var g = groomRef.get();
+ if (g == null) {
+ var n = GroomRuns.seeded(groomPeriod != null ? groomPeriod.value().toMillis() : Long.getLong(GroomRuns.PERIOD_PROPERTY, GroomRuns.DEFAULT_PERIOD_MS));
+ if (groomRef.compareAndSet(null, n)) {
+ g = n;
+ } else {
+ n.close();
+ g = groomRef.get();
+ }
+ }
+ return g;
+ }
+
/** @return The row-key ledger, on the host's {@link KeyClock} if it declares one. */
private JobKeyLedger jobKeys() {
var c = keyClock;
@@ -278,7 +325,9 @@ public class JobsRest extends PetstoreConsolePage implements AsyncJobsMixin, Ter
var j = jobs.get();
if (j != null)
j.close();
- groom.close();
+ var g = groomRef.get();
+ if (g != null)
+ g.close();
}
/**
@@ -291,7 +340,7 @@ public class JobsRest extends PetstoreConsolePage implements AsyncJobsMixin, Ter
*/
@RestGet(path="/")
public View page(RestRequest req, @Query("run") String run) {
- var r = groom.get(run).orElseGet(groom::latest);
+ var r = groomRuns().get(run).orElseGet(groomRuns()::latest);
if (r == null)
throw new NotFound("No groom run to show");
var main = terminal("groom-output", r.id(), "Groom log");
@@ -471,7 +520,7 @@ public class JobsRest extends PetstoreConsolePage implements AsyncJobsMixin, Ter
@Override /* TerminalMixin */
public Optional<TerminalSource> terminalSource(String id, RestRequest req) {
- return groom.source(id); // The run id is an unguessable 128-bit secret; knowing it is the access rule.
+ return groomRuns().source(id); // The run id is an unguessable 128-bit secret; knowing it is the access rule.
}
/**
@@ -503,7 +552,7 @@ public class JobsRest extends PetstoreConsolePage implements AsyncJobsMixin, Ter
public ActionResult groom(@Content JsonMap json) throws IOException {
var fields = json.getMap("fields");
var src = fields != null ? fields : json;
- var run = groom.start(src.getString("pet"), isOn(src.get("verbose")), json.getString("idempotencyKey"));
+ var run = groomRuns().start(src.getString("pet"), isOn(src.get("verbose")), json.getString("idempotencyKey"));
return ActionResult.success(run.row()).message(f("Grooming %s", run.pet()));
}
@@ -517,7 +566,7 @@ public class JobsRest extends PetstoreConsolePage implements AsyncJobsMixin, Ter
*/
@RestGet(path="/groom-rows")
public List<JsonMap> groomRows() {
- return groom.list().stream().map(GroomRuns.Run::row).toList();
+ return groomRuns().list().stream().map(GroomRuns.Run::row).toList();
}
//-----------------------------------------------------------------------------------------------------------------
@@ -606,7 +655,7 @@ public class JobsRest extends PetstoreConsolePage implements AsyncJobsMixin, Ter
}
// After the replay check: a replay answers its job even if this resubmit's fields have changed.
var body = new Restock(Map.of(s, count(submit.fields())), grant.key().value(), 0,
- Math.max(0, Math.min(Integer.getInteger(SPECIES_HOLD_PROPERTY, DEFAULT_HOLD_MS), MAX_HOLD_MS)));
+ Math.max(0, Math.min(speciesHold != null ? (int) speciesHold.value().toMillis() : Integer.getInteger(SPECIES_HOLD_PROPERTY, DEFAULT_HOLD_MS), MAX_HOLD_MS)));
var created = registry.runningCount() >= Integer.getInteger(LIMIT_PROPERTY, 2) ? null : registry.tryCreate(grant.key()).orElse(null);
if (created == null)
return limitReached(res, registry.runningCount());
@@ -696,9 +745,9 @@ public class JobsRest extends PetstoreConsolePage implements AsyncJobsMixin, Ter
return refused(res, RowJobSubmit.bindingRefusal());
var claimed = grant.jobId().orElse(null);
if (claimed != null)
- return groom.get(claimed).map(x -> ActionResult.success(x.row()).message(f("Grooming %s again", x.pet())))
+ return groomRuns().get(claimed).map(x -> ActionResult.success(x.row()).message(f("Grooming %s again", x.pet())))
.orElseGet(() -> refused(res, ActionResult.refusal(JobKeyLedger.SPENT_CODE).message("This key's run has aged out; fetch a new key")));
- var again = groom.start(groomRun(id).pet(), false, grant.key().value());
+ var again = groomRuns().start(groomRun(id).pet(), false, grant.key().value());
ledger.claim(grant.key(), again.id());
return ActionResult.success(again.row()).message(f("Grooming %s again", again.pet()));
}
@@ -733,7 +782,7 @@ public class JobsRest extends PetstoreConsolePage implements AsyncJobsMixin, Ter
}
private GroomRuns.Run groomRun(String id) {
- return groom.get(id).orElseThrow(() -> new NotFound("Unknown groom run '%s'", id));
+ return groomRuns().get(id).orElseThrow(() -> new NotFound("Unknown groom run '%s'", id));
}
private static int count(JsonMap fields) {
diff --git a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreAsyncJob_BrowserTest.java b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreAsyncJob_BrowserTest.java
index 335a67754c..4776e38697 100644
--- a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreAsyncJob_BrowserTest.java
+++ b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreAsyncJob_BrowserTest.java
@@ -77,7 +77,7 @@ class PetstoreAsyncJob_BrowserTest extends TestBase {
// Holds the stream's two subscriber slots open on a slow job, then asks for a third.
private static final String OVER_THE_CAP = "(async () => {"
- + " const ref = await " + start(4000) + ";"
+ + " const ref = await " + start(2500) + ";"
+ " window.__refs = [ref];"
+ " const open = () => new Promise((res, rej) => { const es = new EventSource(ref.streamUrl);"
+ " es.onopen = () => res(es); setTimeout(() => { es.close(); rej(new Error('stream did not open')); }, 8000); });"
@@ -143,33 +143,33 @@ class PetstoreAsyncJob_BrowserTest extends TestBase {
Map.of("name", "cancel-early", "path", "/console/ops/jobs",
"actions", List.of(phase("idle", 8000),
Map.of("evaluate", PET_COUNT + ".then(n => { window.__before = n; })"), wait("window.__before != null", 8000),
- slowMs(3000), Map.of("evaluate", "window.__posted = Date.now()"), SUBMIT, line("Waiting", 8000),
+ slowMs(2000), Map.of("evaluate", "window.__posted = Date.now()"), SUBMIT, line("Waiting", 8000),
wait("!document.querySelector('" + J + " .jc-joblog-submit').disabled", 8000), SUBMIT,
phase("done", 10000),
// A worker that ignored the cancel would restock once its wait ran out, so count only after that.
- wait("Date.now() - window.__posted > 3500", 8000),
+ wait("Date.now() - window.__posted > 2500", 8000),
Map.of("evaluate", PET_COUNT + ".then(n => { window.__after = n; })"), wait("window.__after != null", 8000)),
"queries", Map.of("lines", CARD_QUERIES.get("lines"), "banner", CARD_QUERIES.get("banner"),
"pets", "[window.__before, window.__after]")),
Map.of("name", "cancel-late", "path", "/console/ops/jobs",
- // The 6 s hold keeps the job cancellable well past the Added line and ends 6 s short of the 12 s hard
+ // The 2.5 s hold keeps the job cancellable past the Added line and ends 2.5 s short of the 5 s hard
// timeout, so the cancel always lands in the hold.
- "actions", List.of(phase("idle", 8000), holdMs(6000), SUBMIT, line("Added", 8000), SUBMIT, phase("done", 10000)),
+ "actions", List.of(phase("idle", 8000), holdMs(2500), SUBMIT, line("Added", 8000), SUBMIT, phase("done", 10000)),
"queries", CARD_QUERIES),
// The third subscriber's 429 is expected.
Map.of("name", "subscriber-cap", "path", "/console/ops/jobs", "allowFailedLoads", true,
"actions", List.of(phase("idle", 8000), Map.of("evaluate", "window.__cap = " + OVER_THE_CAP)),
"queries", Map.of("cap", "window.__cap")),
- // The job would run 20 s; the 12 s timeout lands in its hold. This passes even against a scheduler that fires
+ // The job would run 8 s; the 5 s timeout lands in its hold. This passes even against a scheduler that fires
// its timeout task a little late; the early- and on-the-deadline firing it once hung on is pinned
// deterministically by AsyncJobRegistry_Test's k group.
Map.of("name", "timeout", "path", "/console/ops/jobs",
- "actions", List.of(phase("idle", 8000), slowMs(10000), holdMs(10000), SUBMIT, phase("done", 25000)),
+ "actions", List.of(phase("idle", 8000), slowMs(3000), holdMs(5000), SUBMIT, phase("done", 12000)),
"queries", CARD_QUERIES),
// The two fetched jobs fill the limit, so the card's own submit is refused with 429.
Map.of("name", "limit", "path", "/console/ops/jobs", "allowFailedLoads", true,
"actions", List.of(phase("idle", 8000),
- Map.of("evaluate", "window.__started = Promise.all([" + start(4000) + ", " + start(4000) + "]).then(r => { window.__refs = r; })"),
+ Map.of("evaluate", "window.__started = Promise.all([" + start(2500) + ", " + start(2500) + "]).then(r => { window.__refs = r; })"),
wait("window.__refs && window.__refs.length === 2", 8000),
SUBMIT,
wait("document.querySelector('" + J + " .jc-joblog-banner.jc-joblog-error:not([hidden])')", 8000),
diff --git a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreBrowser.java b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreBrowser.java
index 67247ea754..24c8767713 100644
--- a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreBrowser.java
+++ b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreBrowser.java
@@ -24,6 +24,7 @@ import java.io.*;
import java.nio.file.*;
import java.util.*;
import java.util.concurrent.*;
+import java.util.stream.*;
import org.apache.juneau.marshall.marshaller.*;
@@ -32,8 +33,8 @@ import org.apache.juneau.marshall.marshaller.*;
*
* <p>
* A case is a JSON object: {@code name}, {@code path} (appended to the base URL), optional {@code waitUntil} (the
- * navigation's Playwright wait state, {@code networkidle} by default; a polling page can pass {@code load} and start
- * with a {@code waitFor}), optional {@code actions}
+ * navigation's Playwright wait state, {@code load} by default, so start with a {@code waitFor}; a case that asserts
+ * straight after the navigation passes {@code networkidle}), optional {@code actions}
* (each {@code {click|fill|press|waitFor|waitForCaptured|evaluate|screenshot|download: ...}}, or {@code {reload|back|forward: true}};
* {@code reload} waits for the case's {@code waitUntil} and {@code back}/{@code forward} only for the navigation to
* commit, so follow them with a {@code waitFor}), {@code queries} (name to a JS expression
@@ -50,7 +51,8 @@ import org.apache.juneau.marshall.marshaller.*;
* The report for each case has {@code status} (the navigation's HTTP status), {@code jsFailures} (page errors
* and {@code console.error} lines), {@code deprecations} ({@code console.warn} lines matching
* {@code is deprecated}), {@code queries}, {@code fetches}, {@code requests}, {@code downloads} (by the {@code download} action's
- * {@code as} name: {@code {name, bytes, head}}) and {@code timeouts}.
+ * {@code as} name: {@code {name, bytes, head}}), {@code timeouts} and {@code ms} (the case's wall time in
+ * milliseconds, including its browser context).
*
* <h5 class='section'>Example:</h5>
* <p class='bjava'>
@@ -102,9 +104,31 @@ public final class PetstoreBrowser {
})
var r = (Map<String,Map<String,Object>>) Json.to(Files.readString(out), Map.class).get("cases");
assertNotNull(r, () -> "Harness '" + suite + "' wrote no 'cases' object; stdout:\n" + quietRead(out));
+ printSlowest(suite, r);
return r;
}
+ @SuppressWarnings({
+ "java:S106" // The slowest cases are the measurement this class exists to report.
+ })
+ private static void printSlowest(String suite, Map<String,Map<String,Object>> report) {
+ System.out.println("[petstore-browser] " + suite + ": " + report.size() + " cases, slowest: " + slowest(report, 5));
+ }
+
+ /**
+ * @param report The {@link #run} result.
+ * @param n How many cases to list.
+ * @return The {@code n} slowest cases as {@code "name 1234 ms"}, longest first, comma separated.
+ */
+ static String slowest(Map<String,Map<String,Object>> report, int n) {
+ return report.entrySet().stream()
+ .filter(e -> e.getValue().get("ms") instanceof Number)
+ .sorted((a, b) -> Long.compare(((Number) b.getValue().get("ms")).longValue(), ((Number) a.getValue().get("ms")).longValue()))
+ .limit(n)
+ .map(e -> e.getKey() + " " + ((Number) e.getValue().get("ms")).longValue() + " ms")
+ .collect(Collectors.joining(", "));
+ }
+
/**
* Asserts the case loaded with status 200, no script errors, no deprecation warnings and no timeouts.
*
diff --git a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreBrowser_Test.java b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreBrowser_Test.java
new file mode 100644
index 0000000000..1cd5098c7e
--- /dev/null
+++ b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreBrowser_Test.java
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.juneau.petstore.console.browser;
+
+import static org.junit.jupiter.api.Assertions.*;
+
+import java.util.*;
+
+import org.junit.jupiter.api.*;
+
+/**
+ * Pins the formatting of {@link PetstoreBrowser#slowest}, which needs no browser.
+ */
+class PetstoreBrowser_Test {
+
+ @Test void a01_slowestListsTheLongestCasesFirst() {
+ var report = new LinkedHashMap<String,Map<String,Object>>();
+ report.put("fast", Map.of("ms", 40));
+ report.put("slow", Map.of("ms", 9100));
+ report.put("mid", Map.of("ms", 800));
+ assertEquals("slow 9100 ms, mid 800 ms", PetstoreBrowser.slowest(report, 2));
+ }
+
+ @Test void a02_slowestIgnoresCasesWithoutMs() {
+ var report = Map.<String,Map<String,Object>>of("a", Map.of("status", 200), "b", Map.of("ms", 5));
+ assertEquals("b 5 ms", PetstoreBrowser.slowest(report, 5));
+ }
+}
diff --git a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreBulkDiscard_BrowserTest.java b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreBulkDiscard_BrowserTest.java
index 8d90e34158..5746e043d3 100644
--- a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreBulkDiscard_BrowserTest.java
+++ b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreBulkDiscard_BrowserTest.java
@@ -114,7 +114,7 @@ class PetstoreBulkDiscard_BrowserTest extends TestBase {
"states", "Array.from(document.querySelectorAll('#changes tbody input.juneau-view-select-checkbox:disabled'))"
+ ".map(b => b.closest('tr').textContent.includes('APPLIED') ? 'APPLIED' : b.closest('tr').textContent.includes('FAILED') ? 'FAILED' : '?')")),
// Runs before 'discard', which changes the store; this one is refused and changes nothing.
- Map.of("name", "discard-fail", "path", "/console/pets/changes", "allowFailedLoads", true,
+ Map.of("name", "discard-fail", "waitUntil", "networkidle", "path", "/console/pets/changes", "allowFailedLoads", true,
"actions", concat(List.of(Map.of("evaluate", FAIL_DISCARDS)), confirm, submit,
List.of(Map.of("waitFor", "document.querySelector('" + TOAST + "[role=alert]')"))),
"queries", Map.of(
diff --git a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreConsoleWrites_BrowserTest.java b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreConsoleWrites_BrowserTest.java
index 56d16290e4..c1ae16c5b5 100644
--- a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreConsoleWrites_BrowserTest.java
+++ b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreConsoleWrites_BrowserTest.java
@@ -247,19 +247,19 @@ class PetstoreConsoleWrites_BrowserTest extends TestBase {
"queries", Map.of("writes", "window.__writes", "sold", "window.__sold",
"toast", "document.querySelector('.juneau-view-bulk-toast')?.textContent")),
// The tab's <object> asks for the photo before there is one, so its 404 load is expected.
- Map.of("name", "upload", "path", "/console/pets/1/photo", "allowFailedLoads", true,
+ Map.of("name", "upload", "waitUntil", "networkidle", "path", "/console/pets/1/photo", "allowFailedLoads", true,
"actions", List.of(
eval(UPLOAD_PNG),
wait("performance.getEntriesByType('navigation')[0].type === 'reload'")),
"queries", Map.of("status", "document.querySelector('form[data-petstore-upload] [role=status]')?.textContent ?? null",
"marker", "sessionStorage.getItem('uploaded')")),
- Map.of("name", "uploadNoToken", "path", "/console/pets/1/photo", "allowFailedLoads", true,
+ Map.of("name", "uploadNoToken", "waitUntil", "networkidle", "path", "/console/pets/1/photo", "allowFailedLoads", true,
"actions", List.of(
eval(UPLOAD_NO_TOKEN),
wait("document.querySelector('form[data-petstore-upload] [role=status]')")),
"queries", Map.of("status", "document.querySelector('form[data-petstore-upload] [role=status]')?.textContent ?? null",
"navigation", "performance.getEntriesByType('navigation')[0].type")),
- Map.of("name", "uploadFails", "path", "/console/pets/1/photo", "allowFailedLoads", true,
+ Map.of("name", "uploadFails", "waitUntil", "networkidle", "path", "/console/pets/1/photo", "allowFailedLoads", true,
"actions", List.of(
eval(UPLOAD_BAD),
wait("document.querySelector('form[data-petstore-upload] [role=status]')")),
diff --git a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreCoverage_Test.java b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreCoverage_Test.java
index 202de0262e..2d7dcd9e0b 100644
--- a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreCoverage_Test.java
+++ b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreCoverage_Test.java
@@ -52,7 +52,7 @@ class PetstoreCoverage_Test extends TestBase {
PetstoreSecure_BrowserTest.class,
PetstoreThemes_BrowserTest.class,
PetstoreAsyncJob_BrowserTest.class,
- PetstoreRowJob_BrowserTest.class,
+ PetstoreRowJob_BrowserTest.class, PetstoreRowJobCancel_BrowserTest.class,
PetstoreDatatable_BrowserTest.class,
PetstoreTerminal_BrowserTest.class,
PetstorePetDetail_BrowserTest.class,
diff --git a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreHarness_BrowserTest.java b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreHarness_BrowserTest.java
index 12171b45c5..b98d640bc1 100644
--- a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreHarness_BrowserTest.java
+++ b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreHarness_BrowserTest.java
@@ -18,6 +18,7 @@ package org.apache.juneau.petstore.console.browser;
import static org.apache.juneau.petstore.console.browser.PetstoreBrowser.*;
import static org.apache.juneau.test.bct.BctAssertions.*;
+import static org.junit.jupiter.api.Assertions.assertTrue;
import java.util.*;
@@ -43,7 +44,7 @@ class PetstoreHarness_BrowserTest extends TestBase {
static void probe() throws Exception {
var base = PetstoreTestServer.awaitReady(server);
report = run("harness", base, List.of(
- Map.of("name", "store", "path", "/console/store",
+ Map.of("name", "store", "waitUntil", "networkidle", "path", "/console/store",
"queries", Map.of("title", "document.title", "pages", "document.querySelectorAll('#juneau-page').length")),
Map.of("name", "actions", "path", "/console/about", "viewport", List.of(800, 600), "waitUntil", "load",
"initScript", "window.__init = (window.__init || 0) + 1;",
@@ -72,4 +73,8 @@ class PetstoreHarness_BrowserTest extends TestBase {
assertString("?y=2", query(c, "search"));
assertString("1", query(c, "kept")); // reload, goto and follow stay in the case's one context
}
+
+ @Test void a03_everyCaseReportsItsWallTime() {
+ assertTrue(((Number) assertClean(report, "store").get("ms")).longValue() > 0);
+ }
}
diff --git a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreNav_BrowserTest.java b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreNav_BrowserTest.java
index ec198bec8c..b5ba189723 100644
--- a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreNav_BrowserTest.java
+++ b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreNav_BrowserTest.java
@@ -45,9 +45,9 @@ class PetstoreNav_BrowserTest extends TestBase {
static void probe() throws Exception {
var base = PetstoreTestServer.awaitReady(server);
report = run("nav", base, List.of(
- Map.of("name", "flavor-html", "path", "/console/dev/flavors/html", "queries", Map.of("current", CURRENT, "rows", CHILD_ROWS)),
- Map.of("name", "about", "path", "/console/about", "queries", Map.of("current", CURRENT, "rows", CHILD_ROWS)),
- Map.of("name", "click-dev", "path", "/console/store",
+ Map.of("name", "flavor-html", "waitUntil", "networkidle", "path", "/console/dev/flavors/html", "queries", Map.of("current", CURRENT, "rows", CHILD_ROWS)),
+ Map.of("name", "about", "waitUntil", "networkidle", "path", "/console/about", "queries", Map.of("current", CURRENT, "rows", CHILD_ROWS)),
+ Map.of("name", "click-dev", "waitUntil", "networkidle", "path", "/console/store",
"actions", List.of(
Map.of("click", "[data-juneau-nav-id=\"dev\"]"),
Map.of("waitFor", "location.pathname.startsWith('/console/dev/')")),
diff --git a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstorePetDetail_BrowserTest.java b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstorePetDetail_BrowserTest.java
index 7b52e9af10..15ce039294 100644
--- a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstorePetDetail_BrowserTest.java
+++ b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstorePetDetail_BrowserTest.java
@@ -93,8 +93,8 @@ class PetstorePetDetail_BrowserTest extends TestBase {
Map.of("click", "#pet-body .juneau-view-action-trigger"),
Map.of("waitFor", "document.querySelector('.juneau-view-action-item')")),
"queries", Map.of("items", MENU_ITEMS)),
- Map.of("name", "order", "path", "/console/pets/" + petId + "/orders/" + orderId, "queries", Map.of("current", CURRENT, "rows", CHILD_ROWS)),
- Map.of("name", "upload-error", "path", "/console/pets/1/photo", "allowFailedLoads", true,
+ Map.of("name", "order", "waitUntil", "networkidle", "path", "/console/pets/" + petId + "/orders/" + orderId, "queries", Map.of("current", CURRENT, "rows", CHILD_ROWS)),
+ Map.of("name", "upload-error", "waitUntil", "networkidle", "path", "/console/pets/1/photo", "allowFailedLoads", true,
"actions", List.of(
Map.of("evaluate", UPLOAD_TEXT),
Map.of("waitFor", "(" + UPLOAD_STATUS + " || '').startsWith('Upload failed')")),
diff --git a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreRowJobCancel_BrowserTest.java b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreRowJobCancel_BrowserTest.java
new file mode 100644
index 0000000000..0bce46259f
--- /dev/null
+++ b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreRowJobCancel_BrowserTest.java
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.juneau.petstore.console.browser;
+
+import static org.apache.juneau.petstore.console.browser.PetstoreBrowser.*;
+import static org.apache.juneau.test.bct.BctAssertions.*;
+import static org.junit.jupiter.api.Assertions.*;
+
+import java.util.*;
+
+import org.apache.juneau.*;
+import org.apache.juneau.microservice.*;
+import org.junit.jupiter.api.*;
+import org.junit.jupiter.api.condition.*;
+import org.junit.jupiter.api.extension.*;
+
+/**
+ * Cancelling a row job in the real browser: the species table's job-log detail region cancels a running restock
+ * with its submit button, and a re-attached view (after a collapse and re-expand) cancels it too. The server holds
+ * a species restock for 7 s after its effect, so a cancel in either scenario lands as "after effect".
+ */
+@Covers(value="async-job", pages="/console/ops/jobs")
+@HookCovers("jobs")
+@EnabledIfSystemProperty(named=PetstoreBrowser.GATE, matches="true", disabledReason=PetstoreBrowser.DISABLED)
+class PetstoreRowJobCancel_BrowserTest extends TestBase {
+
+ @RegisterExtension
+ static MicroserviceTestFixture server = PetstoreTestServer.fixture(PetstoreTestServer.CancelHoldTimings.class);
+
+ private static Map<String,Map<String,Object>> report;
+
+ /** JS: the species table's DOG row. */
+ private static final String DOG_ROW = "[...document.querySelectorAll('#species-body tbody tr')].find(tr => [...tr.cells].some(td => td.textContent === 'DOG'))";
+
+ /** The DOG panel's job-log region. */
+ private static final String R = "#species-body .juneau-view-detail-panel .jc-joblog-region";
+
+ private static Map<String,Object> wait(String js, int ms) {
+ return Map.of("waitFor", js, "timeoutMs", ms);
+ }
+
+ private static Map<String,Object> phase(String p, int ms) {
+ return wait("document.querySelector('" + R + "[data-phase=" + p + "]')", ms);
+ }
+
+ private static Map<String,Object> toggleDog() {
+ return Map.of("evaluate", DOG_ROW + ".querySelector('.juneau-view-detail-toggle').click()");
+ }
+
+ @BeforeAll
+ static void probe() throws Exception {
+ var base = PetstoreTestServer.awaitReady(server);
+ report = run("row-job-cancel", base, List.of(
+ // The running form's submit button is its cancel; the second click lands in the restock's hold.
+ Map.of("name", "cancel", "path", "/console/ops/jobs",
+ "actions", List.of(
+ wait(DOG_ROW, 8000), toggleDog(), phase("idle", 8000),
+ Map.of("click", R + " .jc-joblog-submit"),
+ phase("running", 8000),
+ Map.of("click", R + " .jc-joblog-submit"),
+ phase("done", 15000)),
+ "queries", Map.of("banner", "document.querySelector('" + R + " .jc-joblog-banner')?.textContent ?? null")),
+ // One collapse and re-expand (the job's two stream slots); the re-attached view's button is the cancel.
+ Map.of("name", "reattachCancel", "path", "/console/ops/jobs",
+ "actions", List.of(
+ wait(DOG_ROW, 8000), toggleDog(), phase("idle", 8000),
+ Map.of("click", R + " .jc-joblog-submit"),
+ phase("running", 8000),
+ toggleDog(), wait("!document.querySelector('" + R + "')", 4000),
+ toggleDog(), phase("running", 8000),
+ Map.of("evaluate", "window.__btn = (b => ({text: b.textContent, hidden: b.hidden, disabled: b.disabled}))(document.querySelector('" + R + " .jc-joblog-submit'))"),
+ Map.of("click", R + " .jc-joblog-submit"),
+ phase("done", 20000),
+ Map.of("evaluate", "window.__active = JuneauViews.jobs.active(" + DOG_ROW + ") == null")),
+ "queries", Map.of("btn", "window.__btn",
+ "banner", "document.querySelector('" + R + " .jc-joblog-banner')?.textContent ?? null",
+ "marker", DOG_ROW + ".hasAttribute('data-juneau-job')",
+ "activeNull", "window.__active"))));
+ }
+
+ @Test void a01_cancelEndsTheRowJob() {
+ assertString("Cancelled after the job had already made changes.", query(assertClean(report, "cancel"), "banner"));
+ }
+
+ @Test void a02_aReattachedPanelCancelsItsJob() {
+ var c = assertClean(report, "reattachCancel");
+ assertBean(query(c, "btn"), "text,hidden,disabled", "Cancel,false,false");
+ assertString("Cancelled after the job had already made changes.", query(c, "banner"));
+ assertEquals(false, query(c, "marker"));
+ assertEquals(true, query(c, "activeNull"));
+ }
+}
diff --git a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreRowJob_BrowserTest.java b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreRowJob_BrowserTest.java
index 8b30f95e0d..0a4b4983cf 100644
--- a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreRowJob_BrowserTest.java
+++ b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreRowJob_BrowserTest.java
@@ -24,7 +24,6 @@ import java.util.*;
import org.apache.juneau.*;
import org.apache.juneau.microservice.*;
-import org.apache.juneau.petstore.console.ops.*;
import org.junit.jupiter.api.*;
import org.junit.jupiter.api.condition.*;
import org.junit.jupiter.api.extension.*;
@@ -32,8 +31,9 @@ import org.junit.jupiter.api.extension.*;
/**
* Row jobs in the real browser: the species table's job-log detail region keys, submits and streams a restock for
* its own row, holds the poll (the pill says Paused) while the open panel's job runs, writes the result into the
- * panel's {@code available} slot, re-attaches after a collapse, and cancels. A key for one row is refused for another. A
- * groom run's "Groom again" button runs through {@code JuneauViews.jobs} and starts a new run.
+ * panel's {@code available} slot, and re-attaches after a collapse. A key for one row is refused for another. A
+ * groom run's "Groom again" button runs through {@code JuneauViews.jobs} and starts a new run. The server's species
+ * hold is 2 s; cancelling a row job is covered by {@link PetstoreRowJobCancel_BrowserTest}.
*/
@Covers(value="async-job", pages="/console/ops/jobs")
@HookCovers("jobs")
@@ -41,7 +41,7 @@ import org.junit.jupiter.api.extension.*;
class PetstoreRowJob_BrowserTest extends TestBase {
@RegisterExtension
- static MicroserviceTestFixture server = PetstoreTestServer.fixture();
+ static MicroserviceTestFixture server = PetstoreTestServer.fixture(PetstoreTestServer.ShortHoldTimings.class);
private static Map<String,Map<String,Object>> report;
@@ -79,23 +79,9 @@ class PetstoreRowJob_BrowserTest extends TestBase {
return Map.of("evaluate", DOG_ROW + ".querySelector('.juneau-view-detail-toggle').click()");
}
- /** Widens every species restock's post-effect hold, so a cancel in any scenario lands in it. */
- private static final String HOLD_MS = "7000";
-
@BeforeAll
static void probe() throws Exception {
- var base = PetstoreTestServer.awaitReady(server);
- System.setProperty(JobsRest.SPECIES_HOLD_PROPERTY, HOLD_MS);
- try {
- report = runBatch(base);
- } finally {
- System.clearProperty(JobsRest.SPECIES_HOLD_PROPERTY);
- }
- }
-
- @AfterAll
- static void restoreHold() {
- System.clearProperty(JobsRest.SPECIES_HOLD_PROPERTY);
+ report = runBatch(PetstoreTestServer.awaitReady(server));
}
private static Map<String,Map<String,Object>> runBatch(String base) throws Exception {
@@ -133,31 +119,6 @@ class PetstoreRowJob_BrowserTest extends TestBase {
toggleDog(), wait("!document.querySelector('" + R + "')", 4000),
toggleDog(), phase("done", 15000)),
"queries", Map.of("lines", "[...document.querySelectorAll('" + R + " .jc-joblog-line')].map(l => l.textContent)")),
- // The running form's submit button is its cancel; the second click lands in the restock's hold.
- Map.of("name", "cancel", "path", "/console/ops/jobs",
- "actions", List.of(
- wait(DOG_ROW, 8000), toggleDog(), phase("idle", 8000),
- Map.of("click", R + " .jc-joblog-submit"),
- phase("running", 8000),
- Map.of("click", R + " .jc-joblog-submit"),
- phase("done", 15000)),
- "queries", Map.of("banner", "document.querySelector('" + R + " .jc-joblog-banner')?.textContent ?? null")),
- // One collapse and re-expand (the job's two stream slots); the re-attached view's button is the cancel.
- Map.of("name", "reattachCancel", "path", "/console/ops/jobs",
- "actions", List.of(
- wait(DOG_ROW, 8000), toggleDog(), phase("idle", 8000),
- Map.of("click", R + " .jc-joblog-submit"),
- phase("running", 8000),
- toggleDog(), wait("!document.querySelector('" + R + "')", 4000),
- toggleDog(), phase("running", 8000),
- Map.of("evaluate", "window.__btn = (b => ({text: b.textContent, hidden: b.hidden, disabled: b.disabled}))(document.querySelector('" + R + " .jc-joblog-submit'))"),
- Map.of("click", R + " .jc-joblog-submit"),
- phase("done", 20000),
- Map.of("evaluate", "window.__active = JuneauViews.jobs.active(" + DOG_ROW + ") == null")),
- "queries", Map.of("btn", "window.__btn",
- "banner", "document.querySelector('" + R + " .jc-joblog-banner')?.textContent ?? null",
- "marker", DOG_ROW + ".hasAttribute('data-juneau-job')",
- "activeNull", "window.__active")),
Map.of("name", "regroom", "path", "/console/ops/jobs",
"actions", List.of(
wait("document.querySelector('#groom-runs-body tbody tr .juneau-view-detail-toggle')", 8000),
@@ -179,8 +140,8 @@ class PetstoreRowJob_BrowserTest extends TestBase {
assertContains("Restocking 3 pets", String.valueOf(query(c, "lines")));
assertString(String.valueOf(before + 3), query(c, "slot"));
assertEquals(false, query(c, "markerAfter"));
- // The speciesHoldMs knob was read for this request: the default hold alone would finish in about a second.
- assertTrue(((Number)query(c, "ms")).intValue() >= 6000, () -> String.valueOf(query(c, "ms")));
+ // The species hold bean was read for this request: the default hold alone would finish in about a second.
+ assertTrue(((Number)query(c, "ms")).intValue() >= 1500, () -> String.valueOf(query(c, "ms")));
}
@Test void a02_aKeyForOneRowIsRefusedForAnother() {
@@ -191,18 +152,6 @@ class PetstoreRowJob_BrowserTest extends TestBase {
assertContains("Restocking 1 pets", String.valueOf(query(assertClean(report, "reattach"), "lines")));
}
- @Test void a05_cancelEndsTheRowJob() {
- assertString("Cancelled after the job had already made changes.", query(assertClean(report, "cancel"), "banner"));
- }
-
- @Test void a06_aReattachedPanelCancelsItsJob() {
- var c = assertClean(report, "reattachCancel");
- assertBean(query(c, "btn"), "text,hidden,disabled", "Cancel,false,false");
- assertString("Cancelled after the job had already made changes.", query(c, "banner"));
- assertEquals(false, query(c, "marker"));
- assertEquals(true, query(c, "activeNull"));
- }
-
@Test void a04_groomAgainStartsANewRun() {
var c = assertClean(report, "regroom");
assertString("Groom again", query(c, "label"));
diff --git a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreSecure_BrowserTest.java b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreSecure_BrowserTest.java
index 0c9a94a459..af5f997273 100644
--- a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreSecure_BrowserTest.java
+++ b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreSecure_BrowserTest.java
@@ -39,7 +39,7 @@ class PetstoreSecure_BrowserTest extends TestBase {
private static Map<String,Map<String,Object>> report;
private static Map<String,Object> click(String token) {
- return Map.of("name", token, "path", "/console/dev/secure", "allowFailedLoads", true,
+ return Map.of("name", token, "waitUntil", "networkidle", "path", "/console/dev/secure", "allowFailedLoads", true,
"actions", List.of(
Map.of("click", "[data-secure-call=\"" + token + "\"]"),
Map.of("waitFor", "document.getElementById('secure-result').getAttribute('data-status')")),
diff --git a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreTerminal_BrowserTest.java b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreTerminal_BrowserTest.java
index 75538876cd..f77420b045 100644
--- a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreTerminal_BrowserTest.java
+++ b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreTerminal_BrowserTest.java
@@ -64,9 +64,9 @@ class PetstoreTerminal_BrowserTest extends TestBase {
report = run("terminal", base, List.of(
Map.of("name", "jobs", "path", "/console/ops/jobs?run=" + runId,
"actions", List.of(
- // The run takes about 35 s at the default period; the terminals follow it to the last lines.
- Map.of("waitFor", ROWS + ".includes('Trimmed nail 12 of 12')", "timeoutMs", 60000),
- Map.of("waitFor", FILE_ROWS + ".includes('[51/52] step done')", "timeoutMs", 60000)),
+ // The run takes about 8 s at the test server's 150 ms period; the terminals follow it to the last lines.
+ Map.of("waitFor", ROWS + ".includes('Trimmed nail 12 of 12')", "timeoutMs", 30000),
+ Map.of("waitFor", FILE_ROWS + ".includes('[51/52] step done')", "timeoutMs", 30000)),
"queries", Map.of(
"cards", CARDS,
"rows", ROWS,
diff --git a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreTestServer.java b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreTestServer.java
index 703fe984c9..d08106824c 100644
--- a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreTestServer.java
+++ b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreTestServer.java
@@ -67,17 +67,50 @@ public final class PetstoreTestServer {
return PetstoreSeed.create().populate(new PetStore(PetstoreSeed.DEFAULT_CLOCK));
}
+ /** @return A 150 ms groom step, so a full groom run takes about 8 s. */
+ @Bean public JobsRest.GroomPeriod groomPeriod() {
+ return new JobsRest.GroomPeriod(Duration.ofMillis(150));
+ }
+
/**
- * @return A 12 s jobs hard timeout, so a browser test can watch a slow restock time out. It leaves room on both
- * sides of the async-job cases: a 6 s hold after the effect ends well before it, and a 10 s wait plus a 10 s
- * hold runs well past it.
+ * @return A 5 s jobs hard timeout, so a browser test can watch a slow restock time out. A species hold must
+ * end well before it (see {@link ShortHoldHost}), or the job is reaped as a hard timeout.
*/
@Bean public JobsRest.HardTimeout jobsHardTimeout() {
+ return new JobsRest.HardTimeout(Duration.ofSeconds(5));
+ }
+ }
+
+ /** {@link BrowserHost} plus a 2 s species-row hold: long enough to see a hold, short enough to wait out. */
+ public static class ShortHoldHost extends BrowserHost {
+ private static final long serialVersionUID = 1L;
+
+ /** @return The species-row hold. */
+ @Bean public JobsRest.SpeciesHold speciesHold() {
+ return new JobsRest.SpeciesHold(Duration.ofSeconds(2));
+ }
+ }
+
+ /**
+ * {@link BrowserHost} plus a 7 s species-row hold, so a cancel in any scenario lands in it, and a 12 s hard timeout:
+ * a hold must end well before the timeout, or the job is reaped as a hard timeout instead of being cancelled after
+ * its effect.
+ */
+ public static class CancelHoldHost extends BrowserHost {
+ private static final long serialVersionUID = 1L;
+
+ /** @return The species-row hold. */
+ @Bean public JobsRest.SpeciesHold speciesHold() {
+ return new JobsRest.SpeciesHold(Duration.ofSeconds(7));
+ }
+
+ /** @return A 12 s jobs hard timeout, longer than the hold. */
+ @Override @Bean public JobsRest.HardTimeout jobsHardTimeout() {
return new JobsRest.HardTimeout(Duration.ofSeconds(12));
}
}
- /** Contributes {@link BrowserHost} and the console and API CSRF boundaries the runners install to the fixture's Jetty. */
+ /** Contributes the console and API CSRF boundaries the runners install to the fixture's Jetty; a timings class adds the host. */
@Configuration
public static class BrowserConfig {
// Shared by both boundary beans, as in the runners.
@@ -100,18 +133,52 @@ public final class PetstoreTestServer {
.order(ServletFilterRegistration.HIGHEST_PRECEDENCE + 10)
.build();
}
+ }
+
+ private PetstoreTestServer() {}
- /** @return The root servlet. */
+ /**
+ * The root servlet of a variant; the variants differ only in the host's job timings. A fixture takes configuration
+ * classes, not parameters, and the timing beans must sit on the host (a bean on a configuration class does not reach
+ * the console resources), so each variant is a class that mounts its own host.
+ */
+ @Configuration
+ public static class DefaultTimings {
+ /** @return The root servlet, with {@link BrowserHost}'s timings. */
@Bean public Servlet root() {
return new BrowserHost();
}
}
- private PetstoreTestServer() {}
+ /** The variant whose species rows hold 2 s after their effect. */
+ @Configuration
+ public static class ShortHoldTimings {
+ /** @return The root servlet, a {@link ShortHoldHost}. */
+ @Bean public Servlet root() {
+ return new ShortHoldHost();
+ }
+ }
- /** @return A new fixture; register it with {@code @RegisterExtension} on a static field. */
+ /** The variant whose species rows hold 7 s after their effect, for the cancel cases. */
+ @Configuration
+ public static class CancelHoldTimings {
+ /** @return The root servlet, a {@link CancelHoldHost}. */
+ @Bean public Servlet root() {
+ return new CancelHoldHost();
+ }
+ }
+
+ /** @return A new fixture with {@link DefaultTimings}; register it with {@code @RegisterExtension} on a static field. */
public static MicroserviceTestFixture fixture() {
- return MicroserviceTestFixture.create().configurations(BrowserConfig.class);
+ return fixture(DefaultTimings.class);
+ }
+
+ /**
+ * @param timings A timings configuration class, e.g. {@link ShortHoldTimings}.
+ * @return A new fixture; register it with {@code @RegisterExtension} on a static field.
+ */
+ public static MicroserviceTestFixture fixture(Class<?> timings) {
+ return MicroserviceTestFixture.create().configurations(BrowserConfig.class, timings);
}
/**
diff --git a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreThemes_BrowserTest.java b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreThemes_BrowserTest.java
index e20ed34109..d5efd4fb49 100644
--- a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreThemes_BrowserTest.java
+++ b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreThemes_BrowserTest.java
@@ -56,7 +56,7 @@ class PetstoreThemes_BrowserTest extends TestBase {
var base = PetstoreTestServer.awaitReady(server);
var cases = new ArrayList<Map<String,Object>>();
for (var t : concat(STOCK, "custom"))
- cases.add(Map.of("name", t, "path", "/console/dev/themes?theme=" + t,
+ cases.add(Map.of("name", t, "waitUntil", "networkidle", "path", "/console/dev/themes?theme=" + t,
"actions", List.of(Map.of("screenshot", "theme-" + t + ".png")),
"queries", Map.of("tokens", "custom".equals(t) ? CUSTOM_TOKENS : TOKENS)));
cases.add(Map.of("name", "scheme-dark", "path", "/console/store",
diff --git a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreTypedConfirm_BrowserTest.java b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreTypedConfirm_BrowserTest.java
index c772f47450..92fafa518b 100644
--- a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreTypedConfirm_BrowserTest.java
+++ b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreTypedConfirm_BrowserTest.java
@@ -114,7 +114,7 @@ class PetstoreTypedConfirm_BrowserTest extends TestBase {
theCase("escape", "{rows: [{id: 7}]}", List.of(
wait("document.querySelector('" + DLG + "')"),
Map.of("press", List.of(INPUT, "Escape")), wait(RES_FALSE))),
- Map.of("name", "ordersDelete", "path", "/console/orders", "capture", "/console/orders/",
+ Map.of("name", "ordersDelete", "waitUntil", "networkidle", "path", "/console/orders", "capture", "/console/orders/",
"actions", List.of(
Map.of("evaluate", SNAP),
wait("document.querySelector('#orders-body .juneau-view-action-trigger')"),
diff --git a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreVisibility_BrowserTest.java b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreVisibility_BrowserTest.java
index 5bcce6e766..a4071d1dd9 100644
--- a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreVisibility_BrowserTest.java
+++ b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/browser/PetstoreVisibility_BrowserTest.java
@@ -84,7 +84,7 @@ class PetstoreVisibility_BrowserTest extends TestBase {
wait("document.querySelector('#pet-body tbody')?.textContent.toUpperCase().includes('SOLD')")),
"queries", Map.of("before", "window.__before", "pencils", PENCILS,
"trigger", "(() => { const t = document.querySelector('" + TRIGGER + "'); return !!t && !t.hidden && !t.disabled; })()")),
- Map.of("name", "photo", "path", "/console/pets/1/photo", "allowFailedLoads", true,
+ Map.of("name", "photo", "waitUntil", "networkidle", "path", "/console/pets/1/photo", "allowFailedLoads", true,
"queries", Map.of("viewer", "!!document.querySelector('" + UPLOAD + "')",
"admin", UPLOAD_FOR + "('petstore-admin')", "user", UPLOAD_FOR + "('petstore-user')"))));
}
diff --git a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/ops/JobsRest_Test.java b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/ops/JobsRest_Test.java
index f0800ea81c..08954682d0 100644
--- a/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/ops/JobsRest_Test.java
+++ b/juneau-petstore/juneau-petstore-core/src/test/java/org/apache/juneau/petstore/console/ops/JobsRest_Test.java
@@ -549,4 +549,95 @@ class JobsRest_Test extends TestBase {
status = c.get("/console/ops/jobs/groom-runs/x/key").run().getStatusCode();
assertTrue(status == 404 || status == 405, "GET on the key endpoint must not be served; got " + status);
}
+
+ //-----------------------------------------------------------------------------------------------------------------
+ // h - per-host timing beans: the groom step period and the species-row hold
+ //-----------------------------------------------------------------------------------------------------------------
+
+ /** A host whose groom runs step every millisecond. */
+ public static class FastGroomHost extends Host {
+ private static final long serialVersionUID = 1L;
+
+ /** @return A 1 ms groom step. */
+ @Bean public JobsRest.GroomPeriod groomPeriod() {
+ return new JobsRest.GroomPeriod(Duration.ofMillis(1));
+ }
+ }
+
+ /** A host whose species rows hold nothing after their effect. */
+ public static class NoHoldHost extends Host {
+ private static final long serialVersionUID = 1L;
+
+ /** @return A zero hold. */
+ @Bean public JobsRest.SpeciesHold speciesHold() {
+ return new JobsRest.SpeciesHold(Duration.ZERO);
+ }
+ }
+
+ /** A host no other test has touched, so its groom runs are created after a test sets {@link GroomRuns#PERIOD_PROPERTY}. */
+ public static class UntouchedHost extends Host {
+ private static final long serialVersionUID = 1L;
+ }
+
+ private static MockRestClient client(Class<?> host) {
+ return MockRestClient.create(host).noTrace().ignoreErrors().header("Accept", "application/json").build();
+ }
+
+ private static String groomState(MockRestClient c, String id) throws Exception {
+ return new JsonMap(c.get("/console/ops/jobs/groom-runs/" + id).run().assertStatus(200).getContent().asString()).getMap("fields").getString("state");
+ }
+
+ private static void awaitGroomDone(MockRestClient c) throws Exception {
+ var id = groom(c, "{\"pet\":\"Rex\"}").getMap("row").getString("id");
+ var deadline = System.nanoTime() + Duration.ofSeconds(5).toNanos();
+ while (! "DONE".equals(groomState(c, id)) && System.nanoTime() < deadline)
+ Thread.sleep(20);
+ assertString("DONE", groomState(c, id));
+ }
+
+ /** Milliseconds from submitting a DOG row restock until its job settles. */
+ private static long speciesRestockMillis(MockRestClient c) throws Exception {
+ var k = key(c, "/console/ops/jobs/species/DOG/key").getString("idempotencyKey");
+ var t0 = System.nanoTime();
+ var ref = new JsonMap(restockSpecies(c, "DOG", envelope("restock-species", "DOG", k, "{\"count\":1}"), 202));
+ assertString("DONE", await(c, ref.getString("jobId")).getString("state"));
+ return TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - t0);
+ }
+
+ @Test void h01_aGroomPeriodBeanSpeedsTheRun() throws Exception {
+ awaitGroomDone(client(FastGroomHost.class));
+ }
+
+ @Test void h02_aSpeciesHoldBeanOfZeroSkipsTheHold() throws Exception {
+ assertTrue(speciesRestockMillis(client(NoHoldHost.class)) < 900);
+ }
+
+ @Test void h03_withoutAGroomPeriodBeanTheRunKeepsItsDefaultPace() throws Exception {
+ var c = client();
+ var id = groom(c, "{\"pet\":\"Rex\"}").getMap("row").getString("id");
+ Thread.sleep(1200);
+ assertString("RUNNING", groomState(c, id));
+ }
+
+ @Test void h04_theGroomPeriodPropertyIsStillHonoredWithoutABean() throws Exception {
+ System.setProperty(GroomRuns.PERIOD_PROPERTY, "1");
+ try {
+ awaitGroomDone(client(UntouchedHost.class));
+ } finally {
+ System.clearProperty(GroomRuns.PERIOD_PROPERTY);
+ }
+ }
+
+ @Test void h05_withoutASpeciesHoldBeanTheDefaultHoldApplies() throws Exception {
+ assertTrue(speciesRestockMillis(client()) >= JobsRest.DEFAULT_HOLD_MS - 100);
+ }
+
+ @Test void h06_theSpeciesHoldPropertyIsStillHonoredWithoutABean() throws Exception {
+ System.setProperty(JobsRest.SPECIES_HOLD_PROPERTY, "0");
+ try {
+ assertTrue(speciesRestockMillis(client()) < 900);
+ } finally {
+ System.clearProperty(JobsRest.SPECIES_HOLD_PROPERTY);
+ }
+ }
}
diff --git a/juneau-petstore/juneau-petstore-core/src/test/js/petstore-browser.cjs b/juneau-petstore/juneau-petstore-core/src/test/js/petstore-browser.cjs
index 809d426f38..15a1f2ad62 100644
--- a/juneau-petstore/juneau-petstore-core/src/test/js/petstore-browser.cjs
+++ b/juneau-petstore/juneau-petstore-core/src/test/js/petstore-browser.cjs
@@ -20,7 +20,7 @@
*
* Input: {baseUrl, snapshots, deviceScaleFactor?, cases:[{name, path, viewport?, deviceScaleFactor?, waitUntil?,
* initScript?, actions?, queries?, fetches?, capture?, allowFailedLoads?}]}
- * Output (stdout): {cases:{<name>:{status, jsFailures, deprecations, queries, fetches, requests, downloads, timeouts}}}
+ * Output (stdout): {cases:{<name>:{status, jsFailures, deprecations, queries, fetches, requests, downloads, timeouts, ms}}}
*
* Each action is one of:
* {click: selector} {fill: [selector, text]} {press: [selector, key]}
@@ -37,8 +37,9 @@
* reload waits for the case's waitUntil, as the first load does. back and forward wait only for the navigation to
* commit: follow them with a waitFor on what the page should then show.
*
- * waitUntil is the navigation's Playwright wait state, networkidle by default. A case on a page that polls can pass
- * 'load' and start its actions with a waitFor, so it never depends on the poll leaving a quiet gap.
+ * waitUntil is the navigation's Playwright wait state, load by default: a case starts its actions with a waitFor on what
+ * the page should show, so it never depends on the network going quiet (a page that polls never does). A case that
+ * asserts straight after the navigation, with no waitFor, passes 'networkidle' so the page has settled first.
*
* viewport is [width, height] (default [1280, 900]). deviceScaleFactor (case, else input, else 1) is the device pixel
* ratio, so 2 gives screenshots at twice the viewport's pixel size. Screenshots are taken with animations disabled
@@ -52,6 +53,8 @@ const fs = require('node:fs');
const path = require('node:path');
const { chromium } = require('playwright');
+const DEFAULT_WAIT_UNTIL = 'load';
+
(async () => {
const input = JSON.parse(fs.readFileSync(process.argv[2], 'utf8'));
const browser = await chromium.launch();
@@ -70,6 +73,7 @@ const { chromium } = require('playwright');
});
async function runCase(browser, input, c) {
+ const t0 = Date.now();
const vp = Array.isArray(c.viewport) ? c.viewport : [1280, 900];
const context = await browser.newContext({ viewport: { width: vp[0], height: vp[1] }, acceptDownloads: true,
deviceScaleFactor: c.deviceScaleFactor || input.deviceScaleFactor || 1 });
@@ -90,7 +94,7 @@ async function runCase(browser, input, c) {
});
}
try {
- const resp = await page.goto(input.baseUrl + c.path, { waitUntil: c.waitUntil || 'networkidle' });
+ const resp = await page.goto(input.baseUrl + c.path, { waitUntil: c.waitUntil || DEFAULT_WAIT_UNTIL });
r.status = resp ? resp.status() : 0;
for (const a of c.actions || []) await act(page, input, c, a, r);
for (const [k, expr] of Object.entries(c.queries || {})) {
@@ -106,6 +110,7 @@ async function runCase(browser, input, c) {
r.jsFailures.push('harness: ' + String(e?.message || e));
} finally {
await context.close();
+ r.ms = Date.now() - t0;
}
return r;
}
@@ -114,7 +119,7 @@ async function act(page, input, c, a, r) { // NOSONAR javascript:S3776 - one fla
if (a.click) return page.click(a.click);
if (a.fill) return page.fill(a.fill[0], a.fill[1]);
if (a.press) return page.press(a.press[0], a.press[1]);
- const until = { waitUntil: c.waitUntil || 'networkidle' };
+ const until = { waitUntil: c.waitUntil || DEFAULT_WAIT_UNTIL };
if (a.reload) return page.reload(until);
if (a.goto) return page.goto(input.baseUrl + a.goto, until);
if (a.follow) return page.goto(String(await page.evaluate(a.follow)), until);
diff --git a/juneau-rest/juneau-rest-server-mcp-v20260728/src/main/java/org/apache/juneau/rest/server/mcp/v20260728/McpMrtrConfig.java b/juneau-rest/juneau-rest-server-mcp-v20260728/src/main/java/org/apache/juneau/rest/server/mcp/v20260728/McpMrtrConfig.java
index b1d6838b59..87caea3412 100644
--- a/juneau-rest/juneau-rest-server-mcp-v20260728/src/main/java/org/apache/juneau/rest/server/mcp/v20260728/McpMrtrConfig.java
+++ b/juneau-rest/juneau-rest-server-mcp-v20260728/src/main/java/org/apache/juneau/rest/server/mcp/v20260728/McpMrtrConfig.java
@@ -16,6 +16,8 @@
*/
package org.apache.juneau.rest.server.mcp.v20260728;
+import java.time.*;
+
import org.apache.juneau.commons.concurrent.*;
/**
@@ -35,8 +37,9 @@ import org.apache.juneau.commons.concurrent.*;
*
* <p>
* Defaults: {@link AeadRequestStateCodec} (a fresh instance per {@link McpMrtrConfig}), a 5-minute
- * {@code requestState} TTL, a max-rounds cap of 10, and no {@link ReplayCache} (replay rejection is opt-in; see
- * {@link #setReplayCache(ReplayCache)}). Operators inject a shared/rotating-key {@link RequestStateCodec} here
+ * {@code requestState} TTL, a max-rounds cap of 10, no {@link ReplayCache} (replay rejection is opt-in; see
+ * {@link #setReplayCache(ReplayCache)}), and a {@link Clock#systemUTC()} clock for stamping and checking token
+ * expiry (see {@link #setClock(Clock)}). Operators inject a shared/rotating-key {@link RequestStateCodec} here
* for multi-instance or restart-durable resumption.
*
* <h5 class='section'>Trust model:</h5>
@@ -66,6 +69,7 @@ public class McpMrtrConfig {
private long ttlMs = DEFAULT_TTL_MS;
private int maxRounds = DEFAULT_MAX_ROUNDS;
private ReplayCache replayCache;
+ private Clock clock = Clock.systemUTC();
/**
* The codec used to seal/unseal {@code requestState} tokens.
@@ -191,4 +195,32 @@ public class McpMrtrConfig {
replayCache = value;
return this;
}
+
+ /**
+ * The clock used to stamp a new {@code requestState}'s expiry and to check an echoed one against it.
+ *
+ * @return The clock. Never {@code null}.
+ */
+ public Clock getClock() {
+ return clock;
+ }
+
+ /**
+ * Sets the clock.
+ *
+ * <p>
+ * Defaults to {@link Clock#systemUTC()}. Tests can supply a {@link Clock#fixed(Instant, ZoneId) fixed} or
+ * manually-advanced clock to drive {@code requestState} expiry deterministically instead of sleeping past the
+ * {@link #getTtlMs() TTL}.
+ *
+ * @param value The new value. Must not be {@code null}.
+ * @return This object.
+ * @throws IllegalArgumentException If {@code value} is {@code null}.
+ */
+ public McpMrtrConfig setClock(Clock value) {
+ if (value == null)
+ throw new IllegalArgumentException("clock must not be null");
+ clock = value;
+ return this;
+ }
}
diff --git a/juneau-rest/juneau-rest-server-mcp-v20260728/src/main/java/org/apache/juneau/rest/server/mcp/v20260728/McpRevision.java b/juneau-rest/juneau-rest-server-mcp-v20260728/src/main/java/org/apache/juneau/rest/server/mcp/v20260728/McpRevision.java
index 436b5ca50e..2dc5d01a77 100644
--- a/juneau-rest/juneau-rest-server-mcp-v20260728/src/main/java/org/apache/juneau/rest/server/mcp/v20260728/McpRevision.java
+++ b/juneau-rest/juneau-rest-server-mcp-v20260728/src/main/java/org/apache/juneau/rest/server/mcp/v20260728/McpRevision.java
@@ -896,7 +896,7 @@ public final class McpRevision implements org.apache.juneau.rest.server.mcp.McpR
.orElseThrow(() -> new McpException(CODE_INVALID_PARAMS, "Invalid or tampered requestState"));
if (! method.equals(sealed.method()))
throw new McpException(CODE_INVALID_PARAMS, "requestState method mismatch");
- if (sealed.expiresAtMs() <= System.currentTimeMillis())
+ if (sealed.expiresAtMs() <= mrtrConfig.getClock().millis())
throw new McpException(CODE_REQUEST_STATE_EXPIRED, "requestState has expired");
if (sealed.round() >= mrtrConfig.getMaxRounds())
// Client-facing message deliberately omits the configured cap value so server config is not leaked.
@@ -958,7 +958,7 @@ public final class McpRevision implements org.apache.juneau.rest.server.mcp.McpR
if (! clientElicitationSupported(params))
throw new McpException(CODE_MISSING_REQUIRED_CLIENT_CAPABILITY,
"Client does not advertise the elicitation capability required for input_required");
- var state = new McpRequestState(signal.getContinuation(), method, currentRound + 1, System.currentTimeMillis() + mrtrConfig.getTtlMs(),
+ var state = new McpRequestState(signal.getContinuation(), method, currentRound + 1, mrtrConfig.getClock().millis() + mrtrConfig.getTtlMs(),
mintJti(), argumentsHash);
var result = new InputRequiredResult().setRequestState(mrtrConfig.getCodec().seal(state, aad(method, target), principal(ctx)));
// Each inputRequests value is carried to the wire byte-for-byte as a raw sub-request object (see
diff --git a/juneau-rest/juneau-rest-server-mcp-v20260728/src/test/java/org/apache/juneau/rest/server/mcp/v20260728/McpMrtrConfig_Test.java b/juneau-rest/juneau-rest-server-mcp-v20260728/src/test/java/org/apache/juneau/rest/server/mcp/v20260728/McpMrtrConfig_Test.java
index 3060858922..cb36ef38d8 100644
--- a/juneau-rest/juneau-rest-server-mcp-v20260728/src/test/java/org/apache/juneau/rest/server/mcp/v20260728/McpMrtrConfig_Test.java
+++ b/juneau-rest/juneau-rest-server-mcp-v20260728/src/test/java/org/apache/juneau/rest/server/mcp/v20260728/McpMrtrConfig_Test.java
@@ -18,6 +18,9 @@ package org.apache.juneau.rest.server.mcp.v20260728;
import static org.junit.jupiter.api.Assertions.*;
+import java.time.Clock;
+import java.time.Instant;
+import java.time.ZoneOffset;
import java.util.Map;
import org.apache.juneau.bean.jsonrpc.JsonRpcRequest;
@@ -48,6 +51,19 @@ class McpMrtrConfig_Test {
assertEquals(10, a.getMaxRounds());
}
+ @Test void a01b_defaultClockIsSystemUtcAndSetClockRoundTrips() {
+ var a = new McpMrtrConfig();
+ assertEquals(Clock.systemUTC(), a.getClock());
+ var fixed = Clock.fixed(Instant.parse("2026-01-01T00:00:00Z"), ZoneOffset.UTC);
+ assertSame(fixed, a.setClock(fixed).getClock());
+ }
+
+ @Test void a01c_setClockNullThrows() {
+ var config = new McpMrtrConfig();
+ var e = assertThrows(IllegalArgumentException.class, () -> config.setClock(null));
+ assertEquals("clock must not be null", e.getMessage());
+ }
+
@Test void a02_setCodecNullThrows() {
var config = new McpMrtrConfig();
var e = assertThrows(IllegalArgumentException.class, () -> config.setCodec(null));
diff --git a/juneau-rest/juneau-rest-server-mcp-v20260728/src/test/java/org/apache/juneau/rest/server/mcp/v20260728/McpMrtrDispatch_Test.java b/juneau-rest/juneau-rest-server-mcp-v20260728/src/test/java/org/apache/juneau/rest/server/mcp/v20260728/McpMrtrDispatch_Test.java
index ea08e715d4..b46186cd0d 100644
--- a/juneau-rest/juneau-rest-server-mcp-v20260728/src/test/java/org/apache/juneau/rest/server/mcp/v20260728/McpMrtrDispatch_Test.java
+++ b/juneau-rest/juneau-rest-server-mcp-v20260728/src/test/java/org/apache/juneau/rest/server/mcp/v20260728/McpMrtrDispatch_Test.java
@@ -20,6 +20,7 @@ import static org.apache.juneau.test.bct.BctAssertions.assertSize;
import static org.junit.jupiter.api.Assertions.*;
import java.security.*;
+import java.time.*;
import java.util.*;
import java.util.concurrent.atomic.*;
import java.util.function.*;
@@ -412,6 +413,47 @@ class McpMrtrDispatch_Test {
assertEquals(0, calls.get());
}
+ /** A test clock the test can advance manually. */
+ static final class MutableClock extends Clock {
+ private Instant now;
+ MutableClock(Instant start) { now = start; }
+ void advance(Duration d) { now = now.plus(d); }
+ @Override public ZoneId getZone() { return ZoneOffset.UTC; }
+ @Override public Clock withZone(ZoneId zone) { return this; }
+ @Override public Instant instant() { return now; }
+ }
+
+ @Test void c09b_pauseStampsExpiryFromConfiguredClockAndResumeExpiresOnIt() {
+ // Clock-driven expiry: the pause seals now + TTL from the configured clock, a resume just inside the TTL
+ // succeeds, and once the clock reaches the sealed expiry the same token is rejected as expired -- no
+ // sleeping past a real TTL.
+ var t0 = Instant.parse("2026-01-01T00:00:00Z");
+ var clock = new MutableClock(t0);
+ var codec = new AeadRequestStateCodec();
+ var rev = revision(mrtr(codec).setTtlMs(60_000L).setClock(clock));
+ var completions = new AtomicInteger();
+ var config = new McpServerConfig().addTool(tool("ask", (args, c) -> {
+ if (c.getBean(McpMrtrResumeContext.class).isEmpty())
+ throw new McpInputRequiredSignal(Map.of("q1", reqEntry("elicitation")), "cont-1");
+ completions.incrementAndGet();
+ return text("done");
+ }));
+ var paused = (InputRequiredResult) send(rev, config, req(1, "tools/call", JsonMap.of("name", "ask"), true), hdrs("tools/call", "ask")).getResult();
+ var token = paused.getRequestState();
+ assertEquals(t0.plusSeconds(60).toEpochMilli(), codec.unseal(token, aad("tools/call", "ask")).orElseThrow().expiresAtMs());
+
+ var params = JsonMap.of("name", "ask", "requestState", token, "inputResponses", JsonMap.of("q1", "answer"));
+ clock.advance(Duration.ofMillis(59_999));
+ var resumed = (CallToolResult) send(rev, config, req(2, "tools/call", params, true), hdrs("tools/call", "ask")).getResult();
+ assertEquals("done", ((TextContent) resumed.getContent().get(0)).getText());
+ assertEquals(1, completions.get());
+
+ clock.advance(Duration.ofMillis(1));
+ var resp = send(rev, config, req(3, "tools/call", params, true), hdrs("tools/call", "ask"));
+ assertEquals(-32022, resp.getError().getCode());
+ assertEquals(1, completions.get());
+ }
+
@Test void c10_promptsGet_resumeToComplete_reInvokesHandlerWithResumeContext() {
var codec = new AeadRequestStateCodec();
var rev = revision(mrtr(codec));
diff --git a/juneau-rest/juneau-rest-server-terminal-bus/pom.xml b/juneau-rest/juneau-rest-server-terminal-bus/pom.xml
index 45058e0992..b7b8c911f1 100644
--- a/juneau-rest/juneau-rest-server-terminal-bus/pom.xml
+++ b/juneau-rest/juneau-rest-server-terminal-bus/pom.xml
@@ -150,13 +150,11 @@
<goal>exec</goal>
</goals>
<configuration>
- <executable>${juneau.jsTests.npm}</executable>
+ <executable>${juneau.jsTests.node}</executable>
<arguments>
- <argument>install</argument>
- <argument>--no-fund</argument>
- <argument>--no-audit</argument>
- <argument>--prefix</argument>
+ <argument>${maven.multiModuleProjectDirectory}/scripts/ensure-playwright.cjs</argument>
<argument>${js.tests.dir}</argument>
+ <argument>${juneau.jsTests.npm}</argument>
<argument>playwright@${playwright.version}</argument>
</arguments>
</configuration>
diff --git a/juneau-rest/juneau-rest-server-terminal/pom.xml b/juneau-rest/juneau-rest-server-terminal/pom.xml
index 0bbdb57226..bdc4f4d9f8 100644
--- a/juneau-rest/juneau-rest-server-terminal/pom.xml
+++ b/juneau-rest/juneau-rest-server-terminal/pom.xml
@@ -105,13 +105,11 @@
<goal>exec</goal>
</goals>
<configuration>
- <executable>${juneau.jsTests.npm}</executable>
+ <executable>${juneau.jsTests.node}</executable>
<arguments>
- <argument>install</argument>
- <argument>--no-fund</argument>
- <argument>--no-audit</argument>
- <argument>--prefix</argument>
+ <argument>${maven.multiModuleProjectDirectory}/scripts/ensure-playwright.cjs</argument>
<argument>${js.tests.dir}</argument>
+ <argument>${juneau.jsTests.npm}</argument>
<argument>playwright@${playwright.version}</argument>
</arguments>
</configuration>
diff --git a/juneau-rest/juneau-rest-server-views/pom.xml b/juneau-rest/juneau-rest-server-views/pom.xml
index 82a202a2b7..d453dd78b3 100644
--- a/juneau-rest/juneau-rest-server-views/pom.xml
+++ b/juneau-rest/juneau-rest-server-views/pom.xml
@@ -126,8 +126,11 @@
standard ASF header.
Prerequisites when you DO opt in: Node and npm on the PATH (override the launcher with
- -Djuneau.jsTests.node=/path/to/node), plus network access on the first run. Both steps below are
- idempotent and cost roughly a second once target/js is warm. On Windows the npm launcher is npm.cmd,
+ -Djuneau.jsTests.node=/path/to/node), plus network access on the first run. The first step below
+ (scripts/ensure-playwright.cjs) skips npm when the pinned packages are already in target/js/node_modules, and
+ otherwise restores them from a per-user cache (~/.cache/juneau-js-tests, override with the
+ JUNEAU_JS_TESTS_CACHE environment variable) before falling back to npm; the second is idempotent. Both cost
+ roughly a second once warm. On Windows the npm launcher is npm.cmd,
so pass -Djuneau.jsTests.npm=npm.cmd there.
The browser download is the one thing whose location is separately overridable, via
@@ -162,13 +165,11 @@
<goal>exec</goal>
</goals>
<configuration>
- <executable>${juneau.jsTests.npm}</executable>
+ <executable>${juneau.jsTests.node}</executable>
<arguments>
- <argument>install</argument>
- <argument>--no-fund</argument>
- <argument>--no-audit</argument>
- <argument>--prefix</argument>
+ <argument>${maven.multiModuleProjectDirectory}/scripts/ensure-playwright.cjs</argument>
<argument>${js.tests.dir}</argument>
+ <argument>${juneau.jsTests.npm}</argument>
<argument>playwright@${playwright.version}</argument>
<argument>jquery@3.7.1</argument>
<argument>datatables.net@2.1.8</argument>
diff --git a/juneau-rest/juneau-rest-server/src/main/java/org/apache/juneau/rest/server/guard/RateLimitGuard.java b/juneau-rest/juneau-rest-server/src/main/java/org/apache/juneau/rest/server/guard/RateLimitGuard.java
index 665ab0a527..bdda7132b8 100644
--- a/juneau-rest/juneau-rest-server/src/main/java/org/apache/juneau/rest/server/guard/RateLimitGuard.java
+++ b/juneau-rest/juneau-rest-server/src/main/java/org/apache/juneau/rest/server/guard/RateLimitGuard.java
@@ -89,7 +89,9 @@ import org.apache.juneau.rest.server.*;
* map to a calendar instant, which slightly complicates debugging when the bucket is in an unexpected state. The
* nanosecond source is pluggable on {@link InMemoryStorage} (package-private constructor overload) so tests can
* pin the refill window with a deterministic clock instead of racing real wall-clock time; production code always
- * goes through the single-arg constructor, which defaults to {@code System::nanoTime}.
+ * goes through the single-arg constructor, which defaults to {@code System::nanoTime}. The wall-clock
+ * {@link BucketState#lastRequest()} instant reported by {@link Storage#snapshot()} is read from a separate
+ * {@link Clock} (default {@link Clock#systemUTC()}), likewise overridable through a package-private constructor.
*
* <h5 class='section'>See Also:</h5><ul>
* <li class='link'><a class="doclink" href="https://juneau.apache.org/docs/topics/RestServerRateLimitAndRequestId">REST Server — Rate-Limiting and Request-Id Propagation</a>
@@ -576,6 +578,7 @@ public class RateLimitGuard extends RestGuard {
private final ConcurrentHashMap<String,Bucket> buckets = new ConcurrentHashMap<>();
private final int maxKeys;
private final LongSupplier nanoClock;
+ private final Clock clock;
InMemoryStorage(int maxKeys) {
this(maxKeys, System::nanoTime);
@@ -594,16 +597,33 @@ public class RateLimitGuard extends RestGuard {
* @param nanoClock Supplies the current time in nanoseconds for refill math and idle-eviction bookkeeping. Must not be <jk>null</jk>.
*/
InMemoryStorage(int maxKeys, LongSupplier nanoClock) {
- reqnn("nanoClock", nanoClock);
+ this(maxKeys, nanoClock, Clock.systemUTC());
+ }
+
+ /**
+ * Constructor allowing both the nanosecond and the wall-clock time sources to be overridden.
+ *
+ * <p>
+ * Test-only seam, like {@link #InMemoryStorage(int, LongSupplier)}. The {@link Clock} only stamps the
+ * {@link BucketState#lastRequest()} instant reported by {@link #snapshot()}; refill and eviction math stay on
+ * {@code nanoClock}.
+ *
+ * @param maxKeys The maximum number of keys held before LRU-style eviction kicks in. Must be {@code > 0}.
+ * @param nanoClock Supplies the current time in nanoseconds for refill math and idle-eviction bookkeeping. Must not be <jk>null</jk>.
+ * @param clock Supplies the wall-clock time recorded as each bucket's last-request instant. Must not be <jk>null</jk>.
+ */
+ InMemoryStorage(int maxKeys, LongSupplier nanoClock, Clock clock) {
+ reqnn("nanoClock", nanoClock, "clock", clock);
if (maxKeys <= 0)
throw new IllegalArgumentException("Argument 'maxKeys' must be > 0.");
this.maxKeys = maxKeys;
this.nanoClock = nanoClock;
+ this.clock = clock;
}
@Override
public AcquireResult tryAcquire(String key, int capacity, double permitsPerSecond) {
- var bucket = buckets.computeIfAbsent(key, k -> new Bucket(capacity, nanoClock));
+ var bucket = buckets.computeIfAbsent(key, k -> new Bucket(capacity, nanoClock, clock));
var result = bucket.tryAcquire(capacity, permitsPerSecond);
if (buckets.size() > maxKeys)
evictOldest();
@@ -659,25 +679,28 @@ public class RateLimitGuard extends RestGuard {
private long lastNanos;
private long lastWallMillis;
private final LongSupplier nanoClock;
+ private final Clock clock;
Bucket(int capacity) {
- this(capacity, System::nanoTime);
+ this(capacity, System::nanoTime, Clock.systemUTC());
}
/**
- * Constructor allowing the nanosecond time source to be overridden.
+ * Constructor allowing the time sources to be overridden.
*
* <p>
- * Test-only seam — see {@link InMemoryStorage#InMemoryStorage(int, LongSupplier)}.
+ * Test-only seam — see {@link InMemoryStorage#InMemoryStorage(int, LongSupplier, Clock)}.
*
* @param capacity The initial (full) token count.
* @param nanoClock Supplies the current time in nanoseconds for refill math. Must not be <jk>null</jk>.
+ * @param clock Supplies the wall-clock time recorded as the last-request instant. Must not be <jk>null</jk>.
*/
- Bucket(int capacity, LongSupplier nanoClock) {
+ Bucket(int capacity, LongSupplier nanoClock, Clock clock) {
this.nanoClock = nanoClock;
+ this.clock = clock;
this.tokens = capacity;
this.lastNanos = nanoClock.getAsLong();
- this.lastWallMillis = System.currentTimeMillis();
+ this.lastWallMillis = clock.millis();
}
synchronized Storage.AcquireResult tryAcquire(int capacity, double permitsPerSecond) {
@@ -685,7 +708,7 @@ public class RateLimitGuard extends RestGuard {
var elapsedSeconds = (now - lastNanos) / 1_000_000_000.0;
tokens = Math.min(capacity, tokens + elapsedSeconds * permitsPerSecond);
lastNanos = now;
- lastWallMillis = System.currentTimeMillis();
+ lastWallMillis = clock.millis();
if (tokens >= 1.0) {
tokens -= 1.0;
return new Storage.AcquireResult(true, (int) Math.floor(tokens), secondsUntilFull(capacity, permitsPerSecond));
diff --git a/juneau-rest/juneau-rest-server/src/main/java/org/apache/juneau/rest/server/stats/ThrownStats.java b/juneau-rest/juneau-rest-server/src/main/java/org/apache/juneau/rest/server/stats/ThrownStats.java
index 0885d39e1b..c705570253 100644
--- a/juneau-rest/juneau-rest-server/src/main/java/org/apache/juneau/rest/server/stats/ThrownStats.java
+++ b/juneau-rest/juneau-rest-server/src/main/java/org/apache/juneau/rest/server/stats/ThrownStats.java
@@ -19,6 +19,7 @@ package org.apache.juneau.rest.server.stats;
import static org.apache.juneau.commons.utils.CollectionUtils.*;
import static org.apache.juneau.commons.utils.Shorts.*;
+import java.time.*;
import java.util.*;
import java.util.concurrent.atomic.*;
@@ -47,6 +48,7 @@ public class ThrownStats {
long hash;
List<String> stackTrace;
ThrownStats causedBy;
+ Clock clock = Clock.systemUTC();
private Class<? extends ThrownStats> implType;
@@ -84,6 +86,21 @@ public class ThrownStats {
return this;
}
+ /**
+ * Specifies the clock used to stamp the first and last occurrence times.
+ *
+ * <p>
+ * Defaults to {@link Clock#systemUTC()}. Tests can pass {@link Clock#fixed(Instant, ZoneId)} to make the
+ * reported occurrence times deterministic.
+ *
+ * @param value The new value for this setting. Must not be <jk>null</jk>.
+ * @return This object.
+ */
+ public Builder clock(Clock value) {
+ clock = reqnn("value", value);
+ return this;
+ }
+
/**
* Specifies the calculated hash.
*
@@ -146,6 +163,7 @@ public class ThrownStats {
private final List<String> stackTrace;
private final Optional<ThrownStats> causedBy;
+ private final Clock clock;
private final AtomicInteger count;
private final AtomicLong firstOccurrence;
private final AtomicLong lastOccurrence;
@@ -160,6 +178,7 @@ public class ThrownStats {
this.stackTrace = u(cp(x.stackTrace));
this.causedBy = o(ThrownStats.copyOrNull(x.causedBy.orElse(null)));
this.hash = x.hash;
+ this.clock = x.clock;
this.count = new AtomicInteger(x.count.get());
this.firstOccurrence = new AtomicLong(x.firstOccurrence.get());
this.lastOccurrence = new AtomicLong(x.lastOccurrence.get());
@@ -187,8 +206,9 @@ public class ThrownStats {
this.stackTrace = u(cp(builder.stackTrace));
this.causedBy = o(builder.causedBy);
this.hash = builder.hash;
+ this.clock = builder.clock;
this.count = new AtomicInteger(0);
- long ct = System.currentTimeMillis();
+ long ct = clock.millis();
this.firstOccurrence = new AtomicLong(ct);
this.lastOccurrence = new AtomicLong(ct);
}
@@ -269,7 +289,7 @@ public class ThrownStats {
*/
public ThrownStats increment() {
count.incrementAndGet();
- lastOccurrence.set(System.currentTimeMillis());
+ lastOccurrence.set(clock.millis());
causedBy.ifPresent(ThrownStats::increment);
return this;
}
diff --git a/juneau-rest/juneau-rest-server/src/main/java/org/apache/juneau/rest/server/stats/ThrownStore.java b/juneau-rest/juneau-rest-server/src/main/java/org/apache/juneau/rest/server/stats/ThrownStore.java
index 68f47b2a8f..440f099120 100644
--- a/juneau-rest/juneau-rest-server/src/main/java/org/apache/juneau/rest/server/stats/ThrownStore.java
+++ b/juneau-rest/juneau-rest-server/src/main/java/org/apache/juneau/rest/server/stats/ThrownStore.java
@@ -22,6 +22,7 @@ import static org.apache.juneau.commons.utils.CollectionUtils.*;
import static org.apache.juneau.commons.utils.ObjectUtils.*;
import static org.apache.juneau.commons.utils.Shorts.*;
+import java.time.*;
import java.util.*;
import java.util.concurrent.*;
@@ -48,6 +49,7 @@ public class ThrownStore {
ThrownStore parent;
Class<? extends ThrownStats> statsImplClass;
Set<Class<?>> ignoreClasses;
+ Clock clock;
private ThrownStore impl;
private Class<? extends ThrownStore> type;
@@ -70,6 +72,21 @@ public class ThrownStore {
return beanStore;
}
+ /**
+ * Specifies the clock used to stamp the first and last occurrence times of recorded exceptions.
+ *
+ * <p>
+ * Defaults to the parent store's clock, or {@link Clock#systemUTC()} if there is no parent. Tests can pass
+ * {@link Clock#fixed(Instant, ZoneId)} to make the occurrence times reported by {@link ThrownStats} deterministic.
+ *
+ * @param value The new value for this setting. Can be <jk>null</jk> (inherits the default).
+ * @return This object.
+ */
+ public Builder clock(Clock value) {
+ clock = value;
+ return this;
+ }
+
/**
* Specifies the list of classes to ignore when calculating stack traces.
*
@@ -180,6 +197,7 @@ public class ThrownStore {
private final BeanStore beanStore;
private final Class<? extends ThrownStats> statsImplClass;
private final Set<String> ignoreClasses;
+ private final Clock clock;
/**
* Constructor.
@@ -198,6 +216,7 @@ public class ThrownStore {
this.beanStore = builder.beanStore();
this.statsImplClass = coalesce(builder.statsImplClass, parent.isPresent() ? parent.get().statsImplClass : null, null);
+ this.clock = coalesce(builder.clock, parent.isPresent() ? parent.get().clock : null, Clock.systemUTC());
Set<String> s = null;
if (nn(builder.ignoreClasses))
@@ -270,6 +289,7 @@ public class ThrownStore {
stc = ThrownStats
.create(beanStore)
.type(statsImplClass)
+ .clock(clock)
.throwable(t)
.hash(hash)
.stackTrace(createStackTrace(t))
diff --git a/juneau-rest/juneau-rest-server/src/test/java/org/apache/juneau/rest/server/stats/ThrownStore_Test.java b/juneau-rest/juneau-rest-server/src/test/java/org/apache/juneau/rest/server/stats/ThrownStore_Test.java
index fa8634c070..6b131e6cf0 100644
--- a/juneau-rest/juneau-rest-server/src/test/java/org/apache/juneau/rest/server/stats/ThrownStore_Test.java
+++ b/juneau-rest/juneau-rest-server/src/test/java/org/apache/juneau/rest/server/stats/ThrownStore_Test.java
@@ -20,6 +20,7 @@ import static org.apache.juneau.commons.utils.Shorts.*;
import static org.apache.juneau.test.bct.BctAssertions.*;
import static org.junit.jupiter.api.Assertions.*;
+import java.time.*;
import java.util.*;
import org.apache.juneau.*;
@@ -308,6 +309,48 @@ class ThrownStore_Test extends TestBase {
assertContains("foo", stats);
}
+ /** A test clock the test can advance manually. */
+ static final class MutableClock extends Clock {
+ private Instant now;
+ MutableClock(Instant start) { now = start; }
+ void advance(Duration d) { now = now.plus(d); }
+ @Override public ZoneId getZone() { return ZoneOffset.UTC; }
+ @Override public Clock withZone(ZoneId zone) { return this; }
+ @Override public Instant instant() { return now; }
+ }
+
+ @Test void c02_thrownStats_occurrenceTimesReadFromInjectedClock() {
+ var t0 = Instant.parse("2026-01-01T00:00:00Z");
+ var clock = new MutableClock(t0);
+ var store = ThrownStore.create().clock(clock).build();
+ var t = new Throwable("foo");
+ t.fillInStackTrace();
+
+ var stats = store.add(t);
+ assertEquals(t0.toEpochMilli(), stats.getFirstOccurrence());
+ assertEquals(t0.toEpochMilli(), stats.getLastOccurrence());
+
+ clock.advance(Duration.ofSeconds(5));
+ stats = store.add(t);
+ assertEquals(t0.toEpochMilli(), stats.getFirstOccurrence());
+ assertEquals(t0.plusSeconds(5).toEpochMilli(), stats.getLastOccurrence());
+ assertEquals(t0.plusSeconds(5).toEpochMilli(), ThrownStats.copyOrNull(stats).getLastOccurrence());
+ }
+
+ @Test void c03_thrownStats_childStoreInheritsParentClock() {
+ var t0 = Instant.parse("2026-01-01T00:00:00Z");
+ var parent = ThrownStore.create().clock(Clock.fixed(t0, ZoneOffset.UTC)).build();
+ var child = ThrownStore.create().parent(parent).build();
+ var t = new Throwable("foo");
+ t.fillInStackTrace();
+ assertEquals(t0.toEpochMilli(), child.add(t).getFirstOccurrence());
+ }
+
+ @Test void c04_thrownStats_nullClockRejected() {
+ var b = ThrownStats.create(BasicBeanStore.INSTANCE);
+ assertThrows(IllegalArgumentException.class, () -> b.clock(null));
+ }
+
//------------------------------------------------------------------------------------------------------------------
// ThrownStore tests.
//------------------------------------------------------------------------------------------------------------------
diff --git a/juneau-run/src/main/python/juneau_run.py b/juneau-run/src/main/python/juneau_run.py
index 46ac26d1b0..927f09bd74 100644
--- a/juneau-run/src/main/python/juneau_run.py
+++ b/juneau-run/src/main/python/juneau_run.py
@@ -592,6 +592,8 @@ _MARK_RE = re.compile("(?<!m)[✓✗]") # not one already coloured (right afte
_NOTE_PREFIX = {"info": "ℹ", "warn": "⚠", "error": "✗"}
PENDING_ENV = "JUNEAU_RUN_PENDING" # a parent session's labelled pending rows, [[id, label], ...], for a nested one
OVERALL_ENV = "JUNEAU_RUN_OVERALL" # a parent session's {elapsed, done, pending: [[id, hist_wall_s], ...]}, for a nested one
+HEADER_ENV = "JUNEAU_RUN_HEADER" # a parent session's header, [[head|more, text, cut], ...], drawn atop a nested view
+HANDBACK_ENV = "JUNEAU_RUN_HANDBACK" # a file a nested session writes its rows' outcomes to, [[id, status], ...]
# A row's icon (spec 5.2): a spinner while it runs, then its outcome: a coloured ● for ok, warn and fail, or ✔ ⚠ ✗
# when colour is off. No VS16 after ⚠: it must stay one column.
SPINNER = "⠋⠙⠹⠸⠼⠴⠦⠧⠇⠏"
@@ -850,12 +852,21 @@ def fmt_totals(totals, tests_mode=False, expected=None):
return f"{total:,}{of} tests" + (f", {fail + err} failed" if fail + err else "")
-def format_final(status, ms, failed_title, full_log):
+def format_final(status, ms, failed_title, full_log, done=None, total=None):
+ """The final row's text (after its ▶): the outcome, the steps that finished ok or warned of all the run counted
+ (none without a plan), then the full log on a failure."""
+ steps = f" · {done} of {total} steps" if total else ""
if status == "ok":
- return f"✅ Done in {fmt_elapsed(ms)}"
+ return f"Overall ✅ Done in {fmt_elapsed(ms)}{steps}"
if status == "cancelled":
- return f"⚠ Cancelled after {fmt_elapsed(ms)}"
- return "❌ Failed" + (f" at {failed_title}" if failed_title else "") + (f" · full log: {full_log}" if full_log else "")
+ return f"Overall ⚠ Cancelled after {fmt_elapsed(ms)}{steps}"
+ return ("Overall ❌ Failed" + (f" at {failed_title}" if failed_title else "") + steps
+ + (f" · full log: {full_log}" if full_log else ""))
+
+
+def final_line(block, color=False):
+ """A final block as the run's last row: ▶ (cyan under colour, like the title row) and format_final's text."""
+ return f"{head_mark(HEAD_MARK, color)} {format_final(*block[1:])}"
BAR_SRCS = ("sub", "row") # unit sources that keep Maven module-bar and Setup bookkeeping (boxes, cells, poms)
@@ -1041,6 +1052,21 @@ def _parent_overall():
return None
+def _parent_header():
+ """JUNEAU_RUN_HEADER as header blocks, or [] when it is unset or malformed."""
+ try:
+ value = json.loads(os.environ.get(HEADER_ENV) or "[]")
+ return [("header", str(text), HEAD_MARK if mark == "head" else MORE_MARK) + (("cut",) if cut else ())
+ for mark, text, cut in value]
+ except (ValueError, TypeError):
+ return []
+
+
+def _first_line(text):
+ """text's first line, followed by " …" when it has more (a multi-line commit message)."""
+ return (text.split("\n", 1)[0].rstrip() + " …") if "\n" in text else text
+
+
class Board:
"""The run's state for the console view. apply(kind, fields) returns the blocks that event produces."""
@@ -1059,6 +1085,7 @@ class Board:
self.setup = None # the run's one Setup row (pom modules), owned by the first Maven row that has poms
self.plan = None # the planned top-level step ids, in order (spec 6), or None
self.plan_labels = {} # planned step id -> label, for the ones that draw a pending row
+ self.handed_back = {} # a handoff child's step id -> its status (JUNEAU_RUN_HANDBACK): counted, never drawn
self.parent_pending = _parent_pending() if nested else [] # a parent session's pending rows, (id, label)
self.parent_overall = _parent_overall() if nested else None # a parent session's Overall inputs (OVERALL_ENV)
self.finished = False # the done event arrived: nothing is pending any more
@@ -1080,10 +1107,11 @@ class Board:
return []
out = []
if self.plan:
- last = max((i for i, sid in enumerate(self.plan) if sid in self.rows), default=-1)
+ started = set(self.rows) | set(self.handed_back)
+ last = max((i for i, sid in enumerate(self.plan) if sid in started), default=-1)
out = [(sid, self.plan_labels[sid]) for i, sid in enumerate(self.plan)
- if i > last and sid in self.plan_labels and sid not in self.rows]
- have = set(self.rows) | {i for i, _ in out}
+ if i > last and sid in self.plan_labels and sid not in started]
+ have = set(self.rows) | set(self.handed_back) | {i for i, _ in out}
return out + [(i, label) for i, label in self.parent_pending if i not in have]
def setup_failed(self):
@@ -1182,11 +1210,11 @@ class Board:
self.session_t0 = self.clock()
if self.nested:
return []
- # "<title>: <item>, <item>", or the bare title
- # an item with a newline (a multi-line commit message) shows its first line and an ellipsis
- items = [(t.split("\n", 1)[0].rstrip() + " …") if "\n" in t else t for t in map(str, f["header"])]
- head = f["title"] + (": " + ", ".join(items) if items else "")
- return [("header", head, HEAD_MARK)] + [("header", t, MORE_MARK) for t in f.get("header_more", ())]
+ # "<title>: <item> · <item>", or the bare title; then a cut line per `lines` entry, then the log lines
+ head = f["title"] + (": " + " · ".join(_first_line(str(t)) for t in f["header"]) if f["header"] else "")
+ return ([("header", head, HEAD_MARK)]
+ + [("header", _first_line(str(t)), MORE_MARK, "cut") for t in f.get("lines", ())]
+ + [("header", t, MORE_MARK) for t in f.get("header_more", ())])
def _on_plan(self, f):
self.plan = list(f["ids"])
@@ -1209,15 +1237,20 @@ class Board:
def _overall_steps(self, now):
"""The plan's steps as overall_figures() takes them, after a parent session's (OVERALL_ENV): its done steps,
and its pending ones that this plan and these rows do not already cover."""
+ started = set(self.rows) | set(self.handed_back)
steps = []
if self.parent_overall is not None:
_, done, pending = self.parent_overall
steps = [("", "done", None, None)] * done
steps += [(sid, "pending", hist, None) for sid, hist in pending
- if sid not in (self.plan or ()) and sid not in self.rows]
- last_started = max((i for i, sid in enumerate(self.plan) if sid in self.rows), default=-1)
+ if sid not in (self.plan or ()) and sid not in started]
+ last_started = max((i for i, sid in enumerate(self.plan) if sid in started), default=-1)
for i, sid in enumerate(self.plan):
r = self.rows.get(sid)
+ if r is None and sid in self.handed_back:
+ if self.handed_back[sid] != "skip":
+ steps.append((sid, "done", None, None))
+ continue
if r is None:
if i < last_started:
continue # passed over by a later planned step
@@ -1263,6 +1296,27 @@ class Board:
return {"elapsed": self._overall_elapsed(now), "done": sum(1 for _, state, _, _ in steps if state == "done"),
"pending": [[sid, hist] for sid, state, hist, _ in steps if state == "pending"]}
+ def hand_back(self, outcomes):
+ """Record a handoff child's outcomes, [(id, status, label)] (JUNEAU_RUN_HANDBACK): Overall and the final counts
+ take them, nothing draws them (the child printed its rows). An id this Board has a row for keeps its row; the
+ first failed one names the run's failure, by this Board's label for it, else the child's (never the bare id)."""
+ for sid, status, label in outcomes:
+ if sid in self.rows:
+ continue
+ self.handed_back[sid] = status
+ if status == "fail" and self.failed_title is None:
+ self.failed_title = self.plan_labels.get(sid) or label
+
+ def step_counts(self):
+ """(done, total) for the final row, or None with no plan: the counted steps (a parent's too) that finished ok
+ or warned, and all the steps Overall counts (skips are left out)."""
+ if not self.plan:
+ return None
+ failed = {sid for sid, r in self.rows.items() if r.status == "fail"}
+ failed |= {sid for sid, status in self.handed_back.items() if status == "fail"}
+ steps = self._overall_steps(self.clock())
+ return sum(1 for sid, state, _, _ in steps if state == "done" and sid not in failed), len(steps)
+
def _on_step_start(self, f):
parent = self.rows.get(f.get("parent"))
if parent is not None: # a sub-row: a box on its parent's row until its tool sends modules (_adopt)
@@ -1652,13 +1706,13 @@ class Board:
return [("logonly", f["text"])]
def _on_prompt(self, f):
- return [("raw", f["text"])]
+ return [("raw", f["text"], "prompt")]
def _on_prompt_echo(self, f):
- return [("raw", f["text"])]
+ return [("raw", f["text"], "echo")]
def _on_prompt_end(self, f):
- return [("raw", "\n")]
+ return [("raw", "\n", "end")]
def _on_quiet(self, f):
r = self._target(f["step"])
@@ -1670,7 +1724,8 @@ class Board:
self.finished = True
if self.nested:
return []
- return [("final", f["status"], int((self.clock() - self.t0) * 1000), self.failed_title, self.full_log)]
+ return [("final", f["status"], int((self.clock() - self.t0) * 1000), self.failed_title, self.full_log,
+ *(self.step_counts() or (None, None)))]
def row_state(row):
@@ -1806,14 +1861,19 @@ def head_mark(mark, color=False):
return mark
-def header_lines(block, cols, color=False):
+def header_lines(block, cols, color=False, live=False):
"""A header block's lines. ("header", text) is the text as is; ("header", text, mark) puts mark in the icon
- column, dim when it is MORE_MARK. The title row (HEAD_MARK) word-wraps at cols with a two-space hanging indent;
- the others don't wrap, so a log path stays on its `tail -f` line."""
+ column, dim when it is MORE_MARK. A fourth field ("cut") marks a `lines` entry. On the live screen (live) the
+ title row and the cut rows are cut to cols with an ellipsis, so each costs one line; elsewhere the title row
+ word-wraps at cols with a two-space hanging indent. The log rows never wrap or cut, so a path stays on its
+ `tail -f` line."""
if len(block) < 3:
return [block[1]]
text, mark = block[1], block[2]
shown = head_mark(mark, color)
+ if live and (mark == HEAD_MARK or len(block) > 3):
+ room = cols - 2
+ return [f"{shown} {text if _vlen(text) <= room else _vcut(text, max(0, room - 1)) + '…'}"]
if mark != HEAD_MARK:
return [f"{shown} {text}"]
lines = [""]
@@ -1903,7 +1963,7 @@ class PlainRenderer:
elif kind == "logonly" and self.log:
self._write(block[1] + "\n")
elif kind == "final":
- self._write(format_final(*block[1:]) + "\n")
+ self._write(final_line(block) + "\n")
def tick(self, board):
now = self.clock()
@@ -2009,6 +2069,182 @@ def collapse_names(names, cols):
return line if _vlen(line) <= budget else _vcut(line, budget)
+def label_list(head, labels, cols):
+ """head, then the labels comma-joined, cut at a label boundary with ", …" to fit cols (the folded pending, done
+ and failed lines)."""
+ line = head + ", ".join(labels)
+ if _vlen(line) <= cols:
+ return line
+ for keep in range(len(labels) - 1, 0, -1):
+ line = head + ", ".join(labels[:keep]) + ", …"
+ if _vlen(line) <= cols:
+ return line
+ return _vcut(head, max(0, cols - 1)) + "…"
+
+
+class _Group:
+ """One finished row of the transcript with what belongs to it: state (ok, warn or fail), label (None for text
+ before any row), head (its Setup and row lines) and body (its notes, hints and module lines)."""
+
+ def __init__(self, state, label, head, body):
+ self.state, self.label, self.head, self.body = state, label, head, body
+
+
+_Run = collections.namedtuple("_Run", "head units") # a running row: its Setup, row and progress lines, then _Units
+_Unit = collections.namedtuple("_Unit", "t0 i lines name") # a running unit's lines, and its name on a packed names line
+
+
+class _Fold:
+ """The live frame's parts and how far each is folded. lines() draws them at the current fold; steps() folds one
+ notch at a time, in the spec's order (3, Stage A then Stage B), yielding after each."""
+
+ def __init__(self, header, groups, running, pending, overall, cols, color):
+ self.title, self.head = header
+ self.groups, self.running, self.pending, self.overall = groups, running, pending, overall
+ self.cols, self.color = cols, color
+ self.kept = len(pending) # pending rows kept whole; the rest fold into one "N more" line
+ self.done = [] # indexes of the folded done groups, oldest first
+ self.units = set() # (run index, unit index) folded onto a names line
+ self.bodies = set() # indexes of failed groups whose body is cut to one line
+ self.no_names = set() # run indexes whose names line is dropped
+ self.drop = set() # "done", "more", "failed": a folded line that is dropped
+ self.failed = False # every failed group folded into one "N failed" line
+ self.head_gone = set() # indexes into head that are dropped
+ self.runs_gone = 0 # running rows dropped, oldest first
+
+ def _summary(self, state, head, labels):
+ text = label_list(head, labels, self.cols - 2)
+ return f"{icon(state, color=self.color)} " + (f"\x1b[2m{text}\x1b[0m" if self.color else text)
+
+ def lines(self):
+ """(text, wraps) pairs: wraps is True for a log line, which is never cut and the terminal wraps."""
+ out = [] if self.title is None else [(self.title, False)]
+ out += [(text, wraps) for i, (text, wraps, _) in enumerate(self.head) if i not in self.head_gone]
+ fails = [i for i, g in enumerate(self.groups) if g.state == "fail"]
+ done = set(self.done)
+ for i, g in enumerate(self.groups):
+ if i in done:
+ if i == self.done[0] and "done" not in self.drop:
+ folded = [self.groups[j] for j in self.done]
+ state = "warn" if any(f.state == "warn" for f in folded) else "ok"
+ labels = [f.label for f in folded if f.label]
+ out.append((self._summary(state, f"{len(labels)} done: ", labels), False))
+ elif g.state == "fail" and self.failed:
+ if i == fails[0] and "failed" not in self.drop:
+ labels = [self.groups[j].label for j in fails if self.groups[j].label]
+ out.append((self._summary("fail", f"{len(labels)} failed: ", labels), False))
+ else:
+ out += [(t, False) for t in g.head]
+ if i in self.bodies:
+ out.append((f" … {len(g.body)} more lines (condensed log)", False))
+ else:
+ out += [(t, False) for t in g.body]
+ for j, run in enumerate(self.running):
+ if j < self.runs_gone:
+ continue
+ out += [(t, False) for t in run.head]
+ out += [(t, False) for k, u in enumerate(run.units) if (j, k) not in self.units for t in u.lines]
+ folded = sorted((u for k, u in enumerate(run.units) if (j, k) in self.units), key=lambda u: (u.t0, u.i))
+ if folded and j not in self.no_names:
+ out.append((" " + collapse_names([u.name for u in folded], self.cols - 2), False))
+ out += [(_vcut(pending_line(label, self.color), self.cols), False) for label in self.pending[:self.kept]]
+ rest = self.pending[self.kept:]
+ if rest and "more" not in self.drop:
+ out.append((pending_line(label_list(f"{len(rest)} more: ", rest, self.cols - 2), self.color), False))
+ if self.overall is not None:
+ out.append((self.overall, False))
+ return out
+
+ def height(self, columns):
+ """The physical height of lines() without building them (a tick folds a long transcript a notch at a time):
+ a log line costs ceil(width / columns), every other line one."""
+ n = 0 if self.title is None else 1
+ n += sum(max(1, -(-_vlen(t) // columns)) if wraps else 1
+ for i, (t, wraps, _) in enumerate(self.head) if i not in self.head_gone)
+ done = set(self.done)
+ n += 1 if done and "done" not in self.drop else 0
+ if self.failed and "failed" not in self.drop and any(g.state == "fail" for g in self.groups):
+ n += 1
+ for i, g in enumerate(self.groups):
+ if i not in done and not (g.state == "fail" and self.failed):
+ n += len(g.head) + (1 if i in self.bodies else len(g.body))
+ for j in range(self.runs_gone, len(self.running)):
+ units = self.running[j].units
+ n += len(self.running[j].head) + sum(len(u.lines) for k, u in enumerate(units) if (j, k) not in self.units)
+ n += 1 if j not in self.no_names and any((j, k) in self.units for k in range(len(units))) else 0
+ n += min(self.kept, len(self.pending)) + (1 if self.kept < len(self.pending) and "more" not in self.drop else 0)
+ return n + (1 if self.overall is not None else 0)
+
+ def steps(self):
+ # A1: pending rows, keeping the first k (never folding just one of several); a lone one folds too, so B3 can drop it
+ n = len(self.pending)
+ for k in range(max(n - 2, 0), -1, -1) if n else ():
+ self.kept = k
+ yield
+ # A2: done groups, oldest first
+ for i, g in enumerate(self.groups):
+ if g.state != "fail":
+ self.done.append(i)
+ yield
+ # A3: unit lines, newest first (of units started at the same instant, the one listed later folds first)
+ units = [(u.t0, u.i, j, k) for j, run in enumerate(self.running) for k, u in enumerate(run.units)]
+ for _, _, j, k in sorted(units, reverse=True):
+ self.units.add((j, k))
+ yield
+ # A4: failed groups' bodies, oldest first
+ for i, g in enumerate(self.groups):
+ if g.state == "fail" and len(g.body) > 1:
+ self.bodies.add(i)
+ yield
+ yield from self._stage_b()
+
+ def _stage_b(self):
+ # B1: the packed names lines, oldest row first
+ for j in range(len(self.running)):
+ if any(jj == j for jj, _ in self.units):
+ self.no_names.add(j)
+ yield
+ # B2, B3: the done line, the more line
+ for name, present in (("done", bool(self.done)), ("more", self.kept < len(self.pending))):
+ if present:
+ self.drop.add(name)
+ yield
+ # B4: the failed groups, into one "N failed" line
+ if any(g.state == "fail" for g in self.groups):
+ self.failed = True
+ yield
+ # B5: the header's lines and log lines (the cut lines in order, then Condensed, Output, Cancel)
+ for i in sorted(range(len(self.head)), key=lambda i: (self.head[i][2], i)):
+ self.head_gone.add(i)
+ yield
+ # B6: running rows, oldest first, with their Setup and progress lines (the newest stays)
+ while self.runs_gone < len(self.running) - 1:
+ self.runs_gone += 1
+ yield
+ # B7: the failed line; B8: the newest running row
+ if self.failed:
+ self.drop.add("failed")
+ yield
+ if self.runs_gone < len(self.running):
+ self.runs_gone += 1
+ yield
+
+
+def fit_view(header, groups, running, pending, overall, budget, cols, columns, color=False):
+ """The live frame's lines, folded until their physical height is at most budget (spec 3). header is (title line
+ or None, [(text, wraps, drop rank)]); a wrapping (log) line costs ceil(width / columns) lines, every other line is
+ cut to cols and costs one. The title and Overall lines are pinned; with budget 1 only the first of them stays."""
+ fold = _Fold(header, groups, running, pending, overall, cols, color)
+ if fold.height(columns) > budget:
+ for _ in fold.steps():
+ if fold.height(columns) <= budget:
+ break
+ lines = [t for t, _ in fold.lines()]
+ if fold.height(columns) > budget:
+ lines = lines[:max(0, budget)]
+ return lines
+
+
def _fit(left, right, cols):
"""left, then right flush with column cols; left is cut with an ellipsis when both don't fit."""
room = cols - (len(right) + 2 if right else 0)
@@ -2049,64 +2285,150 @@ def terminal_size():
return os.terminal_size((size.columns or 80, size.lines or 24))
-# Known limitations: narrowing a reflowing terminal can leave stale fragments above the region, because the cursor-up
-# count is the number of lines drawn at the old width and the terminal has rewrapped them into more.
+# Known limitations: (1) a terminal that does not reflow on resize keeps the old lines' wrapping, so after a narrowing
+# the cursor-up count (which assumes rewrapping at the new width) can overshoot into scrollback or leave a fragment; the
+# next frame repaints from wherever the cursor lands. (2) Output that bypasses this renderer (a stray print or a child
+# writing to the tty) moves the cursor without updating widths, so the whole frame above it goes stale until close():
+# every frame line is redrawn from a wrong origin, not just the last one. Child output goes through the event pipe or
+# the pause()/passthrough paths for this reason.
class LiveRenderer:
- """Finished blocks print permanently; running rows and their modules redraw in place below them, at most once per
- REDRAW_SECONDS, and never taller than the terminal's rows minus 2."""
+ """The live view: one frame — the session header, the transcript of finished blocks, then the running rows,
+ pending rows and Overall — redrawn in place at most once per REDRAW_SECONDS and folded by fit_view to the
+ terminal's rows minus 2. Nothing reaches scrollback while the frame is live; pause() and close() print the header
+ and the whole transcript there, uncut."""
REDRAW_SECONDS = 0.1
- def __init__(self, write, size=terminal_size, clock=time.monotonic, color=True):
+ def __init__(self, write, size=terminal_size, clock=time.monotonic, color=True, sticky=()):
self.write = write
self.size = size
self.clock = clock
self.color = color
self.glyphs = COLOR_GLYPHS if color else NO_COLOR_GLYPHS
self.dots = COLOR_DOT_GLYPHS if color else DOT_GLYPHS # one per test class on a module's sub-lines
- self.height = 0 # lines of the region now on screen
+ self.header = [] # this session's header blocks
+ self.sticky = list(sticky) # a parent session's header blocks, shown while this one has none (nested)
+ self.header_printed = False
+ self.transcript = [] # (kind, block, lines): finished blocks, rendered once when they arrive
+ self.cut = None # (cols, [lines cut to cols, one list per transcript entry]); None after an insert
+ self.widths = [] # display width of each frame line on screen, for the cursor-up count
+ self.partial = 0 # display width of a prompt's partial line under the frame
+ self.height = 0 # lines of the frame now on screen (kept for callers and tests)
+ self.last_frame = None
+ self.last_size = None
self.last_draw = None
self.paused = False
self.prompting = False # a watchdog prompt's partial line is on screen
self.cursor_hidden = False
- self.closed = False # after close() permanent blocks still print, but nothing redraws
+ self.closed = False # after close() blocks print at once, and nothing redraws
self.board = None
- self.flushed = 0 # blocks of the last render() call already printed (Console._fallback)
+ self.flushed = 0 # blocks of the last render() call already taken (Console._fallback)
+ self.prompt_at = 0 # the transcript position a prompt joins when its line ends
+ self.prompt_text = "" # the prompt and its echo, as written under the frame
def render(self, board, blocks):
self.board = board
self.flushed = 0
- lines = []
for i, block in enumerate(blocks):
kind = block[0]
if kind == "raw":
- self._flush(lines)
- lines = []
- self.flushed = i
- self._clear()
- self.write(block[1])
- self.flushed = i + 1
- self.prompting = not block[1].endswith("\n")
- if self.prompting:
- self._show_cursor() # the user types at the prompt; the next _draw hides it again
+ self._raw(block) # Task 10
+ elif self.closed:
+ self.write("".join(self._tint(line) + "\n" for line in self._block_lines(block, self._cols())))
elif kind == "header":
- lines += header_lines(block, self.size().columns - 1, self.color)
- elif kind == "text":
- lines.append(block[1])
- elif kind == "row":
- lines.append(self._row(board, block[1], self.size().columns - 1))
- elif kind == "setup":
- lines.append(setup_line(board, self.glyphs, self.size().columns - 1, self.clock(), self.color))
- elif kind == "hint":
- lines += hint_lines(block)
- elif kind == "module":
- lines.append(module_line(block[2], self.glyphs, self.clock()))
- elif kind == "final":
- lines.append(format_final(*block[1:]))
- self._flush(lines)
- self.flushed = len(blocks)
+ self.header.append(block)
+ else:
+ # rendered now, once: setup and module lines read the clock and board.setup, which move on, and a row
+ # keeps the label column it finished with when a longer label starts later
+ self.transcript.append((kind, block, self._block_lines(block, self._cols())))
+ self.flushed = i + 1
self._draw(force=bool(blocks))
+ def _cols(self):
+ return self.size().columns - 1
+
+ def _block_lines(self, block, cols, live=False):
+ """A finished block's lines, as scrollback prints them (live cuts the header's title and cut lines)."""
+ kind = block[0]
+ if kind == "header":
+ return header_lines(block, cols, self.color, live=live)
+ if kind == "text":
+ return block[1].split("\n")
+ if kind == "row":
+ return self._row(self.board, block[1], cols).split("\n")
+ if kind == "setup":
+ return [setup_line(self.board, self.glyphs, cols, self.clock(), self.color)]
+ if kind == "hint":
+ return hint_lines(block)
+ if kind == "module":
+ return [module_line(block[2], self.glyphs, self.clock())]
+ if kind == "final":
+ return [final_line(block, self.color)]
+ return []
+
+ def _raw(self, block):
+ """A watchdog prompt: the frame stays above it (or clears when the prompt would not leave it a line), the
+ prompt and its echo are written under it, and nothing redraws until the line ends; then the prompt joins the
+ transcript where it started, ahead of the rows that finished while it was up. A block without a role (an older
+ sender) is a prompt; an echo with no prompt up starts one, and an end with no prompt up is dropped, so neither
+ writes under the frame unaccounted.
+
+ Not handled: an echo that scrolls the terminal (a prompt on the bottom row) pushes the frame's top lines into
+ scrollback, where the next redraw cannot reach them, so a stale copy of them stays there; the cursor-up cap
+ (rows - 1) keeps the redraw itself on screen."""
+ text = block[1]
+ role = block[2] if len(block) > 2 else "prompt"
+ if not self.prompting:
+ if role == "end":
+ return
+ self._start_prompt(text)
+ else:
+ self.write(text)
+ self.prompt_text += text
+ self._advance(text)
+ if role == "end" or text.endswith("\n"):
+ self._end_prompt()
+ self.flushed += 1 # the text is on screen: a failing redraw must not make _fallback write it again
+ self._draw(force=True)
+
+ def _start_prompt(self, text):
+ self.prompt_at, self.prompt_text = len(self.transcript), ""
+ size = self.size()
+ k = sum(max(1, -(-_vlen(line) // size.columns)) for line in text.split("\n"))
+ if k >= size.lines - 3:
+ self._clear()
+ else:
+ self._draw(force=True, reserve=k)
+ self.prompting = True
+ self._show_cursor() # the user types at the prompt; the next _draw hides it again
+ self.write(text)
+ self.prompt_text = text
+ self._advance(text)
+
+ def _advance(self, text):
+ """Account for text written under the frame: each newline closes a line of the partial's width."""
+ for ch in text:
+ if ch == "\n":
+ self.widths.append(self.partial)
+ self.partial = 0
+ elif ch != "\r":
+ self.partial += _cw(ch)
+
+ def _end_prompt(self):
+ """The prompt's line is over: end it with a newline if nothing has (close() and dump() reach here mid-line,
+ as on Ctrl-C at the prompt), count it, and record the prompt in the transcript where it started.
+ Counting an unended partial as a whole line would move the next cursor-up one line too far, erasing the line
+ above the frame."""
+ if not self.prompt_text.endswith("\n"):
+ self.write("\n")
+ self._advance("\n")
+ self.prompting = False
+ text = self.prompt_text.rstrip("\n")
+ if text:
+ self.transcript.insert(self.prompt_at, ("text", ("text", text), text.split("\n")))
+ self.cut = None # the cut cache is by position; an insert shifts it
+ self.prompt_text = ""
+
def _row(self, board, r, cols, pending=False):
now = self.clock()
cells, count, text, elapsed, cut = row_text(r, self.glyphs, now, pending)
@@ -2121,72 +2443,99 @@ class LiveRenderer:
rest = f"{rest} · {r.progress}" if rest else r.progress
return _fit_row(label, cells, rest, elapsed, cols, wrap=bool(r.unit_kind))
- def _flush(self, lines):
- if lines:
- self._clear()
- if self.prompting:
- self.write("\n") # end the prompt's partial line first; the prompt is no longer on it
- self.prompting = False
- self.write("".join(self._tint(line) + "\n" for line in lines))
+ def frame(self, cols, columns, budget):
+ """The frame's lines for a cols-wide pane of budget lines (columns is the terminal width a log line wraps
+ at)."""
+ runs, pending, overall = self.region_parts(self.board, cols)
+ return fit_view(self._header_parts(cols), self._groups(cols), runs, pending, overall, budget, cols, columns,
+ self.color)
+
+ _LOG_RANK = {"Condensed": 0, "Output": 1, "Cancel": 2} # Stage B5 drops the header's log lines in this order
+
+ def _header_parts(self, cols):
+ """(title line or None, [(text, wraps, drop rank)]): a cut line ranks -1, so it drops before any log line."""
+ title, head = None, []
+ for block in self.header or self.sticky:
+ text = header_lines(block, cols, self.color, live=True)[0]
+ if len(block) > 2 and block[2] == HEAD_MARK and title is None:
+ title = text
+ elif len(block) > 3:
+ head.append((text, False, -1))
+ else:
+ head.append((text, True, self._LOG_RANK.get(str(block[1]).split(":", 1)[0], 0)))
+ return title, head
+
+ def _groups(self, cols):
+ """The transcript as _Groups: a Setup line joins the row after it, a row opens a group (fail, warn or ok, by
+ row_state), and text, hints and module lines join the open group (text before any row opens an ok group).
+ The final block is not part of the frame. Every line is cut to cols with an ellipsis, once per entry and width
+ (self.cut), so a tick costs a pass over cached lists, not a re-render of the transcript."""
+ def cut(line):
+ return line if _vlen(line) <= cols else _vcut(line, max(0, cols - 1)) + "…"
+
+ if self.cut is None or self.cut[0] != cols:
+ self.cut = (cols, [])
+ done = self.cut[1]
+ for kind, block, lines in self.transcript[len(done):]:
+ done.append([cut(line) for line in lines])
+ groups, setup = [], []
+ for (kind, block, _), lines in zip(self.transcript, done):
+ if kind == "final":
+ continue
+ if kind == "setup":
+ setup += lines
+ elif kind == "row":
+ state = row_state(block[1])
+ groups.append(_Group(state if state in ("fail", "warn") else "ok", block[1].label, setup + lines, []))
+ setup = []
+ else:
+ if not groups:
+ groups.append(_Group("ok", None, [], []))
+ groups[-1].body = groups[-1].body + lines # never extend a cached list in place
+ if setup:
+ groups.append(_Group("ok", None, setup, []))
+ return groups
def tick(self, board):
self.board = board
self._draw()
- def region(self, board, cols, max_lines):
- """The running rows in start order (each with its Setup row and a line per running unit), then the pending
- rows in plan order, then Overall. When it is taller than max_lines the pending rows fold first (into
- "○ +N more pending", kept while anything is pending), then the unit lines, newest first, onto one packed
- names line per row, then the lines above Overall are cut: Overall itself is never dropped."""
+ def region_parts(self, board, cols):
+ """The region unfolded, as fit_view takes it: a _Run per running row in start order (its Setup row when it
+ owns the running Setup, its row, its current-file line, and a _Unit per running unit), the pending rows'
+ labels in plan order, and the Overall line or None."""
running = board.running()
now = self.clock()
- subs = [(r, m) for r in running for m in r.units.values() if m.in_progress and m.id not in r.unbarred]
current = {r.id: " " + _vcut(r.progress, max(1, cols - 4)) for r in running
if r.progress and r.unit_kind in ("tests", "files") and not r.units_running} # no unit line to show
setup_running = (board.setup is not None and not board.setup.ended
and any(r.id == board.setup.row for r in running))
- overall = board.overall()
- row_text_ = {r.id: self._row(board, r, cols, pending=True) for r in running}
- row_lines = sum(t.count("\n") + 1 for t in row_text_.values()) # a wrapped unit bar spans several lines
- fixed = row_lines + len(current) + (1 if setup_running else 0) + (1 if overall else 0)
- pending = board.pending()
- shown = len(pending)
- if pending and fixed + len(subs) + shown > max_lines:
- shown = max(0, max_lines - fixed - len(subs) - 1)
- if shown + 1 >= len(pending):
- shown = len(pending)
- pending_lines = [_vcut(pending_line(label, self.color), cols) for _, label in pending[:shown]]
- if shown < len(pending):
- pending_lines.append(pending_line(f"+{len(pending) - shown} more pending", self.color))
- budget = max_lines - fixed - len(pending_lines)
- folded = {} # row id -> its folded units: the newest started go first, so the slow ones stay in view
- gone = set()
- # newest first; of units started at the same instant the one listed later (started later) folds first
- for i in sorted(range(len(subs)), key=lambda i: (subs[i][1].t0, i), reverse=True):
- if len(subs) - len(gone) + len(folded) <= budget:
- break
- gone.add(i)
- folded.setdefault(subs[i][0].id, []).append(subs[i][1])
- subs = [rm for i, rm in enumerate(subs) if i not in gone]
- lines = []
+ runs, i = [], 0
for r in running:
- if setup_running and board.setup.row == r.id:
- lines.append(setup_line(board, self.glyphs, cols, self.clock(), self.color))
- lines.append(row_text_[r.id])
+ head = [setup_line(board, self.glyphs, cols, now, self.color)] if setup_running and board.setup.row == r.id else []
+ head += self._row(board, r, cols, pending=True).split("\n") # a wrapped unit bar spans several lines
if r.id in current:
- lines.append(current[r.id])
- mine = [m for owner, m in subs if owner is r]
- width = max((_vlen(m.title) for m in mine), default=1) # _sub_lines caps it
- lines += [line for m in mine for line in self._sub_lines(m, cols, width, now)]
- if r.id in folded:
- names = [m.title for m in reversed(folded[r.id])] # folded newest first: listed oldest first
- prefix = common_prefix([m.title for m in r.units.values() if m.id not in r.unbarred])
- lines.append(" " + collapse_names([t[len(prefix):] or t for t in names], cols - 2))
- lines += pending_lines
- lines = [part for line in lines for part in line.split("\n")]
- if not overall:
- return lines[:max(1, max_lines)]
- return lines[:max(0, max_lines - 1)] + [_vcut(f"{head_mark(HEAD_MARK, self.color)} {overall_text(overall)}", cols)]
+ head.append(current[r.id])
+ mine = [m for m in r.units.values() if m.in_progress and m.id not in r.unbarred]
+ width = max((_vlen(m.title) for m in mine), default=1) # _sub_lines caps it
+ prefix = common_prefix([m.title for m in r.units.values() if m.id not in r.unbarred])
+ units = []
+ for m in mine:
+ units.append(_Unit(m.t0, i, self._sub_lines(m, cols, width, now), m.title[len(prefix):] or m.title))
+ i += 1
+ runs.append(_Run(head, units))
+ overall = board.overall()
+ line = _vcut(f"{head_mark(HEAD_MARK, self.color)} {overall_text(overall)}", cols) if overall else None
+ return runs, [label for _, label in board.pending()], line
+
+ def region(self, board, cols, max_lines=None):
+ """A test-facing shim, not used by the renderer: the running rows, pending rows and Overall, unfolded, or with
+ max_lines folded by fit_view alone (no header, no transcript), as the region tests that size it themselves
+ expect. The live frame is built by frame()."""
+ runs, pending, overall = self.region_parts(board, cols)
+ if max_lines is None:
+ return [t for t, _ in _Fold((None, []), [], runs, pending, overall, cols, self.color).lines()]
+ return fit_view((None, []), [], runs, pending, overall, max_lines, cols, cols + 1, self.color)
def _sub_lines(self, m, cols, width=None, now=None):
"""A unit's line: its title padded to width (its row's longest shown title, at most half the line and room for
@@ -2214,42 +2563,88 @@ class LiveRenderer:
line = prefix + strip + " " * max(2, cols - _vlen(prefix) - _vlen(strip) - len(elapsed)) + elapsed
return [line if _vlen(line) <= cols else _vcut(line, cols)] # a terminal too narrow for even one dot
- def _draw(self, force=False):
+ def _draw(self, force=False, reserve=0):
+ """Redraw the frame in place. reserve keeps that many lines free under it (a prompt about to be written)."""
if self.closed or self.paused or self.prompting or self.board is None:
return
now = self.clock()
if not force and self.last_draw is not None and now - self.last_draw < self.REDRAW_SECONDS:
return
size = self.size()
- lines = self.region(self.board, size.columns - 1, size.lines - 2)
+ budget = size.lines - 2 - reserve
+ if budget < 1:
+ self._clear() # not even one line fits: draw nothing until the pane grows
+ return
+ lines = self.frame(size.columns - 1, size.columns, budget)
+ if lines == self.last_frame and size == self.last_size:
+ return
out = []
if not self.cursor_hidden:
out.append("\x1b[?25l")
self.cursor_hidden = True
- if self.height:
- out.append(f"\x1b[{self.height}A")
+ up = self._up(size)
+ if up:
+ out.append(f"\r\x1b[{up}A")
out += [self._tint(line) + "\x1b[K\n" for line in lines]
out.append("\x1b[J")
self.write("".join(out))
- self.height = len(lines)
- self.last_draw = now
+ self.widths, self.partial, self.height = [_vlen(line) for line in lines], 0, len(lines)
+ self.last_frame, self.last_size, self.last_draw = lines, size, now # only a written frame moves the throttle
+
+ def _up(self, size):
+ """Lines from the cursor back to the frame's first line at the terminal's current width (a reflowing terminal
+ rewraps a line that no longer fits), capped at rows - 1, where the cursor stops. A line exactly as wide as
+ the terminal takes one line: the cursor waits at its end (pending wrap), and the newline after it moves down
+ once. The partial line under the frame is on the cursor's own line, so only its wrapped lines above count."""
+ cols = size.columns
+ rows = sum(max(1, -(-w // cols)) for w in self.widths) + (max(0, self.partial - 1) // cols)
+ return min(rows, max(0, size.lines - 1))
def _tint(self, line):
return tint(line) if self.color else line
def _clear(self):
- if self.height:
- self.write(f"\x1b[{self.height}A\x1b[J")
- self.height = 0
+ if self.widths or self.partial:
+ up = self._up(self.size())
+ self.write("\r" + (f"\x1b[{up}A" if up else "") + "\x1b[J")
+ self.widths, self.partial, self.height, self.last_frame = [], 0, 0, None
def _show_cursor(self):
if self.cursor_hidden:
self.write("\x1b[?25h")
self.cursor_hidden = False
+ def _scrollback_lines(self):
+ """The header (once per session, wrapped, never the sticky one) and the whole transcript uncut, taken: the
+ transcript is emptied, so the frame goes on with only the live part."""
+ cols = self._cols()
+ out = []
+ if self.header and not self.header_printed:
+ for block in self.header:
+ out += self._block_lines(block, cols)
+ self.header_printed = True
+ transcript, self.transcript, self.cut = self.transcript, [], None
+ for kind, block, lines in transcript:
+ out += lines
+ return out
+
+ def _print_scrollback(self):
+ lines = self._scrollback_lines()
+ if lines:
+ self.write("".join(self._tint(line) + "\n" for line in lines))
+
+ def dump(self):
+ """The record the frame held (as _scrollback_lines), untinted and taken, ending a prompt first, for
+ Console._fallback to print plainly."""
+ if self.prompting:
+ self._end_prompt()
+ return [_ANSI.sub("", line) for line in self._scrollback_lines()]
+
def pause(self):
- """Clear the region and give the terminal back (ask, show, passthrough, handoff)."""
+ """Clear the frame, print the record into scrollback and give the terminal back (ask, show, passthrough,
+ handoff)."""
self._clear()
+ self._print_scrollback()
self._show_cursor()
self.paused = True
@@ -2260,7 +2655,10 @@ class LiveRenderer:
def close(self):
if self.closed:
return
+ if self.prompting:
+ self._end_prompt() # ends the prompt's line with a newline first if nothing has (Task 10)
self._clear()
+ self._print_scrollback()
self._show_cursor()
self.closed = True
@@ -2277,6 +2675,9 @@ class Console:
self.size = size
self.board = Board(resolve_detail(detail), nested=nested, full_log=self.sinks.full_path, clock=clock,
estimator=make_estimator(repo, script_name(), history_key=history_key))
+ self.handback = os.environ.pop(HANDBACK_ENV, None) if self.board.nested else None # a grandchild must not write it
+ self.sticky = _parent_header() if nested else [] # drawn by the live screen only, never by the Board
+ self.header_blocks = [] # this session's header blocks, for a handoff child (HEADER_ENV)
self.mode = self.screen_mode()
self.screen = self._make_screen()
self._log = None
@@ -2309,7 +2710,7 @@ class Console:
return None
if self.mode == "live":
return LiveRenderer(self._screen_write, size=self.size, clock=self.clock,
- color=not os.environ.get("NO_COLOR"))
+ color=not os.environ.get("NO_COLOR"), sticky=self.sticky)
return PlainRenderer(self._screen_write, heartbeat=True, clock=self.clock)
def _screen_write(self, text):
@@ -2328,6 +2729,8 @@ class Console:
self._append_full(kind, fields)
if self.board_failed:
self._plain(kind, fields, self._final_blocks(kind, fields), owned)
+ if kind == "done":
+ self._done_nested()
return
try:
blocks = self.board.apply(kind, fields)
@@ -2335,7 +2738,11 @@ class Console:
self.board_failed = True # view and the log stop for the rest of the run (the full log has everything)
self._stop(e, log=True)
self._plain(kind, fields, self._final_blocks(kind, fields), owned)
+ if kind == "done":
+ self._done_nested()
return
+ if kind == "session":
+ self.header_blocks = [b for b in blocks if b[0] == "header"]
unshown = blocks
if self.screen is not None:
try:
@@ -2350,6 +2757,8 @@ class Console:
except Exception as e: # the condensed log is a convenience; the full log still has everything
self.log = None
self._note(f"⚠ condensed log stopped ({e!r})")
+ if kind == "done":
+ self._done_nested()
self._plain(kind, fields, unshown, owned)
# The script text each event carries, as the full log gets it: every level, whatever the view shows (spec 5.1).
@@ -2388,6 +2797,31 @@ class Console:
elif self.stopped:
self._plain_write(text)
+ def log_only(self, text):
+ """One line in the condensed log and nowhere else (a note about the view itself), best-effort."""
+ if self.log is not None:
+ try:
+ self.log.render(self.board, [("text", text)])
+ except Exception: # the condensed log is a convenience
+ pass
+
+ def _done_nested(self):
+ """At a nested session's done: hand its rows' outcomes back to the session that handed off to it
+ (self.handback), and note in the condensed log when it declared no plan under a parent's Overall, which then
+ has no row on screen. Best-effort: the Board may be the thing that failed."""
+ if not self.board.nested:
+ return
+ try:
+ if self.handback:
+ # a sub-row is a unit on its parent's row, never a row, so every row here is top-level
+ rows = [[r.id, "fail" if r.running else "warn" if r.status == "ok" and r.warned else r.status, r.label]
+ for r in self.board.rows.values()]
+ _write_atomic(Path(self.handback), json.dumps(rows))
+ if self.board.parent_overall is not None and not self.board.plan:
+ self.log_only("ℹ this script declared no plan, so its view has no Overall row")
+ except Exception: # never a reason to fail the run; the parent notes that nothing came back
+ pass
+
def _final_blocks(self, kind, fields):
"""The final line's block for a done event, from what the failed Board still knows, best-effort."""
if kind != "done":
@@ -2406,7 +2840,7 @@ class Console:
self._plain_write(fields["text"])
for block in unshown:
if block[0] == "final":
- self._plain_write(format_final(*block[1:]))
+ self._plain_write(final_line(block))
def _plain_write(self, text):
try:
@@ -2415,18 +2849,22 @@ class Console:
pass
def _fallback(self, error, blocks):
- """Swap the screen renderer for a plain one, say so, and re-render the blocks it had not printed. A plain
- screen that fails, or a plain re-render that fails too (the bug is in code both renderers share), stops the
- view instead; the condensed log goes on."""
+ """Swap the screen renderer for a plain one: take what the live frame held, close it, then print that
+ record, the warning and the blocks it had not taken. A plain screen that fails, or a plain re-render that
+ fails too (the bug is in code both renderers share), stops the view instead; the condensed log goes on."""
if self.mode != "live":
self._stop(error)
return
+ try:
+ held = self.screen.dump()
+ except Exception: # the record is lost with the broken renderer; the full log still has it
+ held = []
self._close_screen()
self.mode = "plain"
self.screen = PlainRenderer(self._screen_write, heartbeat=True, clock=self.clock)
try:
- self.screen.render(self.board, [("text", f"⚠ console view failed ({error!r}); using plain output")]
- + blocks)
+ self.screen.render(self.board, [("text", line) for line in held]
+ + [("text", f"⚠ console view failed ({error!r}); using plain output")] + blocks)
except Exception as e:
self._stop(e)
@@ -2456,8 +2894,8 @@ class Console:
def _close_screen(self):
"""Close the screen renderer as best it can, ending a watchdog prompt's partial line first."""
try:
- if getattr(self.screen, "prompting", False):
- self._screen_write("\n")
+ if isinstance(self.screen, PlainRenderer) and self.screen.prompting:
+ self._screen_write("\n") # LiveRenderer.close() ends its own prompt line (_end_prompt)
self.screen.close()
except Exception: # the broken renderer may not close cleanly either; at least show the cursor again
try:
@@ -2497,9 +2935,10 @@ def _sigterm(signum, frame):
raise KeyboardInterrupt
-def session(title, header=(), *, detail=None, sinks=None, repo=".", history_key=None):
+def session(title, header=(), *, lines=(), detail=None, sinks=None, repo=".", history_key=None):
"""Start this process's console view and return its Console. Inside another session (JUNEAU_RUN_SESSION set)
- it is nested: no header and no final line, so a child script's rows join its parent's view."""
+ it is nested: no header and no final line, so a child script's rows join its parent's view. `lines` are header
+ lines under the title, one each, cut to the width on the live screen (a commit message, a path)."""
global _console
if _console is not None:
return _console
@@ -2524,7 +2963,7 @@ def session(title, header=(), *, detail=None, sinks=None, repo=".", history_key=
more.append(f"Condensed: tail -f {shlex.quote(str(sinks.condensed_path))}")
if os.environ.get(CANCEL_ENV):
more.append(f"Cancel: {os.environ[CANCEL_ENV]}")
- publish("session", title=title, header=list(header), header_more=more, nested=nested)
+ publish("session", title=title, header=list(header), lines=[str(t) for t in lines], header_more=more, nested=nested)
return _console
@@ -2801,8 +3240,10 @@ def passthrough(cmd, step_id, title, *, n=None, parent=None, cwd=None, env=None,
@contextmanager
def handoff():
"""Give the terminal to a child session (push.py running test.py) for the block. Yields the env the child needs
- so its rows line up with ours (label width, detail level) and our pending rows stay visible under its own
- (JUNEAU_RUN_PENDING) and its Overall line carries on (JUNEAU_RUN_OVERALL); empty with no session."""
+ so its rows line up with ours (label width, detail level), our pending rows stay visible under its own
+ (JUNEAU_RUN_PENDING), its Overall line carries on (JUNEAU_RUN_OVERALL), our header stays on top of its view
+ (JUNEAU_RUN_HEADER), and it hands back its rows' outcomes for our Overall and final counts (JUNEAU_RUN_HANDBACK);
+ empty with no session."""
if _console is None:
yield {}
return
@@ -2813,8 +3254,38 @@ def handoff():
overall = _console.board.overall_handoff()
if overall:
env[OVERALL_ENV] = json.dumps(overall)
- with _paused():
- yield env
+ header = _console.header_blocks or _console.sticky # a nested session passes its parent's on
+ if header:
+ env[HEADER_ENV] = json.dumps([["head" if b[2] == HEAD_MARK else "more", b[1], len(b) > 3]
+ for b in header if len(b) > 2])
+ fd, handback = tempfile.mkstemp(prefix="juneau-run-handback-", suffix=".json")
+ os.close(fd)
+ env[HANDBACK_ENV] = handback
+ try:
+ with _paused():
+ try:
+ yield env
+ except BaseException:
+ _take_handback(handback, note=False) # Ctrl-C or a failure: count what came back, but say nothing
+ raise
+ _take_handback(handback) # before resume, so the redrawn view already counts the child's steps
+ finally:
+ with suppress(OSError):
+ os.unlink(handback)
+
+
+def _take_handback(handback, note=True):
+ """Record a handoff child's outcomes on this session's Board, or (note) say once in the condensed log that it
+ sent none (it crashed, or runs an older juneau-run): its steps are then not counted, as before. The note is
+ left out when the handoff ended in an exception, which has its own report."""
+ try:
+ outcomes = [(str(i), str(status), str(label))
+ for i, status, label in json.loads(Path(handback).read_text(encoding="utf-8"))]
+ except (OSError, ValueError, TypeError):
+ if note:
+ _console.log_only("ℹ the handed-off script reported no step outcomes; Overall does not count its steps")
+ return
+ _console.board.hand_back(outcomes)
# ---------------------------------------------------------------------------------------------------------------------
@@ -3218,19 +3689,58 @@ def history_series(name, values):
def make_estimator(repo, script, store=None, history_key=None):
"""The Board's estimator: fn(step, tool, sig) -> Estimate or None, from this script's history in `repo` (spec 5.3).
The store is read afresh for each lookup. A `tool` of None takes the tool of the newest record for the step that
- matches `sig` (else the newest record's). Like-for-like records are those with the same `sig` (spec 5.1). Off
- when history is disabled."""
+ matches `sig` (else the newest record's). Like-for-like records are those with the same `sig` (spec 5.1). When
+ this script's records give no estimate for the step, the other scripts' histories in the same directory are asked
+ (push.py's pending rows for the steps test.py records): the one whose newest record for the step is the latest
+ wins. Step ids are assumed unique across one repo's scripts. Off when history is disabled."""
if not history_enabled():
return lambda step, tool, sig=None: None
store = store or HistoryStore(repo, script, history_key)
- def estimator(step, tool, sig=None):
- records = [r for r in store.read() if r["step"] == step]
+ def lookup(records, tool, sig):
if tool is None and records:
same = [r for r in records if sig and r.get("sig") == sig]
tool = (same or records)[-1]["tool"]
- records = [r for r in records if r["tool"] == tool]
- return estimate_from(records, sig)
+ return estimate_from([r for r in records if r["tool"] == tool], sig)
+
+ cache = {} # sibling path -> ((mtime_ns, size), records): the Board asks per pending row, per redraw
+
+ def sibling_read(sibling):
+ try:
+ st = sibling.stat()
+ except OSError:
+ return []
+ key = (st.st_mtime_ns, st.st_size)
+ hit = cache.get(sibling)
+ if hit is None or hit[0] != key:
+ hit = cache[sibling] = (key, HistoryStore(repo, sibling.stem, history_key).read())
+ return hit[1]
+
+ def sibling_records(step):
+ """The records for step from the sibling whose newest one is the latest. A sibling file is re-read only when
+ its size or mtime changed (a sibling script appending while this one runs)."""
+ path = getattr(store, "path", None)
+ best = None
+ if path is None:
+ return best
+ try:
+ siblings = sorted(path.parent.glob("*.jsonl"))
+ except OSError:
+ return best
+ for sibling in siblings:
+ if sibling == path:
+ continue
+ records = [r for r in sibling_read(sibling) if r["step"] == step]
+ if records and (best is None or records[-1].get("ts", "") > best[-1].get("ts", "")):
+ best = records
+ return best
+
+ def estimator(step, tool, sig=None):
+ own = lookup([r for r in store.read() if r["step"] == step], tool, sig)
+ if own is not None:
+ return own
+ records = sibling_records(step)
+ return lookup(records, tool, sig) if records else None
return estimator
diff --git a/juneau-run/src/main/python/juneau_scripts/run.py b/juneau-run/src/main/python/juneau_scripts/run.py
index 9dc2b4ccd8..bf9315cf76 100644
--- a/juneau-run/src/main/python/juneau_scripts/run.py
+++ b/juneau-run/src/main/python/juneau_scripts/run.py
@@ -516,7 +516,7 @@ def _n_base():
@contextlib.contextmanager
-def session(title, *, header=(), bell=False, sound=False, silence_env=(), mode=None, project=None, repo=".",
+def session(title, *, header=(), lines=(), bell=False, sound=False, silence_env=(), mode=None, project=None, repo=".",
exit=True, plan=None, history_key=None):
"""
One script run: the run marker, the console view when requested, a SIGTERM handler, and on every path the summary,
@@ -524,7 +524,8 @@ def session(title, *, header=(), bell=False, sound=False, silence_env=(), mode=N
other exception 1 (reported as "❌ ERROR: <Type>: <msg>"). With exit=True it raises SystemExit(code) at the end;
with exit=False the code is left in s.code. plan is the list of top-level step ids for the Overall figure; the run's
history is recorded for repo (JUNEAU_RUN_HISTORY=0 turns it off). history_key names that history instead of
- the repo's, so separate clones can share one; JUNEAU_RUN_HISTORY_KEY is the fallback.
+ the repo's, so separate clones can share one; JUNEAU_RUN_HISTORY_KEY is the fallback. lines are header lines under
+ the title (juneau_run.session).
"""
global _session, _n
if _session is not None:
@@ -550,7 +551,7 @@ def session(title, *, header=(), bell=False, sound=False, silence_env=(), mode=N
jr.run(mode, project, git.current_branch(repo) or "HEAD", git.short_head(repo))
if console_requested():
opened_view = not jr.session_open() if hasattr(jr, "session_open") else False
- jr.session(title, header, repo=repo, **keyed)
+ jr.session(title, header, lines=tuple(lines), repo=repo, **keyed)
if plan is not None:
s.plan(plan)
yield s
diff --git a/juneau-run/src/test/python/support.py b/juneau-run/src/test/python/support.py
index 52d68408b0..307427ae80 100644
--- a/juneau-run/src/test/python/support.py
+++ b/juneau-run/src/test/python/support.py
@@ -55,7 +55,8 @@ def reset_run_state():
jr._console = None
for key in (jr.ACTIVE_ENV, "JUNEAU_RUN_FULL_LOG", "JUNEAU_RUN_SESSION", "JUNEAU_RUN_DETAIL", "JUNEAU_RUN_LABEL_WIDTH",
"JUNEAU_RUN_LIVE", "JUNEAU_RUN_CONSOLE", "RUN_MARKERS", "JUNEAU_RUN_CONDENSED_LOG",
- "JUNEAU_RUN_PENDING", "JUNEAU_RUN_OVERALL", jr.CANCEL_ENV, jr.MARKERS_FILE_ENV, jr.RESULT_ENV, jr.RESULT_PARTS_ENV):
+ "JUNEAU_RUN_PENDING", "JUNEAU_RUN_OVERALL", "JUNEAU_RUN_HEADER", "JUNEAU_RUN_HANDBACK",
+ jr.CANCEL_ENV, jr.MARKERS_FILE_ENV, jr.RESULT_ENV, jr.RESULT_PARTS_ENV):
os.environ.pop(key, None)
@@ -79,17 +80,25 @@ def isolate_run_env(case):
class Screen:
"""A minimal VT100 model, enough for what LiveRenderer writes: text, \\r, \\n, ESC[nA, ESC[K and ESC[J.
SGR colour is ignored; cursor show/hide sets cursor_visible. clamped is set if a cursor-up ever went above the first
- line."""
+ line. With rows (and cols) it is a viewport: a \\n on the last row scrolls the top line into scrollback, a line
+ autowraps at cols, and a wide or emoji code point takes two columns (jr._cw), as the renderer assumes. start_row
+ puts the cursor that far down first, as a run started near the bottom of the window."""
_CSI = re.compile(r"\x1b\[(\??)(\d*)([A-Za-z])")
- def __init__(self):
- self.lines = [""]
- self.row = 0
+ def __init__(self, rows=None, cols=None, start_row=0):
+ self.rows, self.cols = rows, cols
+ self.cells = [[] for _ in range(start_row + 1)] # "" marks a wide character's second column
+ self.row = start_row
self.col = 0
+ self.scrollback = []
self.clamped = False
self.cursor_visible = True
+ @property
+ def lines(self):
+ return ["".join(self.cells[i]) for i in range(len(self.cells))]
+
def feed(self, data):
i = 0
while i < len(data):
@@ -102,16 +111,27 @@ class Screen:
if ch == "\r":
self.col = 0
elif ch == "\n":
- self.row += 1
- self.col = 0
- if self.row == len(self.lines):
- self.lines.append("")
+ self._newline()
else:
- line = self.lines[self.row].ljust(self.col)
- self.lines[self.row] = line[:self.col] + ch + line[self.col + 1:]
- self.col += 1
+ w = jr._cw(ch) if self.rows else 1
+ if self.cols and self.col + w > self.cols:
+ self._newline()
+ line = self.cells[self.row]
+ line.extend([" "] * (self.col - len(line)))
+ line[self.col:self.col + w] = [ch] + [""] * (w - 1)
+ self.col += w
i += 1
+ def _newline(self):
+ self.col = 0
+ if self.rows and self.row == self.rows - 1:
+ self.scrollback.append("".join(self.cells.pop(0)).rstrip())
+ self.cells.append([])
+ return
+ self.row += 1
+ if self.row == len(self.cells):
+ self.cells.append([])
+
def _csi(self, private, n, cmd):
if private:
if n == 25 and cmd in "hl":
@@ -123,10 +143,23 @@ class Screen:
self.clamped = self.clamped or n > self.row
self.row = max(0, self.row - n)
elif cmd == "K":
- self.lines[self.row] = self.lines[self.row][:self.col]
+ del self.cells[self.row][self.col:]
elif cmd == "J":
- self.lines[self.row] = self.lines[self.row][:self.col]
- del self.lines[self.row + 1:]
+ del self.cells[self.row][self.col:]
+ del self.cells[self.row + 1:]
+
+ def resize(self, rows=None, cols=None):
+ """Change the viewport as a terminal that does not reflow would: lines already written keep their cells, and
+ only later writes wrap and scroll at the new size. Rows that no longer fit go to scrollback."""
+ self.rows, self.cols = rows, cols
+ while self.rows and len(self.cells) > self.rows:
+ self.scrollback.append("".join(self.cells.pop(0)).rstrip())
+ self.row -= 1
+ self.row = max(0, self.row)
def text(self):
return "\n".join(line.rstrip() for line in self.lines).rstrip("\n")
+
+ def history(self):
+ """Everything the terminal holds: the scrollback, then the screen."""
+ return "\n".join(self.scrollback + [self.text()]).strip("\n")
diff --git a/juneau-run/src/test/python/test_estimate.py b/juneau-run/src/test/python/test_estimate.py
index e36a79c97f..68ada3f33c 100644
--- a/juneau-run/src/test/python/test_estimate.py
+++ b/juneau-run/src/test/python/test_estimate.py
@@ -217,6 +217,78 @@ class MakeEstimatorTest(unittest.TestCase):
self.assertEqual(est("mvn", "maven", "cur").wall_s, 100.0)
self.assertEqual(est("mvn", "maven", "other").runs, 4) # fewer than 3 like: falls back to all
+ def stores(self, hist, repo, *scripts):
+ import subprocess
+ subprocess.run(["git", "init", "-q", repo], check=True)
+ return [jr.HistoryStore(repo, s) for s in scripts]
+
+ @staticmethod
+ def add(store, step, wall, ts, ok=True, tool="maven"):
+ store.append({"v": 1, "ts": ts, "step": step, "tool": tool, "ok": ok, "complete": ok, "wall_s": wall})
+
+ def test_a_step_with_no_history_of_its_own_takes_a_sibling_scripts(self):
+ import os, tempfile
+ from unittest import mock
+ with tempfile.TemporaryDirectory() as hist, tempfile.TemporaryDirectory() as repo:
+ with mock.patch.dict(os.environ, {"JUNEAU_RUN_HISTORY": "1", "JUNEAU_RUN_HISTORY_DIR": hist}):
+ push, test = self.stores(hist, repo, "push", "test")
+ self.add(push, "install", 10.0, "2026-10-01T00:00:00Z")
+ for _ in range(3):
+ self.add(test, "tests", 120.0, "2026-10-01T00:00:00Z")
+ est = jr.make_estimator(repo, "push")
+ self.assertEqual(est("tests", "maven").wall_s, 120.0)
+ self.assertEqual(est("tests", None).wall_s, 120.0) # tool left open: the sibling's newest
+ self.assertEqual(est("install", "maven").wall_s, 10.0)
+
+ def test_own_records_with_no_estimate_fall_back_too(self):
+ import os, tempfile
+ from unittest import mock
+ with tempfile.TemporaryDirectory() as hist, tempfile.TemporaryDirectory() as repo:
+ with mock.patch.dict(os.environ, {"JUNEAU_RUN_HISTORY": "1", "JUNEAU_RUN_HISTORY_DIR": hist}):
+ push, test = self.stores(hist, repo, "push", "test")
+ self.add(push, "tests", 5.0, "2026-10-02T00:00:00Z", ok=False) # only a failed run: no estimate
+ self.add(test, "tests", 120.0, "2026-10-01T00:00:00Z")
+ self.assertEqual(jr.make_estimator(repo, "push")("tests", "maven").wall_s, 120.0)
+
+ def test_own_usable_records_win_over_a_newer_sibling(self):
+ import os, tempfile
+ from unittest import mock
+ with tempfile.TemporaryDirectory() as hist, tempfile.TemporaryDirectory() as repo:
+ with mock.patch.dict(os.environ, {"JUNEAU_RUN_HISTORY": "1", "JUNEAU_RUN_HISTORY_DIR": hist}):
+ push, test = self.stores(hist, repo, "push", "test")
+ self.add(push, "tests", 50.0, "2026-10-01T00:00:00Z")
+ self.add(test, "tests", 120.0, "2026-10-09T00:00:00Z")
+ self.assertEqual(jr.make_estimator(repo, "push")("tests", "maven").wall_s, 50.0)
+
+ def test_the_sibling_with_the_latest_record_timestamp_wins(self):
+ import os, tempfile
+ from unittest import mock
+ with tempfile.TemporaryDirectory() as hist, tempfile.TemporaryDirectory() as repo:
+ with mock.patch.dict(os.environ, {"JUNEAU_RUN_HISTORY": "1", "JUNEAU_RUN_HISTORY_DIR": hist}):
+ old, new = self.stores(hist, repo, "aaa", "zzz")
+ self.add(new, "tests", 120.0, "2026-10-01T00:00:00Z")
+ self.add(old, "tests", 90.0, "2026-10-05T00:00:00Z") # written later, though its name sorts first
+ self.assertEqual(jr.make_estimator(repo, "push")("tests", "maven").wall_s, 90.0)
+ self.add(new, "tests", 120.0, "2026-10-07T00:00:00Z")
+ self.assertEqual(jr.make_estimator(repo, "push")("tests", "maven").wall_s, 120.0)
+
+ def test_a_sibling_file_is_read_again_only_when_it_changes(self):
+ import os, tempfile
+ from unittest import mock
+ with tempfile.TemporaryDirectory() as hist, tempfile.TemporaryDirectory() as repo:
+ with mock.patch.dict(os.environ, {"JUNEAU_RUN_HISTORY": "1", "JUNEAU_RUN_HISTORY_DIR": hist}):
+ push, test = self.stores(hist, repo, "push", "test")
+ self.add(test, "tests", 120.0, "2026-10-01T00:00:00Z")
+ est = jr.make_estimator(repo, "push")
+ real = jr.HistoryStore.read
+ with mock.patch.object(jr.HistoryStore, "read", autospec=True, side_effect=real) as read:
+ est("tests", "maven")
+ est("tests", "maven")
+ self.assertEqual(read.call_count, 3) # own store twice, the sibling once
+ self.add(test, "tests", 60.0, "2026-10-02T00:00:00Z")
+ est("tests", "maven")
+ self.assertEqual(read.call_count, 5) # the sibling grew: read again
+
class ExpectedTestsTest(unittest.TestCase):
def rec(self, tests, **kw):
diff --git a/juneau-run/src/test/python/test_render.py b/juneau-run/src/test/python/test_render.py
index f21d20282d..a415fad5f8 100644
--- a/juneau-run/src/test/python/test_render.py
+++ b/juneau-run/src/test/python/test_render.py
@@ -272,17 +272,28 @@ class BoardTest(unittest.TestCase):
def test_header_and_final_line(self):
b = self.board(full_log="/tmp/push-full.log")
blocks = feed(b, ("session", {"title": "Push", "header": ["Juneau", "master", "PGP ✓"], "nested": False}))
- self.assertEqual(plain(b, blocks, self.clock), "▶ Push: Juneau, master, PGP ✓\n")
+ self.assertEqual(plain(b, blocks, self.clock), "▶ Push: Juneau · master · PGP ✓\n")
self.clock.advance(348)
self.assertEqual(plain(b, feed(b, ("done", {"status": "ok", "commit": None})), self.clock),
- "✅ Done in 5:48\n")
+ "▶ Overall ✅ Done in 5:48\n")
def test_final_line_forms(self):
- self.assertEqual(jr.format_final("fail", 0, "Tests", "/tmp/f.log"), "❌ Failed at Tests · full log: /tmp/f.log")
- self.assertEqual(jr.format_final("fail", 0, None, None), "❌ Failed")
- self.assertEqual(jr.format_final("cancelled", 61000, None, None), "⚠ Cancelled after 1:01")
+ f = jr.format_final
+ self.assertEqual(f("ok", 243000, None, None), "Overall ✅ Done in 4:03")
+ self.assertEqual(f("ok", 243000, None, None, 5, 5), "Overall ✅ Done in 4:03 · 5 of 5 steps")
+ self.assertEqual(f("fail", 0, "Tests", "/tmp/f.log"), "Overall ❌ Failed at Tests · full log: /tmp/f.log")
+ self.assertEqual(f("fail", 0, "Docs build", "/tmp/f.log", 9, 13),
+ "Overall ❌ Failed at Docs build · 9 of 13 steps · full log: /tmp/f.log")
+ self.assertEqual(f("fail", 0, None, None), "Overall ❌ Failed")
+ self.assertEqual(f("cancelled", 62000, None, None, 4, 13), "Overall ⚠ Cancelled after 1:02 · 4 of 13 steps")
+ self.assertEqual(f("ok", 0, None, None, 0, 0), "Overall ✅ Done in 0:00") # no counted steps: no counts
self.assertEqual(jr.fmt_elapsed(3723000), "1:02:03")
+ def test_final_line_marks_the_row_cyan_under_colour(self):
+ block = ("final", "ok", 1000, None, None, 1, 1)
+ self.assertEqual(jr.final_line(block), "▶ Overall ✅ Done in 0:01 · 1 of 1 steps")
+ self.assertTrue(jr.final_line(block, color=True).startswith("\x1b[36m▶"))
+
def test_a_nested_board_has_no_header_and_no_final_line(self):
b = self.board(nested=True)
self.assertEqual(feed(b, ("session", {"title": "t", "header": [], "nested": True}),
@@ -464,7 +475,7 @@ class OverallDisplayTest(unittest.TestCase):
for cap in range(1, len(full)):
with self.subTest(cap=cap):
lines = self.live().region(b, 79, cap)
- self.assertEqual(len(lines), cap)
+ self.assertLessEqual(len(lines), cap) # a running row drops with its Setup line
self.assertTrue(lines[-1].startswith("▶ Overall"), lines)
def test_overall_sits_below_the_pending_rows(self):
@@ -528,7 +539,7 @@ class SessionTest(unittest.TestCase):
c = jr.session("Test", ["Juneau", "master", "a message"])
self.assertIs(jr._console, c)
self.assertEqual(os.environ["JUNEAU_RUN_SESSION"], "1")
- self.assertRegex(out.getvalue(), r'^▶ Test: Juneau, master, a message\n▶ Output: tail -f \S+/test-\d{8}-\d{6}(-\d+)?\.log\n'
+ self.assertRegex(out.getvalue(), r'^▶ Test: Juneau · master · a message\n▶ Output: tail -f \S+/test-\d{8}-\d{6}(-\d+)?\.log\n'
r'▶ Condensed: tail -f \S+\.condensed\.log\n$')
def test_a_console_that_fails_to_start_leaves_the_env_unset(self):
@@ -554,7 +565,7 @@ class SessionTest(unittest.TestCase):
with contextlib.redirect_stdout(out):
jr.session("Push", ["Juneau", "master", "Fix it"], sinks=jr.Sinks(full_log=f"{tmp}/push-full.log"))
self.assertEqual(out.getvalue(),
- f'▶ Push: Juneau, master, Fix it\n▶ Output: tail -f {shlex.quote(tmp + "/push-full.log")}\n')
+ f'▶ Push: Juneau · master · Fix it\n▶ Output: tail -f {shlex.quote(tmp + "/push-full.log")}\n')
def test_the_condensed_tail_command_takes_its_own_line_after_the_full_logs(self):
with tempfile.TemporaryDirectory() as tmp:
@@ -566,12 +577,12 @@ class SessionTest(unittest.TestCase):
def test_each_scripts_header_shape(self):
shapes = [ # push.py, push.py --test-only, test.py, release.py
- ("Push", ["Juneau", "master", "Fix it"], "▶ Push: Juneau, master, Fix it"),
- ("Test", ["Juneau", "master"], "▶ Test: Juneau, master"),
- ("Test", ["Juneau", "master"], "▶ Test: Juneau, master"),
- ("Release", ["Juneau", "9.9.9", "Java 17", "Maven 3.9.9"], "▶ Release: Juneau, 9.9.9, Java 17, Maven 3.9.9"),
- ("Push", ["Juneau", "master", "Fix it \n\nlong body\nmore"], "▶ Push: Juneau, master, Fix it …"), # multi-line item: first line, trimmed, then " …"
- ("Push", ["Juneau", "one\ntwo", "master"], "▶ Push: Juneau, one …, master"),
+ ("Push", ["Juneau", "master", "Fix it"], "▶ Push: Juneau · master · Fix it"),
+ ("Test", ["Juneau", "master"], "▶ Test: Juneau · master"),
+ ("Test", ["Juneau", "master"], "▶ Test: Juneau · master"),
+ ("Release", ["Juneau", "9.9.9", "Java 17", "Maven 3.9.9"], "▶ Release: Juneau · 9.9.9 · Java 17 · Maven 3.9.9"),
+ ("Push", ["Juneau", "master", "Fix it \n\nlong body\nmore"], "▶ Push: Juneau · master · Fix it …"), # multi-line item: first line, trimmed, then " …"
+ ("Push", ["Juneau", "one\ntwo", "master"], "▶ Push: Juneau · one … · master"),
]
for title, header, first in shapes:
with self.subTest(title=title, header=header), tempfile.TemporaryDirectory() as tmp:
@@ -592,7 +603,7 @@ class SessionTest(unittest.TestCase):
return_value=os.terminal_size((32, 24))):
jr.session("Push", ["Juneau", "master", "one two three four five six seven"], sinks=jr.Sinks(full_log=log))
self.assertEqual(out.getvalue().splitlines(),
- ["▶ Push: Juneau, master, one two", " three four five six seven", f"▶ Output: tail -f {log}"])
+ ["▶ Push: Juneau · master · one", " two three four five six seven", f"▶ Output: tail -f {log}"])
def test_the_cancel_row_follows_the_log_rows_when_the_variable_is_set(self):
os.environ["JUNEAU_RUN_CANCEL"] = "stop-it %3"
@@ -1010,6 +1021,103 @@ class LiveTest(unittest.TestCase):
def spin(self):
return jr.icon("running", self.clock()) + " "
+ def session(self, title="Push", header=("Juneau", "master"), lines=("Commit Message: m",), more=()):
+ self.push(("session", {"title": title, "header": list(header), "lines": list(lines),
+ "header_more": list(more), "nested": False}))
+
+ def test_finished_rows_stay_in_the_frame_under_the_header(self):
+ self.screen = Screen(rows=10, cols=60)
+ self.live.write = self.screen.feed
+ self.session()
+ self.push(*start())
+ self.push(("step_end", {"id": "tests", "status": "ok", "ms": 4000, "exit": None, "summary": "done",
+ "totals": None}))
+ self.assertEqual(self.screen.lines[:2], ["▶ Push: Juneau · master", "▶ Commit Message: m"])
+ self.assertTrue(self.screen.lines[2].startswith("✔ " + "Tests".ljust(10) + " done"), self.screen.lines)
+ self.assertEqual(self.screen.scrollback, []) # nothing reached scrollback yet
+ self.assertEqual(self.live.height, 3)
+
+ def test_the_frame_never_exceeds_rows_minus_two_and_the_title_stays_on_top(self):
+ self.dims = (60, 8)
+ self.screen = Screen(rows=8, cols=60)
+ self.live.write = self.screen.feed
+ self.session()
+ for n in "abcdefgh":
+ self.push(("step_start", {"id": n, "n": 1, "title": f"Step {n}", "parent": None}),
+ ("step_end", {"id": n, "status": "ok", "ms": 1, "exit": None, "summary": None, "totals": None}))
+ self.push(("step_start", {"id": "z", "n": 1, "title": "Last", "parent": None}))
+ shown = [line for line in self.screen.lines if line.strip()]
+ self.assertLessEqual(len(shown), 6)
+ self.assertEqual(shown[0], "▶ Push: Juneau · master")
+ self.assertTrue(any(line.startswith("✔ ") and " done: " in line for line in shown), shown)
+ self.assertTrue(shown[-1].startswith(self.spin() + "Last"), shown)
+ self.assertFalse(self.screen.clamped)
+
+ def test_a_transcript_line_wider_than_the_pane_is_cut_with_an_ellipsis(self):
+ self.live.render(self.board, [("text", "x" * 200)])
+ self.assertTrue(all(jr._vlen(line) <= 59 for line in self.screen.lines), self.screen.lines)
+ self.assertIn("…", self.screen.text())
+
+ def test_an_identical_frame_is_not_rewritten(self):
+ writes = []
+ live = jr.LiveRenderer(writes.append, size=lambda: os.terminal_size(self.dims), clock=self.clock, color=False)
+ live.render(self.board, [("text", "hello")])
+ drawn = len(writes)
+ self.clock.advance(1)
+ live.tick(self.board) # no running row: nothing on screen moves
+ self.assertEqual(len(writes), drawn)
+
+ def test_a_pane_too_short_for_one_line_draws_nothing(self):
+ self.dims = (60, 2)
+ self.push(*start())
+ self.assertEqual(self.screen.text(), "")
+ self.dims = (60, 10)
+ self.clock.advance(1)
+ self.live.tick(self.board)
+ self.assertTrue(self.screen.text().startswith(self.spin() + "Tests"))
+
+ def test_a_tick_does_not_re_render_the_transcript(self):
+ for n in range(200):
+ self.push(("step_start", {"id": f"s{n}", "n": 1, "title": f"Step {n}", "parent": None}),
+ ("step_end", {"id": f"s{n}", "status": "ok", "ms": 1, "exit": None, "summary": None,
+ "totals": None}))
+ self.push(*start())
+ calls = []
+ real = self.live._block_lines
+ self.live._block_lines = lambda *a, **k: calls.append(a) or real(*a, **k)
+ for _ in range(10):
+ self.clock.advance(0.1)
+ self.live.tick(self.board)
+ self.assertEqual(calls, []) # transcript lines are rendered once, cut once per width
+ self.dims = (50, 10)
+ self.clock.advance(0.1)
+ self.live.tick(self.board)
+ self.assertEqual(calls, []) # a new width re-cuts the cached lines; it does not re-render
+
+ def test_last_draw_moves_only_when_a_frame_is_written(self):
+ self.push(*start())
+ t0 = self.clock()
+ self.assertEqual(self.live.last_draw, t0)
+ self.clock.advance(0.06)
+ self.live.tick(self.board) # throttled
+ self.assertEqual(self.live.last_draw, t0)
+ self.live.last_frame = None # force the next frame to differ
+ self.clock.advance(0.06)
+ self.live.tick(self.board) # 0.12 s after the last frame written: draws
+ self.assertAlmostEqual(self.live.last_draw, t0 + 0.12)
+ self.clock.advance(0.2)
+ cols, rows = self.dims
+ self.live.last_frame = self.live.frame(cols - 1, cols, rows - 2) # the frame this tick would draw
+ self.live.tick(self.board) # identical frame: nothing written, last_draw stays
+ self.assertAlmostEqual(self.live.last_draw, t0 + 0.12)
+
+ def test_finished_rows_keep_the_label_column_they_finished_with(self):
+ self.push(("step_start", {"id": "a", "n": 1, "title": "Alpha", "parent": None}),
+ ("step_end", {"id": "a", "status": "ok", "ms": 1, "exit": None, "summary": "done", "totals": None}))
+ before = self.screen.text().splitlines()[0]
+ self.push(("step_start", {"id": "b", "n": 1, "title": "A much longer label", "parent": None}))
+ self.assertEqual(self.screen.text().splitlines()[0], before) # a longer running label does not shift it
+
def test_a_running_row_and_its_module_redraw_in_place(self):
self.push(*start(), mod_start("a"))
self.push(cls("a"), cls("a", "fail"))
@@ -1120,8 +1228,12 @@ class LiveTest(unittest.TestCase):
self.assertFalse(self.screen.clamped)
self.push(("step_end", {"id": "s", "status": "ok", "ms": 5000, "exit": None, "summary": None,
"totals": None}))
- lines = self.screen.text().splitlines() # finished block prints wrapped too
- self.assertEqual((len(lines), self.live.height), (2, 0))
+ lines = self.screen.text().splitlines() # the finished row stays in the frame, wrapped
+ self.assertEqual((len(lines), self.live.height), (2, 2))
+ self.assertEqual(self.screen.text().count("\u2588"), 84)
+ self.live.close() # close prints it into scrollback
+ self.assertEqual(self.live.height, 0)
+ self.assertEqual(len(self.screen.text().splitlines()), 2)
self.assertEqual(self.screen.text().count("\u2588"), 84)
def test_the_height_cap_holds_with_a_wrapped_bar(self):
@@ -1141,6 +1253,16 @@ class LiveTest(unittest.TestCase):
live.tick(self.board)
self.assertEqual(len(writes), drawn + 1)
+ def test_a_fold_keeps_the_padding_of_the_unit_lines_that_stay(self):
+ self.push(*start(size=3),
+ ("unit_start", {"step": "tests", "id": "tests/a", "title": "Short"}),
+ ("unit_start", {"step": "tests", "id": "tests/b", "title": "A much longer module"}),
+ ("unit_start", {"step": "tests", "id": "tests/c", "title": "Mid length"}))
+ full = self.live.region(self.board, 59)
+ folded = self.live.region(self.board, 59, max_lines=3) # A3 packs b and c, newest first; a stays
+ line = lambda lines: next(x for x in lines if "Short" in x and "," not in x)
+ self.assertEqual(line(folded), line(full)) # padded to the longest running unit both times
+
def test_the_region_never_exceeds_rows_minus_two(self):
self.dims = (60, 6)
self.push(*start(size=5), *[mod_start(n) for n in "abcde"], *[cls(n) for n in "abcde"])
@@ -1282,15 +1404,18 @@ class LiveTest(unittest.TestCase):
def test_header_rows_mark_the_icon_column_and_the_title_wraps_with_a_hanging_indent(self):
for color in (True, False):
with self.subTest(color=color):
- writes = []
- live = jr.LiveRenderer(writes.append, size=lambda: os.terminal_size((21, 10)), clock=self.clock,
- color=color)
- live.render(jr.Board(clock=self.clock), [("header", "Push: Juneau, master, a b c d", jr.HEAD_MARK),
+ writes, screen = [], Screen()
+ live = jr.LiveRenderer(lambda s: writes.append(s) or screen.feed(s),
+ size=lambda: os.terminal_size((21, 10)), clock=self.clock, color=color)
+ live.render(jr.Board(clock=self.clock), [("header", "Push: Juneau · master · a b c d", jr.HEAD_MARK),
("header", "Output: tail -f /a/very/long/path.log", jr.MORE_MARK)])
- lines = jr._ANSI.sub("", "".join(writes)).splitlines()
- self.assertEqual(lines[:3], ["▶ Push: Juneau,", " master, a b c d", "▶ Output: tail -f /a/very/long/path.log"])
+ self.assertEqual(screen.text().splitlines(), # the frame cuts the title to one line
+ ["▶ Push: Juneau · ma…", "▶ Output: tail -f /a/very/long/path.log"])
self.assertEqual("\x1b[2m▶\x1b[0m" in "".join(writes), color) # the rows under the title are dim
self.assertEqual("\x1b[36m▶\x1b[0m Push:" in "".join(writes), color) # the title marker is cyan, like the spinner
+ live.close() # scrollback wraps it, hanging indent
+ self.assertEqual(screen.text().splitlines(),
+ ["▶ Push: Juneau ·", " master · a b c d", "▶ Output: tail -f /a/very/long/path.log"])
def test_hides_the_cursor_while_drawing_and_shows_it_on_close(self):
writes = []
@@ -1300,35 +1425,251 @@ class LiveTest(unittest.TestCase):
live.close()
self.assertTrue(writes[-1].endswith("\x1b[?25h"))
+ def test_close_prints_the_header_wrapped_then_the_transcript_uncut_then_the_final_row(self):
+ self.dims = (30, 10)
+ self.session(header=("Juneau", "master", "a b c d e f"))
+ self.push(("step_start", {"id": "a", "n": 1, "title": "Alpha", "parent": None}),
+ ("step_end", {"id": "a", "status": "ok", "ms": 1, "exit": None, "summary": None, "totals": None}))
+ self.live.render(self.board, [("text", "y" * 80)])
+ self.live.render(self.board, [("final", "ok", 4000, None, None, None, None)])
+ self.assertNotIn("Overall", self.screen.text()) # the final row is not part of the frame
+ self.live.close()
+ lines = self.screen.text().splitlines()
+ self.assertEqual(lines[:2], ["▶ Push: Juneau · master · a b", " c d e f"])
+ self.assertIn("y" * 80, lines) # uncut in scrollback
+ self.assertEqual(lines[-1], "▶ Overall ✅ Done in 0:04")
+ self.assertTrue(self.screen.cursor_visible)
+
+ def test_the_header_reaches_scrollback_exactly_once(self):
+ self.session()
+ self.push(*start())
+ for _ in range(3):
+ self.live.pause()
+ self.live.resume()
+ self.live.close()
+ self.assertEqual(self.screen.text().count("Push: Juneau · master"), 1)
+ self.assertEqual(self.screen.text().count("Commit Message: m"), 1)
+
+ def test_pause_prints_the_transcript_and_resume_draws_only_the_live_part(self):
+ self.session()
+ self.push(("step_start", {"id": "a", "n": 1, "title": "Alpha", "parent": None}),
+ ("step_end", {"id": "a", "status": "ok", "ms": 1, "exit": None, "summary": None, "totals": None}))
+ self.push(*start())
+ self.live.pause()
+ self.assertTrue(self.screen.text().splitlines()[-1].startswith("✔ Alpha"))
+ self.assertEqual(self.live.transcript, [])
+ self.live.resume()
+ lines = self.screen.text().splitlines()
+ self.assertEqual(lines.count("▶ Push: Juneau · master"), 2) # scrollback's copy and the frame's title
+ self.assertEqual(sum(line.startswith("✔ Alpha") for line in lines), 1)
+
+ def test_after_close_a_block_prints_at_once(self):
+ self.live.close()
+ self.live.render(self.board, [("text", "late")])
+ self.assertEqual(self.screen.text(), "late")
+
+ def test_a_nested_renderer_never_prints_the_sticky_header(self):
+ live = jr.LiveRenderer(self.screen.feed, size=lambda: os.terminal_size(self.dims), clock=self.clock,
+ color=False, sticky=[("header", "Push: Juneau · master", jr.HEAD_MARK)])
+ live.render(self.board, feed(self.board, *start()))
+ self.assertTrue(self.screen.text().startswith("▶ Push: Juneau · master"))
+ live.pause()
+ live.close()
+ self.assertNotIn("Push:", self.screen.text())
+
+ def test_the_up_count_follows_a_narrower_terminal_and_caps_at_rows_minus_one(self):
+ self.live.widths, self.live.partial = [59, 59, 10], 0
+ self.assertEqual(self.live._up(os.terminal_size((60, 20))), 3)
+ self.assertEqual(self.live._up(os.terminal_size((30, 20))), 5) # 2 + 2 + 1 after a reflow
+ self.assertEqual(self.live._up(os.terminal_size((30, 4))), 3) # the cursor stops at the top row
+ self.live.partial = 45
+ self.assertEqual(self.live._up(os.terminal_size((30, 20))), 6) # a wrapped prompt line under it
+
+ def test_a_run_started_on_the_bottom_line_redraws_without_climbing_past_its_frame(self):
+ self.screen = Screen(rows=10, cols=60, start_row=9)
+ self.live.write = self.screen.feed
+ self.session(more=("Output: tail -f /" + "o" * 80,)) # a log line that wraps to 2 lines
+ self.push(*start(), mod_start("a"))
+ for _ in range(5):
+ self.push(cls("a"))
+ self.assertFalse(self.screen.clamped)
+ shown = [line for line in self.screen.lines if line.strip()]
+ self.assertEqual(shown.count("▶ Push: Juneau · master"), 1) # no stale copy of the frame
+ self.assertEqual(sum("Mod A" in line for line in shown), 1)
+
+ def test_a_log_line_longer_than_the_terminal_costs_its_wrapped_height(self):
+ self.dims = (40, 8)
+ self.screen = Screen(rows=8, cols=40)
+ self.live.write = self.screen.feed
+ self.session(lines=(), more=("Output: tail -f /" + "o" * 60,))
+ self.push(*start())
+ self.assertLessEqual(len([line for line in self.screen.lines if line.strip()]), 6)
+ self.assertFalse(self.screen.clamped)
+
def test_pause_clears_the_region_and_resume_redraws_it(self):
self.push(*start(), mod_start("a"))
self.live.pause()
- self.assertEqual(self.screen.text(), "")
+ self.assertNotIn(self.spin() + "Tests", self.screen.text())
self.clock.advance(1)
self.live.tick(self.board)
self.assertEqual(self.screen.text(), "")
self.live.resume()
self.assertTrue(self.screen.text().startswith(self.spin() + "Tests"))
- def test_a_watchdog_prompt_stops_redraws_until_the_line_ends(self):
+ def test_a_watchdog_prompt_writes_under_the_frame_and_stops_redraws_until_the_line_ends(self):
self.push(*start())
self.push(("prompt", {"step": "tests", "text": "Passphrase: "}))
- self.assertEqual(self.screen.text(), "Passphrase:")
+ lines = self.screen.text().splitlines()
+ self.assertTrue(lines[0].startswith(self.spin() + "Tests"), lines)
+ self.assertEqual(lines[1], "Passphrase:")
self.assertTrue(self.screen.cursor_visible)
self.clock.advance(1)
self.live.tick(self.board)
- self.assertEqual(self.screen.text(), "Passphrase:")
+ self.assertEqual(self.screen.text().splitlines()[1], "Passphrase:")
+ self.push(("prompt_echo", {"step": "tests", "text": "***"}))
self.push(("prompt_end", {"step": "tests"}))
- self.assertTrue(self.screen.text().splitlines()[1].startswith(self.spin() + "Tests"))
+ lines = self.screen.text().splitlines()
+ self.assertEqual(lines[0], "Passphrase: ***") # the prompt joined the transcript
+ self.assertTrue(lines[1].startswith(self.spin() + "Tests"), lines)
+ self.assertEqual(len(lines), 2) # no stale copy of the old frame
self.assertFalse(self.screen.cursor_visible)
- def test_a_line_printed_during_a_prompt_starts_on_its_own_line(self):
+ def test_rows_that_finish_during_a_prompt_are_held_until_it_ends(self):
self.push(*start())
self.push(("prompt", {"step": "tests", "text": "Passphrase: "}))
self.push(("step_end", {"id": "tests", "status": "ok", "ms": 0, "exit": None, "summary": "done",
"totals": None}))
+ self.assertEqual(self.screen.text().splitlines()[1], "Passphrase:") # the frame is not redrawn
+ self.push(("prompt_end", {"step": "tests"}))
self.assertEqual(self.screen.text().splitlines()[:2], ["Passphrase:", "✔ " + "Tests".ljust(10) + " done"])
+ def viewport(self, cols, rows):
+ """A Screen of the renderer's size, with a shell line above where the run starts."""
+ self.dims = (cols, rows)
+ self.screen = Screen(rows=rows, cols=cols)
+ self.screen.feed("$ push\n")
+ self.live.write = self.screen.feed
+
+ def test_a_shrink_folds_to_the_new_height_and_a_grow_unfolds(self):
+ self.dims = (60, 40)
+ self.screen = Screen(rows=40, cols=60)
+ self.live.write = self.screen.feed
+ self.session()
+ for n in "abcdef":
+ self.push(("step_start", {"id": n, "n": 1, "title": f"Step {n}", "parent": None}),
+ ("step_end", {"id": n, "status": "ok", "ms": 1, "exit": None, "summary": None, "totals": None}))
+ self.push(*start())
+ self.assertTrue(any(line.startswith("✔ Step a") for line in self.screen.lines))
+ self.dims = (60, 10)
+ self.screen.resize(rows=10, cols=60) # Task 1's viewport: keeps the bottom rows, as a terminal does
+ self.clock.advance(1)
+ self.live.tick(self.board)
+ shown = [line for line in self.screen.lines if line.strip()]
+ self.assertLessEqual(len(shown), 8)
+ self.assertEqual(shown[0], "▶ Push: Juneau · master")
+ self.dims = (60, 40)
+ self.screen.resize(rows=40, cols=60)
+ self.clock.advance(1)
+ self.live.tick(self.board)
+ self.assertTrue(any(line.startswith("✔ Step a") for line in self.screen.lines)) # unfolded again
+
+ def test_the_spinner_still_turns_when_nothing_else_changes(self):
+ self.push(*start())
+ first = self.screen.text()
+ self.clock.advance(0.15) # past the next frame: 101.1 / 0.1 rounds down to the same one
+ self.live.tick(self.board)
+ self.assertNotEqual(self.screen.text(), first)
+
+ def test_close_leaves_the_plain_renderers_record_apart_from_alignment(self):
+ events = [("session", {"title": "Push", "header": ["Juneau", "master"], "lines": ["Commit Message: m"],
+ "header_more": [], "nested": False}),
+ ("step_start", {"id": "a", "n": 1, "title": "Alpha", "parent": None}),
+ ("step_end", {"id": "a", "status": "ok", "ms": 1000, "exit": None, "summary": "fine", "totals": None}),
+ ("step_start", {"id": "b", "n": 2, "title": "Beta", "parent": None}),
+ ("note", {"step": "b", "level": "warn", "text": "careful"}),
+ ("step_end", {"id": "b", "status": "warn", "ms": 2000, "exit": None, "summary": None, "totals": None}),
+ ("done", {"status": "ok"})]
+ self.dims = (120, 40)
+ self.screen = Screen(rows=40, cols=120)
+ self.live.write = self.screen.feed
+ board = jr.Board(clock=self.clock) # started with the live one, so both finals read the same elapsed
+ self.push(*events)
+ self.live.close()
+ live = [line.rstrip() for line in self.screen.text().splitlines() if line.strip()]
+ expected = [line.rstrip() for line in plain(board, feed(board, *events), self.clock).splitlines()
+ if line.strip()]
+ squash = lambda lines: [" ".join(line.split()) for line in lines] # label-width padding may differ
+ self.assertEqual(squash(live), squash(expected))
+
+ def test_a_dump_at_a_prompt_keeps_the_line_above_the_frame(self):
+ self.viewport(60, 10)
+ self.push(*start())
+ self.push(("prompt", {"step": "tests", "text": "Passphrase: "}))
+ held = self.live.dump() # Console._fallback's path
+ self.live.close()
+ self.assertEqual(self.screen.history().splitlines()[0], "$ push")
+ self.assertIn("Passphrase:", [line.rstrip() for line in held])
+
+ def test_a_prompt_and_echo_that_wrap_are_counted_at_the_width(self):
+ self.viewport(30, 10)
+ self.push(*start())
+ self.push(("prompt", {"step": "tests", "text": "Enter the signing key passphrase: "})) # 34 columns: wraps
+ self.push(("prompt_echo", {"step": "tests", "text": "*" * 30})) # and wraps again
+ self.push(("prompt_end", {"step": "tests"}))
+ lines = self.screen.history().splitlines()
+ self.assertFalse(self.screen.clamped)
+ self.assertEqual(lines[0], "$ push") # the cursor-up did not overshoot
+ self.assertEqual(sum(line.startswith("Enter the") for line in lines), 1) # nor leave a stale copy
+ self.assertTrue(lines[-1].startswith(self.spin() + "Tests"), lines)
+
+ def test_a_prompt_line_exactly_as_wide_as_the_terminal_counts_once(self):
+ self.viewport(30, 10)
+ self.push(*start())
+ self.push(("prompt", {"step": "tests", "text": "P" * 29 + " "})) # 30 columns: the cursor waits at its end
+ self.live._clear() # what pause() or a resize does mid-prompt
+ self.assertEqual(self.screen.history().splitlines()[0], "$ push")
+ self.assertFalse(self.screen.clamped)
+
+ def test_a_prompt_too_tall_for_the_frame_clears_it(self):
+ self.viewport(20, 5) # B = 3; a prompt wrapping to 2 lines is too tall
+ self.push(*start())
+ self.push(("prompt", {"step": "tests", "text": "Type the full passphrase for the key: "}))
+ self.assertEqual(self.screen.text(), "$ push\nType the full passph\nrase for the key:")
+ self.push(("prompt_end", {"step": "tests"}))
+ self.assertTrue(self.screen.text().splitlines()[-1].startswith(self.spin() + "Tests"))
+
+ def test_closing_at_a_prompt_keeps_the_line_above_the_frame(self):
+ self.viewport(60, 10)
+ self.push(*start())
+ self.push(("prompt", {"step": "tests", "text": "Passphrase: "}))
+ self.live.close() # Ctrl-C at the prompt: _end_session -> Console.close -> close()
+ lines = self.screen.history().splitlines()
+ self.assertEqual(lines[0], "$ push")
+ self.assertEqual(lines.count("Passphrase:"), 1)
+ self.assertFalse(self.screen.clamped)
+
+ def test_a_legacy_raw_block_without_a_role_is_a_prompt(self):
+ self.push(*start())
+ self.live.render(self.board, [("raw", "Old: ")])
+ self.assertTrue(self.live.prompting)
+ self.live.render(self.board, [("raw", "\n")])
+ self.assertFalse(self.live.prompting)
+ self.assertEqual(self.screen.text().splitlines()[0], "Old:")
+
+ def test_an_echo_or_end_without_a_prompt_does_not_misplace_the_frame(self):
+ self.viewport(60, 10)
+ self.push(*start())
+ self.live.render(self.board, [("raw", "\n", "end")]) # an end with no prompt up: ignored
+ self.assertEqual(self.screen.history().splitlines()[0], "$ push")
+ self.live.render(self.board, [("raw", "x", "echo")]) # an echo with no prompt starts one
+ self.assertTrue(self.live.prompting)
+
+ def test_board_raw_blocks_carry_their_role(self):
+ b = jr.Board(clock=self.clock)
+ self.assertEqual(b.apply("prompt", {"step": None, "text": "P: "}), [("raw", "P: ", "prompt")])
+ self.assertEqual(b.apply("prompt_echo", {"step": None, "text": "x"}), [("raw", "x", "echo")])
+ self.assertEqual(b.apply("prompt_end", {"step": None}), [("raw", "\n", "end")])
+
def test_a_large_reactor_keeps_its_count_and_totals_and_wraps_the_bar(self):
self.dims = (64, 10) # 8 region lines of 10 cells hold the 80 modules
self.push(*start(size=80))
@@ -1698,13 +2039,48 @@ class RobustnessTest(unittest.TestCase):
self.assertEqual(out.getvalue().count("hello-once"), 1)
def test_a_fallback_re_renders_only_the_blocks_after_a_raw_one_that_printed(self):
+ for case in ("after", "raw"):
+ with self.subTest(case):
+ out = TTY()
+ console = self.live_console(out)
+ console.board.apply = mock.Mock(return_value=[("raw", "raw-once\n"), ("text", "after")])
+ real, written = console.screen.frame, []
+ console.screen.write = lambda text, write=console.screen.write: (written.append(text), write(text))
+
+ def frame(*args):
+ if case == "after" and any(block[1] == "after" for _, block, _ in console.screen.transcript):
+ raise RuntimeError("frame")
+ if case == "raw" and "raw-once\n" in written: # the raw block's closing redraw
+ raise RuntimeError("frame")
+ return real(*args)
+
+ console.screen.frame = frame
+ console("say", {"text": "x", "level": "warn", "step": None})
+ self.assertIsInstance(console.screen, jr.PlainRenderer)
+ screen = Screen(rows=40, cols=80)
+ screen.feed(out.getvalue())
+ self.assertEqual(screen.history().count("raw-once"), 1)
+ self.assertEqual(screen.history().count("after"), 1)
+
+ def test_a_fallback_prints_what_the_frame_held_untinted_then_the_warning_then_the_rest(self):
out = TTY()
console = self.live_console(out)
- console.screen._flush = mock.Mock(side_effect=[None, RuntimeError("flush")])
- console.board.apply = mock.Mock(return_value=[("raw", "raw-once\n"), ("text", "after")])
- console("say", {"text": "x", "level": "warn", "step": None})
- self.assertEqual(out.getvalue().count("raw-once"), 1)
- self.assertEqual(out.getvalue().count("after"), 1)
+ console("step_start", {"id": "a", "n": 1, "title": "Alpha", "parent": None})
+ console("step_end", {"id": "a", "status": "ok", "ms": 0, "exit": None, "summary": "held-once",
+ "totals": None})
+ with mock.patch.object(jr.LiveRenderer, "frame", side_effect=RuntimeError("frame")):
+ console("step_start", {"id": "b", "n": 2, "title": "Beta", "parent": None})
+ console("step_end", {"id": "b", "status": "ok", "ms": 0, "exit": None, "summary": "after-once",
+ "totals": None})
+ self.assertIsInstance(console.screen, jr.PlainRenderer)
+ screen = Screen(rows=40, cols=80)
+ screen.feed(out.getvalue())
+ history = screen.history()
+ self.assertEqual(history.count("held-once"), 1)
+ self.assertEqual(history.count("after-once"), 1)
+ self.assertLess(history.index("held-once"), history.index("⚠ console view failed"))
+ tail = out.getvalue()[out.getvalue().index("⚠ console view failed"):]
+ self.assertNotIn("\x1b[3", tail) # the plain part is untinted
def test_a_fallback_during_a_prompt_starts_the_warning_on_its_own_line(self):
out = TTY()
@@ -1821,23 +2197,24 @@ class RobustnessTest(unittest.TestCase):
self.assertTrue(console.stopped)
self.assertIn("⚠ console view stopped (RuntimeError('pause'))", out.getvalue())
- def test_a_fallback_re_renders_a_raw_block_whose_clear_failed(self):
+ def test_a_fallback_after_a_prompt_keeps_it_once(self):
out = TTY()
console = self.live_console(out)
console.board.apply = mock.Mock(return_value=[("text", "before"), ("raw", "raw-once\n")])
- clear, calls = console.screen._clear, []
+ real = console.screen.frame
- def failing_clear():
- calls.append(None)
- if len(calls) == 2: # the raw block's, after the text block's in _flush
- raise RuntimeError("clear")
- clear()
+ def frame(*args):
+ if any(block[1] == "before" for _, block, _ in console.screen.transcript):
+ raise RuntimeError("frame")
+ return real(*args)
- console.screen._clear = failing_clear
+ console.screen.frame = frame
console("say", {"text": "x", "level": "warn", "step": None})
self.assertIsInstance(console.screen, jr.PlainRenderer)
- self.assertEqual(out.getvalue().count("before"), 1)
- self.assertEqual(out.getvalue().count("raw-once"), 1)
+ screen = Screen(rows=40, cols=80)
+ screen.feed(out.getvalue())
+ self.assertEqual(screen.history().count("before"), 1)
+ self.assertEqual(screen.history().count("raw-once"), 1)
def test_a_plain_stop_during_a_prompt_starts_the_warning_on_its_own_line(self):
out = io.StringIO()
@@ -2247,6 +2624,11 @@ class PendingTest(PendingCase):
class FoldTest(PendingCase):
+ MORE = "○ {} more: {}"
+
+ def more(self, *indexes):
+ return self.MORE.format(len(indexes), ", ".join(f"Pending {i}" for i in indexes))
+
def busy(self):
b = self.planned([("tests", "Tests")] + [(f"p{i}", f"Pending {i}") for i in range(6)])
feed(b, *start(size=4), *[mod_start(n) for n in "abcd"], *[cls(n) for n in "abcd"])
@@ -2256,18 +2638,18 @@ class FoldTest(PendingCase):
region = self.live().region(self.busy(), 79, 8)
self.assertEqual(len(region), 8)
self.assertEqual(sum(line.startswith(" Mod ") for line in region), 4) # every module line kept
- self.assertEqual(region[-1], "○ +4 more pending")
+ self.assertEqual(region[-1], self.more(2, 3, 4, 5))
self.assertEqual(region[-3:-1], ["○ Pending 0", "○ Pending 1"])
def test_more_pending_line_always_kept(self):
region = self.live().region(self.busy(), 79, 4)
- self.assertEqual(region[-1], "○ +6 more pending")
+ self.assertEqual(region[-1], self.more(*range(6)))
def test_then_module_lines_fold(self):
region = self.live().region(self.busy(), 79, 5)
self.assertEqual(len(region), 5)
self.assertIn(" C, D", region)
- self.assertEqual(region[-1], "○ +6 more pending")
+ self.assertEqual(region[-1], self.more(*range(6)))
def test_folding_across_two_rows_takes_the_newest_first_wherever_it_runs(self):
b = self.planned([("one", "One"), ("two", "Two")])
@@ -2340,12 +2722,111 @@ class SetupRowTest(RowCase):
self.assertEqual(feed(b, step_start("next")), []) # a later row does not reopen it
+class StickyHeaderTest(unittest.TestCase):
+ HEADER = [["head", "Push: Juneau · master", False], ["more", "Commit Message: m", True],
+ ["more", "Output: tail -f /o", False]]
+
+ def setUp(self):
+ reset_run_state()
+ self.addCleanup(reset_run_state)
+
+ def nested_console(self, out, value, color=False):
+ env = {"TERM": "xterm", "RUN_MARKERS": "", "JUNEAU_RUN_HEADER": value}
+ if not color:
+ env["NO_COLOR"] = "1"
+ with mock.patch.dict(os.environ, env):
+ if color:
+ os.environ.pop("NO_COLOR", None)
+ console = jr.Console(jr.Sinks(console="condensed"), stream=out, size=lambda: os.terminal_size((80, 24)),
+ nested=True)
+ self.addCleanup(console.close)
+ return console
+
+ def test_the_parent_header_parses_into_header_blocks(self):
+ with mock.patch.dict(os.environ, {"JUNEAU_RUN_HEADER": json.dumps(self.HEADER)}):
+ self.assertEqual(jr._parent_header(), [("header", "Push: Juneau · master", jr.HEAD_MARK),
+ ("header", "Commit Message: m", jr.MORE_MARK, "cut"),
+ ("header", "Output: tail -f /o", jr.MORE_MARK)])
+
+ def test_a_missing_or_malformed_value_is_no_sticky_header_silently(self):
+ for value in ("", "{", "[[1]]", '"x"', "[[\"head\"]]"):
+ with self.subTest(value=value), mock.patch.dict(os.environ, {"JUNEAU_RUN_HEADER": value}):
+ self.assertEqual(jr._parent_header(), [])
+
+ def test_a_nested_live_console_draws_it_on_top_and_never_prints_it(self):
+ out = TTY()
+ console = self.nested_console(out, json.dumps(self.HEADER))
+ console("step_start", {"id": "t", "n": 1, "title": "Tests", "parent": None})
+ self.assertTrue(jr._ANSI.sub("", out.getvalue()).lstrip("\r").startswith("▶ Push: Juneau · master"))
+ console.pause()
+ console.resume()
+ console("step_end", {"id": "t", "status": "ok", "ms": 0, "exit": None, "summary": None, "totals": None})
+ console.close()
+ screen = Screen(rows=24, cols=80)
+ screen.feed(out.getvalue())
+ self.assertNotIn("Push: Juneau", screen.history()) # drawn live only: not in scrollback after close
+
+ def test_under_colour_the_sticky_title_is_cyan(self):
+ out = TTY()
+ console = self.nested_console(out, json.dumps(self.HEADER), color=True)
+ console("step_start", {"id": "t", "n": 1, "title": "Tests", "parent": None})
+ self.assertIn("\x1b[36m▶\x1b[0m Push: Juneau · master", out.getvalue())
+
+ def test_handoff_exports_this_sessions_header_or_its_own_sticky_one(self):
+ console = mock.Mock(mode="live", header_blocks=[("header", "Push: Juneau · master", jr.HEAD_MARK),
+ ("header", "Commit Message: m", jr.MORE_MARK, "cut")],
+ sticky=[])
+ console.board.pending.return_value = []
+ console.board.overall_handoff.return_value = None
+ with mock.patch.object(jr, "_console", console), jr.handoff() as env:
+ self.assertEqual(json.loads(env["JUNEAU_RUN_HEADER"]), [["head", "Push: Juneau · master", False],
+ ["more", "Commit Message: m", True]])
+ console.header_blocks, console.sticky = [], [("header", "Push: Juneau", jr.HEAD_MARK)]
+ with mock.patch.object(jr, "_console", console), jr.handoff() as env:
+ self.assertEqual(json.loads(env["JUNEAU_RUN_HEADER"]), [["head", "Push: Juneau", False]])
+
+ def test_handoff_reads_the_child_outcomes_and_removes_the_file(self):
+ board = jr.Board(clock=Clock())
+ board.apply("plan", {"ids": ["build", "install"], "labels": {"build": "Compile", "install": "Install"}})
+ console = mock.Mock(mode="live", board=board, header_blocks=[], sticky=[])
+ with mock.patch.object(jr, "_console", console), jr.handoff() as env:
+ path = Path(env["JUNEAU_RUN_HANDBACK"])
+ path.write_text(json.dumps([["build", "ok", "Compile"]]))
+ self.assertEqual(board.handed_back, {"build": "ok"})
+ self.assertFalse(path.exists())
+ console.log_only.assert_not_called()
+
+ def test_a_handoff_ended_by_an_exception_still_counts_but_never_notes(self):
+ for error in (KeyboardInterrupt, RuntimeError):
+ with self.subTest(error=error.__name__):
+ board = jr.Board(clock=Clock())
+ board.apply("plan", {"ids": ["build"], "labels": {"build": "Compile"}})
+ console = mock.Mock(mode="live", board=board, header_blocks=[], sticky=[])
+ with self.assertRaises(error), mock.patch.object(jr, "_console", console), jr.handoff() as env:
+ Path(env["JUNEAU_RUN_HANDBACK"]).write_text(json.dumps([["build", "fail", "Compile"]]))
+ raise error()
+ self.assertEqual(board.handed_back, {"build": "fail"})
+ with self.assertRaises(error), mock.patch.object(jr, "_console", console), jr.handoff():
+ raise error() # Ctrl-C before the child wrote anything
+ console.log_only.assert_not_called()
+
+ def test_a_missing_or_malformed_handback_counts_nothing_and_notes_once(self):
+ for text in ("", "{", '[["build", "ok"]]'):
+ with self.subTest(text=text):
+ board = jr.Board(clock=Clock())
+ console = mock.Mock(mode="live", board=board, header_blocks=[], sticky=[])
+ with mock.patch.object(jr, "_console", console), jr.handoff() as env:
+ Path(env["JUNEAU_RUN_HANDBACK"]).write_text(text)
+ self.assertEqual(board.handed_back, {})
+ console.log_only.assert_called_once()
+
+
class NestedPendingTest(PendingCase):
def test_handoff_exports_pending(self):
b = self.planned([("pre", "Prechecks"), ("tests", "Tests"), ("install", "Install")])
feed(b, step_start("pre", "Prechecks"), step_end("pre"), step_start("tests", "Tests"))
self.assertEqual(b.pending(), [("install", "Install")])
- console = mock.Mock(mode="live", board=b)
+ console = mock.Mock(mode="live", board=b, header_blocks=[], sticky=[])
with mock.patch.object(jr, "_console", console), jr.handoff() as env:
self.assertEqual(json.loads(env["JUNEAU_RUN_PENDING"]), [["install", "Install"]])
@@ -2361,6 +2842,95 @@ class NestedPendingTest(PendingCase):
self.assertEqual(b.pending(), [])
+class HandBackTest(unittest.TestCase):
+ PLAN = [("pre", "Prechecks"), ("build", "Compile"), ("tests", "Java Tests"), ("js-tests", "Browser Tests"),
+ ("install", "Install")]
+
+ def setUp(self):
+ reset_run_state()
+ self.addCleanup(reset_run_state)
+ self.clock = Clock()
+
+ def parent(self):
+ b = jr.Board(clock=self.clock)
+ b.apply("plan", {"ids": [i for i, _ in self.PLAN], "labels": dict(self.PLAN)})
+ feed(b, step_start("pre", "Prechecks"), step_end("pre"))
+ return b
+
+ def test_handed_back_ids_count_as_done_skips_are_left_out_and_no_row_is_drawn(self):
+ b = self.parent()
+ b.hand_back([("build", "ok", "Compile"), ("tests", "warn", "Java Tests"), ("js-tests", "skip", "Browser Tests")])
+ feed(b, step_start("install", "Install"))
+ self.assertEqual(set(b.rows), {"pre", "install"})
+ self.assertEqual(b.pending(), [])
+ feed(b, step_end("install"))
+ self.assertEqual(b.step_counts(), (4, 4))
+
+ def test_a_handed_back_fail_sets_the_failed_title_and_is_not_counted_done(self):
+ b = self.parent()
+ b.hand_back([("build", "ok", "Compile"), ("tests", "fail", "Tests")])
+ self.assertEqual(b.failed_title, "Java Tests") # the parent's own label wins
+ self.assertEqual(b.step_counts(), (2, 5))
+
+ def test_a_fail_the_parent_has_no_label_for_takes_the_childs_label_not_the_id(self):
+ b = self.parent()
+ b.hand_back([("py-x", "fail", "Python Tests")])
+ self.assertEqual(b.failed_title, "Python Tests")
+
+ def test_an_id_with_a_row_keeps_its_row(self):
+ b = self.parent()
+ b.hand_back([("pre", "fail", "Prechecks")])
+ self.assertIsNone(b.failed_title)
+ self.assertEqual(b.handed_back, {})
+
+ def nested(self, path):
+ return mock.patch.dict(os.environ, {"JUNEAU_RUN_HANDBACK": str(path), "RUN_MARKERS": ""})
+
+ def test_a_nested_console_writes_its_top_level_rows_outcomes_at_done(self):
+ with tempfile.TemporaryDirectory() as tmp:
+ path = Path(tmp) / "hb.json"
+ with self.nested(path):
+ console = jr.Console(jr.Sinks(console="none"), nested=True)
+ for kind, fields in (step_start("build", "Compile"), step_end("build"),
+ step_start("js-tests", "Browser Tests"), step_end("js-tests", "skip"),
+ step_start("tests", "Java Tests"),
+ ("step_start", {"id": "tests/core", "n": 1, "title": "core", "parent": "tests"})):
+ console(kind, fields)
+ console("done", {"status": "fail", "commit": None})
+ self.assertEqual(json.loads(path.read_text()), [["build", "ok", "Compile"], ["js-tests", "skip", "Browser Tests"],
+ ["tests", "fail", "Java Tests"]]) # no sub-row
+
+ def test_a_nested_console_takes_the_handback_path_out_of_the_environment(self):
+ with tempfile.TemporaryDirectory() as tmp, self.nested(Path(tmp) / "hb.json"):
+ jr.Console(jr.Sinks(console="none"), nested=True)
+ self.assertNotIn("JUNEAU_RUN_HANDBACK", os.environ) # a grandchild session must not write our file
+
+ def test_a_nested_console_hands_back_after_its_board_failed(self):
+ with tempfile.TemporaryDirectory() as tmp:
+ path = Path(tmp) / "hb.json"
+ with self.nested(path):
+ console = jr.Console(jr.Sinks(console="none"), nested=True)
+ for kind, fields in (step_start("build", "Compile"), step_end("build")):
+ console(kind, fields)
+ with mock.patch.object(console.board, "apply", side_effect=RuntimeError("board")):
+ console(*step_start("tests", "Java Tests"))
+ console("done", {"status": "fail", "commit": None})
+ self.assertEqual(json.loads(path.read_text()), [["build", "ok", "Compile"]])
+
+ def test_a_planless_nested_session_under_a_parent_overall_notes_it_in_the_condensed_log_only(self):
+ overall = json.dumps({"elapsed": 5.0, "done": 1, "pending": [["install", 30.0]]})
+ with tempfile.TemporaryDirectory() as tmp:
+ log = Path(tmp) / "c.log"
+ out = io.StringIO()
+ with mock.patch.dict(os.environ, {"JUNEAU_RUN_OVERALL": overall, "RUN_MARKERS": ""}):
+ console = jr.Console(jr.Sinks(console="condensed", condensed_log=str(log)), stream=out, nested=True)
+ self.assertIsNone(console.board.overall()) # pinned: no plan draws no Overall
+ console("done", {"status": "ok", "commit": None})
+ console.close()
+ self.assertEqual(log.read_text().count("declared no plan"), 1)
+ self.assertNotIn("declared no plan", out.getvalue())
+
+
class LogTest(RowCase):
def run_log(self, *events):
with tempfile.TemporaryDirectory() as tmp:
@@ -2425,5 +2995,230 @@ class CompileOnlyMavenRowTest(unittest.TestCase):
self.assertNotIn("no tests run", jr.row_text(row, jr.PLAIN_GLYPHS, 100.0)[2])
+class ScreenTest(unittest.TestCase):
+ """The viewport mode of support.Screen, which the fit tests rely on."""
+
+ def test_a_newline_on_the_last_row_scrolls_into_scrollback(self):
+ s = Screen(rows=3, cols=10)
+ s.feed("a\nb\nc\nd")
+ self.assertEqual((s.scrollback, s.lines), (["a"], ["b", "c", "d"]))
+
+ def test_cursor_up_clamps_at_the_top_row(self):
+ s = Screen(rows=3, cols=10, start_row=2)
+ s.feed("\x1b[3A")
+ self.assertEqual((s.row, s.clamped), (0, True))
+
+ def test_a_long_line_autowraps_at_cols(self):
+ s = Screen(rows=5, cols=4)
+ s.feed("abcdef\n")
+ self.assertEqual(s.lines[:2], ["abcd", "ef"])
+
+ def test_a_wide_character_takes_two_columns(self):
+ s = Screen(rows=5, cols=4)
+ s.feed("🧱🧱x")
+ self.assertEqual(s.lines[:2], ["🧱🧱", "x"])
+
+ def test_without_a_viewport_it_behaves_as_before(self):
+ s = Screen()
+ s.feed("ab\ncd\x1b[1A\rX\x1b[K")
+ self.assertEqual(s.text(), "X\ncd")
+ self.assertEqual(s.scrollback, [])
+
+ def test_history_is_scrollback_then_the_screen(self):
+ s = Screen(rows=2, cols=10)
+ s.feed("a\nb\nc")
+ self.assertEqual(s.history(), "a\nb\nc")
+
+ def test_a_newline_after_a_full_width_line_does_not_add_a_blank_line(self):
+ s = Screen(rows=5, cols=4)
+ s.feed("abcd\nef")
+ self.assertEqual(s.lines[:2], ["abcd", "ef"])
+
+ def test_resize_changes_the_viewport_without_reflowing(self):
+ s = Screen(rows=5, cols=10)
+ s.feed("abcdefgh\nx")
+ s.resize(rows=3, cols=4)
+ self.assertEqual((s.rows, s.cols, s.lines[0]), (3, 4, "abcdefgh")) # a non-reflowing terminal
+
+
+class HeaderTest(unittest.TestCase):
+ def setUp(self):
+ reset_run_state()
+ self.addCleanup(reset_run_state)
+ self.clock = Clock()
+
+ def blocks(self, title, header=(), lines=(), more=()):
+ return jr.Board(clock=self.clock).apply("session", {"title": title, "header": list(header), "lines": list(lines),
+ "header_more": list(more), "nested": False})
+
+ def test_items_join_with_a_middle_dot_and_lines_are_cut_blocks(self):
+ self.assertEqual(self.blocks("Push", ["Juneau", "master"], ["Commit Message: Fix it"], ["Output: tail -f /l"]), [
+ ("header", "Push: Juneau · master", jr.HEAD_MARK),
+ ("header", "Commit Message: Fix it", jr.MORE_MARK, "cut"),
+ ("header", "Output: tail -f /l", jr.MORE_MARK)])
+
+ def test_a_multi_line_entry_shows_its_first_line_and_an_ellipsis(self):
+ self.assertEqual(self.blocks("Push", ["Juneau"], ["Commit Message: Fix it\n\nBody"])[1][1],
+ "Commit Message: Fix it …")
+
+ def test_no_items_is_the_bare_title_and_no_lines_is_no_line_two(self):
+ self.assertEqual(self.blocks("Test"), [("header", "Test", jr.HEAD_MARK)])
+
+ def test_the_live_screen_cuts_the_title_and_the_cut_lines(self):
+ title = ("header", "Push: Juneau · master · " + "x" * 80, jr.HEAD_MARK)
+ line = ("header", "Commit Message: " + "y" * 80, jr.MORE_MARK, "cut")
+ log = ("header", "Output: tail -f /" + "z" * 80, jr.MORE_MARK)
+ for cols in (40, 60, 120):
+ with self.subTest(cols=cols):
+ for block in (title, line):
+ shown = jr.header_lines(block, cols, live=True)
+ self.assertEqual(len(shown), 1)
+ self.assertLessEqual(jr._vlen(shown[0]), cols)
+ self.assertTrue(shown[0].endswith("…") or jr._vlen(block[1]) + 2 <= cols)
+ self.assertEqual(jr.header_lines(log, cols, live=True), ["▶ " + log[1]]) # a log line is never cut
+
+ def test_the_plain_screen_never_cuts_and_still_wraps_the_title(self):
+ line = ("header", "Commit Message: " + "y" * 80, jr.MORE_MARK, "cut")
+ self.assertEqual(jr.header_lines(line, 40), ["▶ " + line[1]])
+ self.assertGreater(len(jr.header_lines(("header", "Push: " + "w " * 40, jr.HEAD_MARK), 40)), 1)
+
+ def test_an_emoji_title_is_cut_by_display_width(self):
+ shown = jr.header_lines(("header", "Push: SSC · future · 🧱 Sandbox mode", jr.HEAD_MARK), 21, live=True)[0]
+ self.assertLessEqual(jr._vlen(shown), 21)
+
+ def test_session_publishes_lines(self):
+ seen = []
+ jr.subscribe(lambda kind, fields: seen.append((kind, fields)))
+ with mock.patch.object(jr, "Console", side_effect=lambda *a, **k: mock.MagicMock()):
+ jr.session("Push", ["Juneau"], lines=["Commit Message: m"], sinks=jr.Sinks(console="none"))
+ fields = [f for k, f in seen if k == "session"][0]
+ self.assertEqual(fields["lines"], ["Commit Message: m"])
+
+
+def _g(state, label, body=0):
+ """A transcript group: its row line, then body lines."""
+ return jr._Group(state, label, [f"{'✗' if state == 'fail' else '✔'} {label}"], [f" note {i}" for i in range(body)])
+
+
+def _run(label, units=0, t0=0.0):
+ return jr._Run([f"⠋ {label}"], [jr._Unit(t0 + i, i, [f" Mod {i}"], f"Mod {i}") for i in range(units)])
+
+
+class FitTest(unittest.TestCase):
+ TITLE = ("▶ Push: Juneau · master", [("▶ Commit Message: m", False, -1), ("▶ Output: tail -f /o", True, 1),
+ ("▶ Condensed: tail -f /c", True, 0)])
+ OVERALL = "▶ Overall 38% ~7:40 left · 4 of 13 steps"
+
+ def fit(self, budget, groups=(), running=(), pending=(), header=None, overall=OVERALL, cols=59):
+ return jr.fit_view(header or self.TITLE, list(groups), list(running), list(pending), overall, budget, cols,
+ cols + 1)
+
+ def full(self):
+ return dict(groups=[_g("ok", "Prechecks"), _g("ok", "Setup"), _g("ok", "Compile")],
+ running=[_run("Java Tests", units=4)], pending=["Python Tests", "Browser Tests", "Commit"])
+
+ def test_everything_fits_unfolded(self):
+ lines = self.fit(30, **self.full())
+ self.assertEqual(len(lines), 16)
+ self.assertEqual((lines[0], lines[-1]), (self.TITLE[0], self.OVERALL))
+
+ def test_a1_folds_pending_rows_partially_then_all(self):
+ lines = self.fit(15, **self.full())
+ self.assertIn("○ Python Tests", lines)
+ self.assertIn("○ 2 more: Browser Tests, Commit", lines)
+ lines = self.fit(14, **self.full())
+ self.assertIn("○ 3 more: Python Tests, Browser Tests, Commit", lines)
+ self.assertNotIn("○ Python Tests", lines)
+
+ def test_a2_folds_done_groups_oldest_first_where_the_first_stood(self):
+ lines = self.fit(12, **self.full())
+ self.assertEqual(lines[4], "✔ 3 done: Prechecks, Setup, Compile")
+ self.assertEqual(len(lines), 12)
+
+ def test_a2_a_warn_row_folds_with_the_warn_icon_and_its_notes_fold_with_it(self):
+ lines = self.fit(8, groups=[_g("ok", "A", body=2), _g("warn", "B", body=1)], running=[_run("C")])
+ self.assertIn("⚠ 2 done: A, B", lines)
+ self.assertFalse(any("note" in line for line in lines))
+
+ def test_a3_packs_unit_lines_newest_first(self):
+ lines = self.fit(10, **self.full())
+ self.assertIn(" Mod 0", lines)
+ self.assertIn(" Mod 1, Mod 2, Mod 3", lines) # folded newest first, listed oldest first
+
+ def test_a4_cuts_failed_bodies_oldest_first_and_keeps_the_failed_row(self):
+ groups = [_g("fail", "A", body=5), _g("fail", "B", body=5)]
+ lines = self.fit(12, groups=groups, header=("▶ T", []), overall=None) # 13 lines unfolded
+ self.assertEqual(lines[1:3], ["✗ A", " … 5 more lines (condensed log)"])
+ self.assertIn(" note 4", lines) # B's body is still whole
+
+ def test_the_worked_example_ten_line_pane(self):
+ lines = self.fit(8, **self.full())
+ self.assertEqual(lines, [
+ "▶ Push: Juneau · master", "▶ Commit Message: m", "▶ Output: tail -f /o", "▶ Condensed: tail -f /c",
+ "✔ 3 done: Prechecks, Setup, Compile", "⠋ Java Tests", "○ 3 more: Python Tests, Browser Tests, Commit",
+ self.OVERALL])
+
+ def test_the_more_line_is_cut_at_a_label_boundary(self):
+ lines = self.fit(4, running=[_run("X")], pending=[f"Step {i}" for i in range(20)], header=("▶ T", []), cols=40)
+ more = [line for line in lines if "more:" in line][0]
+ self.assertTrue(more.endswith(", …") and jr._vlen(more) <= 40, more)
+
+ def test_a_log_line_costs_its_wrapped_height(self):
+ header = ("▶ T", [("▶ Output: tail -f /" + "o" * 30, True, 1)]) # 49 columns: 2 lines at 30
+ self.assertEqual(jr.fit_view(header, [], [_run("X")], [], None, 4, 29, 30), ["▶ T", header[1][0][0], "⠋ X"])
+ self.assertEqual(jr.fit_view(header, [], [_run("X")], [], None, 3, 29, 30), ["▶ T", "⠋ X"]) # B5 drops it
+
+ def test_b1_drops_the_names_lines_then_b2_the_done_line_then_b3_the_more_line(self):
+ full = self.full()
+ self.assertNotIn(" Mod 1, Mod 2, Mod 3", self.fit(8, **full))
+ self.assertNotIn("✔ 3 done: Prechecks, Setup, Compile", self.fit(7, **full))
+ lines = self.fit(6, **full)
+ self.assertFalse(any("more:" in line for line in lines))
+
+ def test_failures_stay_visible_until_b4(self):
+ groups = [_g("fail", "A", body=3), _g("ok", "B"), _g("fail", "C", body=3)]
+ self.assertTrue({"✗ A", "✗ C"} <= set(self.fit(10, groups=groups, running=[_run("X")])))
+ lines = self.fit(7, groups=groups, running=[_run("X")])
+ self.assertIn("✗ 2 failed: A, C", lines)
+
+ def test_b5_drops_the_commit_message_then_condensed_then_output(self):
+ lines = self.fit(5, groups=[_g("fail", "A")], running=[_run("X")])
+ self.assertNotIn("▶ Commit Message: m", lines)
+ self.assertIn("▶ Output: tail -f /o", lines)
+ self.assertNotIn("▶ Condensed: tail -f /c", lines)
+
+ def test_a_six_line_pane_keeps_the_title_overall_and_the_newest_running_row(self):
+ lines = self.fit(4, groups=[_g("ok", "A")], running=[_run("Old"), _run("New")], pending=["P"])
+ self.assertEqual(lines, ["▶ Push: Juneau · master", "⠋ Old", "⠋ New", self.OVERALL]) # B5 before B6
+ self.assertEqual(self.fit(3, running=[_run("Old"), _run("New")]), ["▶ Push: Juneau · master", "⠋ New",
+ self.OVERALL])
+
+ def test_b7_drops_the_failed_line_then_b8_the_newest_row(self):
+ groups = [_g("fail", "A")]
+ self.assertEqual(self.fit(2, groups=groups, running=[_run("X")]), ["▶ Push: Juneau · master", self.OVERALL])
+
+ def test_three_and_two_line_panes(self):
+ self.assertEqual(self.fit(1, **self.full()), ["▶ Push: Juneau · master"]) # B = 1: the title only
+ self.assertEqual(self.fit(0, **self.full()), [])
+
+ def test_no_title_pins_overall_alone(self):
+ self.assertEqual(self.fit(1, running=[_run("X")], header=(None, [])), [self.OVERALL])
+
+ def test_the_counted_height_matches_the_lines_at_every_fold(self):
+ header = ("▶ T", [("▶ Commit Message: m", False, -1), ("▶ Output: tail -f /" + "o" * 70, True, 1)])
+ groups = [_g("ok", "A", body=2), _g("fail", "B", body=3), _g("warn", "C", body=1), _g("fail", "D", body=2)]
+ fold = jr._Fold(header, groups, [_run("X", units=3), _run("Y", units=2)], ["P", "Q", "R"], "▶ Overall", 59,
+ False)
+ drawn = lambda: sum(max(1, -(-jr._vlen(t) // 60)) if wraps else 1 for t, wraps in fold.lines())
+ self.assertEqual(fold.height(60), drawn())
+ for _ in fold.steps():
+ self.assertEqual(fold.height(60), drawn())
+
+ def test_the_height_is_never_over_budget(self):
+ for budget in range(0, 20):
+ with self.subTest(budget=budget):
+ self.assertLessEqual(len(self.fit(budget, **self.full())), max(budget, 0))
+
+
if __name__ == "__main__":
unittest.main()
diff --git a/juneau-run/src/test/python/test_runner.py b/juneau-run/src/test/python/test_runner.py
index de56abbe12..4990ca5b46 100644
--- a/juneau-run/src/test/python/test_runner.py
+++ b/juneau-run/src/test/python/test_runner.py
@@ -1251,7 +1251,7 @@ class ScriptApiTest(unittest.TestCase):
reset_run_state()
self.addCleanup(reset_run_state)
self.bus = jr.subscribe(Events())
- self.console = mock.Mock(mode="live")
+ self.console = mock.Mock(mode="live", header_blocks=[], sticky=[])
self.console.board.width.return_value = 9
self.console.board.detail = "modules"
self.console.board.pending.return_value = []
@@ -1404,8 +1404,11 @@ class ScriptApiTest(unittest.TestCase):
def test_handoff_pauses_and_exports_the_label_width_and_detail(self):
with jr.handoff() as env:
+ handback = env.pop("JUNEAU_RUN_HANDBACK")
self.assertEqual(env, {"JUNEAU_RUN_LABEL_WIDTH": "9", "JUNEAU_RUN_DETAIL": "modules"})
+ self.assertTrue(os.path.exists(handback))
self.assertEqual(self.calls(), ["pause"])
+ self.assertFalse(os.path.exists(handback))
self.assertEqual(self.calls(), ["pause", "resume"])
def test_handoff_exports_the_pending_rows(self):
diff --git a/juneau-run/src/test/python/test_scripts_session.py b/juneau-run/src/test/python/test_scripts_session.py
index a7701d2e23..3e15a8ca7a 100644
--- a/juneau-run/src/test/python/test_scripts_session.py
+++ b/juneau-run/src/test/python/test_scripts_session.py
@@ -340,5 +340,17 @@ class CondensedTest(SessionCase):
self.assertFalse(jr.session_open())
+class HeaderLinesTest(SessionCase):
+ def test_lines_are_forwarded_to_the_console_session(self):
+ seen = []
+ sys.modules["juneau_run"] = jr
+ with mock.patch.object(run, "console_requested", return_value=True), \
+ mock.patch.object(jr, "session", side_effect=lambda *a, **k: seen.append((a, k))):
+ with run.session("Push", header=("Juneau", "master"), lines=("Commit Message: m",), exit=False):
+ pass
+ self.assertEqual(seen[0][0][:2], ("Push", ("Juneau", "master")))
+ self.assertEqual(seen[0][1]["lines"], ("Commit Message: m",))
+
+
if __name__ == "__main__":
unittest.main()
diff --git a/juneau-run/src/test/python/test_session_plan.py b/juneau-run/src/test/python/test_session_plan.py
index 26ab260a4f..f332238f10 100644
--- a/juneau-run/src/test/python/test_session_plan.py
+++ b/juneau-run/src/test/python/test_session_plan.py
@@ -206,6 +206,25 @@ class BoardPlanTest(unittest.TestCase):
b.estimator = lambda *a: 1 / 0
self.assertIsNone(b.overall())
+ def test_step_counts_count_ok_and_warn_steps_of_all_counted(self):
+ b = self.board(plan=("a", "b", "c"))
+ b.apply(*start("a")); b.apply(*end("a"))
+ b.apply(*start("b")); b.apply("step_end", dict(id="b", status="fail", ms=0, exit=1, summary=None, totals=None))
+ self.assertEqual(b.step_counts(), (1, 3))
+
+ def test_step_counts_leave_out_skips_and_are_none_without_a_plan(self):
+ b = self.board(plan=("a", "b"))
+ b.apply(*start("a")); b.apply("step_end", dict(id="a", status="skip", ms=0, exit=None, summary=None, totals=None))
+ self.assertEqual(b.step_counts(), (0, 1))
+ self.assertIsNone(self.board(plan=()).step_counts())
+ self.assertIsNone(self.board(plan=None).step_counts())
+
+ def test_the_final_block_carries_the_counts(self):
+ b = self.board(plan=("a",))
+ b.apply(*start("a")); b.apply(*end("a"))
+ final = b.apply("done", dict(status="ok", commit=None))[0]
+ self.assertEqual(final[0:2] + final[5:], ("final", "ok", 1, 1))
+
class PlanApiTest(unittest.TestCase):
def setUp(self):
diff --git a/scripts/README_SONARQUBE.md b/scripts/README_SONARQUBE.md
index 6780ef76cb..6742fed4cd 100644
--- a/scripts/README_SONARQUBE.md
+++ b/scripts/README_SONARQUBE.md
@@ -1,219 +1,3 @@
-# SonarQube Issues Management Tools
+# SonarQube tools
-> **For ad-hoc terminal triage, use `python3 scripts/sonarqube.py <path>` (or the `/sonarqube` Cursor command). For live-local IDE triage and quick-fix application, use the `eclipse-warnings` Cursor skill. The TSV-export workflow below is preserved for batch categorization but is no longer the primary path.**
-
-This directory contains tools for categorizing and managing SonarQube issues in the Apache Juneau project.
-
-## Overview
-
-The Apache Juneau project has **3,116 SonarQube issues** that have been categorized into **30 categories** for systematic fixing.
-
-## Quick Start
-
-### 1. View Category Summary
-
-```bash
-cd /Users/james.bognar/git/apache/juneau/master
-cat SONARQUBE_ISSUES_SUMMARY.md
-```
-
-### 2. List All Categories
-
-```bash
-python3 scripts/view-sonar-category.py
-```
-
-### 3. View Specific Category Details
-
-```bash
-# View Security Issues (high priority)
-python3 scripts/view-sonar-category.py "Security Issues"
-
-# View with custom limit
-python3 scripts/view-sonar-category.py "Brain Methods (Complexity)" --limit 5
-```
-
-### 4. Start Interactive Fixing Session
-
-```bash
-python3 scripts/categorize-sonar-issues.py /Users/james.bognar/Downloads/SonarQubeIssues.txt
-```
-
-## Tools
-
-### `categorize-sonar-issues.py`
-
-Main script for categorizing and interactively fixing SonarQube issues.
-
-**Usage:**
-```bash
-python3 scripts/categorize-sonar-issues.py <sonarqube-issues-file> [--save-json]
-```
-
-**Features:**
-- Parses SonarQube TSV export file
-- Categorizes issues into 30 categories
-- Interactive session for fixing issues category by category
-- Saves categorized JSON for later reference
-
-**Options:**
-- `--save-json`: Save categorized issues to JSON file
-
-**Interactive Commands:**
-- `fix` - Start fixing issues in current category
-- `skip` - Move to next category
-- `details` - Show detailed issue information
-- `list` - List all categories
-- `<number>` - Jump to specific category number
-- `quit` - Exit
-
-### `view-sonar-category.py`
-
-Helper script to view category details without interactive input.
-
-**Usage:**
-```bash
-# List all categories
-python3 scripts/view-sonar-category.py
-
-# View specific category
-python3 scripts/view-sonar-category.py "Category Name"
-
-# View with limit
-python3 scripts/view-sonar-category.py "Category Name" --limit 10
-```
-
-## Recommended Workflow
-
-### Step 1: Review Summary
-
-Read `SONARQUBE_ISSUES_SUMMARY.md` to understand:
-- Total issues and categories
-- Priority levels
-- Recommended fix order
-
-### Step 2: Explore Categories
-
-```bash
-# List all categories
-python3 scripts/view-sonar-category.py
-
-# Review high-priority categories
-python3 scripts/view-sonar-category.py "Security Issues"
-python3 scripts/view-sonar-category.py "Null Check Issues"
-python3 scripts/view-sonar-category.py "Brain Methods (Complexity)"
-```
-
-### Step 3: Start Fixing
-
-Begin with high-priority categories:
-
-```bash
-python3 scripts/categorize-sonar-issues.py /Users/james.bognar/Downloads/SonarQubeIssues.txt
-```
-
-Then:
-1. Navigate to high-priority categories (Security, Null Checks, Brain Methods)
-2. For each category:
- - Review sample issues with `details`
- - Decide on fix strategy (auto-fix vs manual review)
- - Use `fix` to start fixing issues
- - Process by file (recommended) or individually
-
-### Step 4: Track Progress
-
-The categorized JSON file (`SonarQubeIssues.categorized.json`) can be used to:
-- Track which categories have been fixed
-- Generate progress reports
-- Identify remaining issues
-
-## Category Priorities
-
-### High Priority (Fix First)
-- **Security Issues** (6 issues)
-- **Null Check Issues** (15 issues)
-- **Brain Methods (Complexity)** (30 issues)
-- **Missing Exception Handling** (1 issue)
-
-### Medium Priority
-- **Exception Catching Issues** (30 issues)
-- **Generic Type Issues** (121 issues)
-- **Empty Method Implementations** (99 issues)
-- **Test Assertion Issues** (61 issues)
-- **Optional Access Issues** (18 issues)
-- **Code Duplication** (16 issues)
-- **ThreadLocal Cleanup Issues** (7 issues)
-
-### Low Priority
-- **Unused Code** (96 issues)
-- **Missing Private Constructors** (91 issues)
-- All other categories
-
-## Fix Strategies
-
-### Auto-Fix Categories (Mechanical Fixes)
-These can often be fixed automatically or with simple find/replace:
-- Missing @Override Annotations
-- Unnecessary toString() Calls
-- Missing Private Constructors
-- Line Separator Issues
-- Python f-string Issues
-
-### Manual Review Categories (Require Code Review)
-These need careful review and testing:
-- Security Issues
-- Brain Methods (Complexity)
-- Null Check Issues
-- Exception Catching Issues
-- Code Duplication
-
-### Batch Fix Categories (Similar Patterns)
-These can be fixed in batches:
-- Missing Private Constructors
-- Generic Type Issues
-- Field Shadowing
-- Unused Code
-
-## Files
-
-- `SonarQubeIssues.txt` - Original SonarQube export (TSV format)
-- `SonarQubeIssues.categorized.json` - Categorized issues (JSON format)
-- `SONARQUBE_ISSUES_SUMMARY.md` - Summary document
-- `scripts/categorize-sonar-issues.py` - Main categorization tool
-- `scripts/view-sonar-category.py` - Category viewer tool
-
-## Tips
-
-1. **Start Small**: Begin with categories that have few issues to build momentum
-2. **Batch Similar Fixes**: Group similar issues together for efficient fixing
-3. **Test After Each Category**: Run tests after fixing each category
-4. **Use Version Control**: Commit fixes category by category for easier review
-5. **Document Decisions**: For "Empty Method Implementations" and similar categories, document why methods are intentionally empty
-
-## Example Session
-
-```bash
-# 1. View summary
-cat SONARQUBE_ISSUES_SUMMARY.md
-
-# 2. Check Security Issues (high priority, only 6 issues)
-python3 scripts/view-sonar-category.py "Security Issues"
-
-# 3. Start interactive session
-python3 scripts/categorize-sonar-issues.py /Users/james.bognar/Downloads/SonarQubeIssues.txt
-
-# In interactive session:
-# > list # See all categories
-# > 1 # Jump to Security Issues
-# > details # See issue details
-# > fix # Start fixing
-# > [choose file mode] # Process by file
-# > auto # Auto-fix each issue
-# > skip # Move to next category
-```
-
-## Support
-
-For questions or issues with these tools, refer to:
-- `SONARQUBE_ISSUES_SUMMARY.md` - Detailed category descriptions
-- The categorized JSON file for complete issue details
+The scripts described here moved. See `sonarqube.md` for the live SonarCloud findings reporter (`sonarqube.py`), and `view-sonar-category.md` for the legacy offline category viewer.
diff --git a/scripts/apply-eclipse-prefs.md b/scripts/apply-eclipse-prefs.md
new file mode 100644
index 0000000000..ea3686bfed
--- /dev/null
+++ b/scripts/apply-eclipse-prefs.md
@@ -0,0 +1,52 @@
+# apply-eclipse-prefs.py
+
+Copies the tracked Eclipse JDT preference files from `scripts/eclipse-preferences/` into the `.settings/` directory of every Maven module.
+
+## When to use
+
+- You added a module to the reactor, made a fresh checkout, or edited the canonical `.prefs` files.
+- Never hand-edit a module's `.settings/` files. Edit the canonical files and run this script again.
+- `eclipse-warnings.py` reads the copied preferences, so run this first if it reports them missing.
+
+## Usage
+
+```bash
+python3 scripts/apply-eclipse-prefs.py
+python3 scripts/apply-eclipse-prefs.py /path/to/another/checkout
+```
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `root-dir` | path (optional positional) | repository root | Root of the checkout whose modules should be updated. Only `sys.argv[1]` is read. |
+
+## What it does
+
+1. Takes the root from the first argument, or the parent of `scripts/` (`main`).
+2. Scans the reactor groups `juneau-bean`, `juneau-core`, `juneau-examples`, `juneau-microservice`, `juneau-petstore`, `juneau-rest`, `juneau-sc` and `juneau-shaded` (one level down), plus `juneau-integration-tests` and `juneau-secret-macos-keychain`, for directories with a `pom.xml` (`discover_projects`).
+3. Skips poms that declare `<packaging>pom</packaging>` (`is_aggregator_pom`).
+4. Classifies a module as a test module when it is named `juneau-integration-tests` or ends in `-ftest`; everything else is a source module (`is_test_module`).
+5. Creates `.settings/` if needed and copies `org.eclipse.jdt.core.prefs` and `org.eclipse.jdt.ui.prefs` from `eclipse-preferences/source-prefs/` or `eclipse-preferences/test-prefs/` (`apply_preferences`).
+6. Prints a line per module and a summary.
+
+## Output and exit codes
+
+`Preferences applied to <module>` per module, then a summary with total, updated and failed counts.
+
+| Code | Meaning |
+|---|---|
+| 0 | Every module was updated. |
+| 1 | At least one module failed, or a preferences directory or file was missing. |
+
+## Gotchas
+
+- The `.settings/` directories are ignored by git. The copies are local to your machine; only the files under `scripts/eclipse-preferences/` are tracked.
+- Source modules get warnings on resource leaks and unused parameters; test modules get those relaxed.
+- A new top-level reactor group is not discovered until it is added to `REACTOR_GROUPS` in the script. New modules inside existing groups are found automatically.
+- Existing preference files in `.settings/` are overwritten.
+
+## See also
+
+- `eclipse-warnings.py`
+- Docs site: Developer Info, Script: apply-eclipse-prefs (`04.02.07.ScriptApplyEclipsePrefs.md` in juneau-docs)
diff --git a/scripts/apply-eclipse-prefs.py b/scripts/apply-eclipse-prefs.py
index b2189eac1a..f96b33e70f 100755
--- a/scripts/apply-eclipse-prefs.py
+++ b/scripts/apply-eclipse-prefs.py
@@ -16,30 +16,9 @@
# limitations under the License.
"""
-Applies Eclipse IDE preference files to every Juneau Maven module.
-
-Walks the reactor under each REACTOR_GROUPS entry plus STANDALONE_PROJECTS,
-finds every leaf directory with a pom.xml that is NOT a packaging=pom aggregator,
-classifies each as source or test (test = name == 'juneau-integration-tests' or
-*-ftest suffix), and copies the matching .prefs files into each module's
-.settings/ directory.
-
-Two sets of preferences live under scripts/eclipse-preferences/:
-- source-prefs/ — main source modules (warnings on resource leaks, unused params, etc.)
-- test-prefs/ — test modules (those warnings relaxed to ignore)
-
-Each set contains:
-- org.eclipse.jdt.core.prefs — JDT compiler problem severities + warning-token settings
-- org.eclipse.jdt.ui.prefs — JDT UI cleanup-on-save settings
-
-Run this whenever a new Maven module is added to the reactor, or whenever the
-canonical preference baseline is updated. Module discovery is automatic — the
-script does not need to be edited when modules are added or removed (only when
-a new top-level reactor group is introduced).
-
-The .settings/ directories are gitignored locally; this script's output is a
-developer-machine-only convenience and never enters version control. The
-canonical preference files under scripts/eclipse-preferences/ ARE tracked.
+Copies the tracked Eclipse JDT preference files from `scripts/eclipse-preferences/` into the `.settings/` directory of every Maven module.
+
+Full docs: apply-eclipse-prefs.md
"""
import os
diff --git a/scripts/check-bom-completeness.md b/scripts/check-bom-completeness.md
new file mode 100644
index 0000000000..a2315495de
--- /dev/null
+++ b/scripts/check-bom-completeness.md
@@ -0,0 +1,45 @@
+# check-bom-completeness.py
+
+Checks that `juneau-bom/pom.xml` lists every consumer-facing reactor module, and nothing that is not one.
+
+## When to use
+
+- You added or removed a publishable module and want to confirm the BOM matches.
+- A build gate reports a BOM mismatch and you want the detail. `push.py` runs this script before the tests.
+
+## Usage
+
+```bash
+python3 scripts/check-bom-completeness.py
+```
+
+## Options
+
+None.
+
+## What it does
+
+1. Finds the repository root as the parent of `scripts/` and requires `juneau-bom/pom.xml` (`main`).
+2. Walks every `pom.xml` under the root (`collect_publishable`). It skips poms under a `target/` directory, under `juneau-shaded/`, `juneau-examples/`, `juneau-petstore/` and `juneau-distrib/`, and poms whose `artifactId` is in the exclusion set (the BOM itself, the test-support modules, `juneau-integration-tests`, `juneau-run`, `juneau-microservice-examples`, and a few others, listed in `EXCLUDED_ARTIFACTS`). It also skips aggregators (packaging `pom` with `<modules>`). Unparseable poms [...]
+3. Reads the `org.apache.juneau` artifacts from the BOM's `<dependencyManagement>` (`collect_bom_entries`).
+4. Compares the two sets: modules missing from the BOM, and BOM entries that are not publishable modules.
+
+## Output and exit codes
+
+On success, one line: `OK: juneau-bom enumerates all <n> consumer-facing modules.` On failure, the missing and stale artifact ids go to stderr, followed by a fix hint.
+
+| Code | Meaning |
+|---|---|
+| 0 | The BOM is complete and has no stale entries. |
+| 1 | The BOM file is missing, a module is missing from the BOM, or the BOM lists a non-publishable artifact. |
+
+## Gotchas
+
+- When a module is intentionally unpublished, the fix is to add it to `EXCLUDED_ARTIFACTS` or `EXCLUDED_PATH_PREFIXES` in the script, not to the BOM.
+- Only the project's own `artifactId` counts; the parent's is ignored.
+- The scan is a plain `rglob` over the tree, so it also looks at poms in untracked directories.
+
+## See also
+
+- `push.py`, which runs this check before the tests
+- `check-container-tags.py`
diff --git a/scripts/check-bom-completeness.py b/scripts/check-bom-completeness.py
index 4559ed2324..c15c162901 100755
--- a/scripts/check-bom-completeness.py
+++ b/scripts/check-bom-completeness.py
@@ -12,30 +12,9 @@
# * specific language governing permissions and limitations under the License.
# ***************************************************************************************************************************
"""
-Drift guard for the published juneau-bom.
-
-Every consumer-facing reactor module must be enumerated in juneau-bom/pom.xml's
-<dependencyManagement>. This catches the common failure mode where a new
-publishable module is added to the reactor but the maintainer forgets to add it
-to the BOM, so downstreams importing the BOM silently can't omit its version.
-
-A module is "consumer-facing publishable" when it is a leaf artifact
-(packaging jar/bundle, or a leaf packaging=pom dependency bundle with no
-<modules>) AND it is not one of the documented non-published modules:
-
- - reactor aggregators (packaging=pom WITH <modules>) - not artifacts
- - juneau-bom itself
- - juneau-test-utils (test-only support)
- - juneau-test-marshall-utils (test-only support, marshall-level fixtures)
- - juneau-shaded/* (uber-jars, not BOM-managed)
- - juneau-examples/*, juneau-petstore/* (sample apps, not published)
- - juneau-microservice-examples (sample/demo REST resources, not shipped)
- - juneau-integration-tests (test-only)
- - juneau-rest-server-console-ui-freemarker-tests (test-only)
- - juneau-run (build tooling, ships a .py script)
- - juneau-distrib (assembly only)
-
-Exit 0 when the BOM is complete (and free of stale entries); exit 1 otherwise.
+Checks that `juneau-bom/pom.xml` lists every consumer-facing reactor module, and nothing that is not one.
+
+Full docs: check-bom-completeness.md
"""
from __future__ import annotations
diff --git a/scripts/check-container-tags.md b/scripts/check-container-tags.md
new file mode 100644
index 0000000000..c0f59117a0
--- /dev/null
+++ b/scripts/check-container-tags.md
@@ -0,0 +1,46 @@
+# check-container-tags.py
+
+Fails when a test class that boots a Spring Boot, Jetty or similar container is missing its container marker annotation.
+
+## When to use
+
+- You added a test that starts a container and want to confirm it carries the right marker.
+- A build gate reports an untagged container test. `push.py` runs this script before the tests.
+
+## Usage
+
+```bash
+python3 scripts/check-container-tags.py
+```
+
+## Options
+
+None.
+
+## What it does
+
+1. Finds every `src/test/java` directory under the repository root, skipping any path with a `target` component (`discover_test_roots`).
+2. Reads each `*Test.java` file in those trees (`scan_counted`).
+3. Decides that a file needs a tag when its source matches a Spring hint (`@SpringBootTest`, `@AutoConfigureMockMvc`, `SpringExtension`, `SpringApplication.run(`) or a Jetty hint (`JettyMicroservice.create/builder(`, `Microservice.create/builder(`, a `new org.eclipse.jetty...Server(`).
+4. Counts the file as tagged when it contains any of the markers `@SpringbootTest`, `@JettyMicroserviceTest`, `@TomcatMicroserviceTest` or `@MicroserviceTest`, plain or fully qualified.
+5. Lists the files that need a tag and have none.
+
+## Output and exit codes
+
+On success: `Container tag guard passed (<n> container tests tagged).` On failure, a list of offending paths relative to the root and a reminder of the four annotations.
+
+| Code | Meaning |
+|---|---|
+| 0 | Every container-booting test has a marker. |
+| 1 | At least one container-booting test has no marker. |
+
+## Gotchas
+
+- Detection is textual. A hint in a comment or string counts, and a marker in a comment counts as a tag.
+- Only files named `*Test.java` are scanned.
+- The Spring hints look for `@SpringBootTest` (capital B) while the marker annotation is `@SpringbootTest` (lower b); the two are different names.
+
+## See also
+
+- `check-bom-completeness.py`
+- `push.py`, which runs this check before the tests
diff --git a/scripts/check-container-tags.py b/scripts/check-container-tags.py
index fdc8717dff..588e1d236e 100644
--- a/scripts/check-container-tags.py
+++ b/scripts/check-container-tags.py
@@ -16,10 +16,9 @@
# limitations under the License.
#
"""
-Guardrail for container-backed tests.
+Fails when a test class that boots a Spring Boot, Jetty or similar container is missing its container marker annotation.
-Fails when a container-booting test class is missing @SpringbootTest,
-@JettyMicroserviceTest, or @TomcatMicroserviceTest.
+Full docs: check-container-tags.md
"""
from __future__ import annotations
diff --git a/scripts/check-fluent-setter-overrides.md b/scripts/check-fluent-setter-overrides.md
new file mode 100644
index 0000000000..64460c28b5
--- /dev/null
+++ b/scripts/check-fluent-setter-overrides.md
@@ -0,0 +1,44 @@
+# check-fluent-setter-overrides.py
+
+Reports subclasses that do not override an inherited fluent setter, so the narrowed return type is lost in method chains.
+
+## When to use
+
+- You changed a builder or bean hierarchy and want to see which subclasses still return the parent type from an inherited setter.
+- It is informational only; it never fails the build.
+
+## Usage
+
+```bash
+python3 scripts/check-fluent-setter-overrides.py
+```
+
+## Options
+
+None.
+
+## What it does
+
+1. Walks the repository root for `.java` files, skipping dot-directories, `target`, `node_modules`, `build`, `dist`, any directory whose path has a `test` or `juneau-integration-tests` component, and files ending in `Test.java` or `Tests.java` (`find_java_files`).
+2. For each file, uses regular expressions to find top-level `public class X extends Y` declarations (`extract_class_info`).
+3. In each class body, records the public methods that return the class's own type and contain `return this` within the next 500 characters (the fluent setters), and the methods marked `@Override`.
+4. Groups the classes by name and, for each class with a parent found in the scan, looks for parent setters that the child neither overrides nor redefines (`check_missing_overrides`). Setters on a parent that carry `@BeanProp` are skipped.
+5. Prints the findings grouped by child class.
+
+## Output and exit codes
+
+Progress lines with the file, class and setter counts, then either `MISSING OVERRIDES (n found)` with each child class, its file and the missing methods with their parent, or a line saying all setters are overridden.
+
+| Code | Meaning |
+|---|---|
+| 0 | Always, with or without findings. |
+
+## Gotchas
+
+- The analysis is regular-expression based, not a Java parser. Inner classes are ignored, generics on `extends` are stripped, and a parent is matched by simple name, so name collisions across packages can give false results.
+- Because any path component named `test` skips a directory, a checkout that lives under a directory called `test` scans nothing.
+- It scans the whole tree and takes a while.
+
+## See also
+
+- Docs site: Developer Info, Script: check-fluent-setter-overrides (`04.02.09.ScriptCheckFluentSetterOverrides.md` in juneau-docs)
diff --git a/scripts/check-fluent-setter-overrides.py b/scripts/check-fluent-setter-overrides.py
index 1b73a9a36a..ef9f27c27e 100755
--- a/scripts/check-fluent-setter-overrides.py
+++ b/scripts/check-fluent-setter-overrides.py
@@ -12,19 +12,9 @@
# * specific language governing permissions and limitations under the License. *
# ***************************************************************************************************************************
"""
-Script to check for missing fluent setter overrides in the Juneau codebase.
+Reports subclasses that do not override an inherited fluent setter, so the narrowed return type is lost in method chains.
-This script:
-1. Scans all Java files in the source tree
-2. Identifies classes and their parent classes
-3. Finds fluent setter methods (methods that return 'this')
-4. Checks if subclasses override these setters with the correct return type
-5. Reports any missing overrides
-
-A fluent setter is a method that:
-- Starts with 'set' or is a builder-style method
-- Returns the class type (for method chaining)
-- Is public
+Full docs: check-fluent-setter-overrides.md
"""
import os
diff --git a/scripts/check-topic-links.md b/scripts/check-topic-links.md
new file mode 100644
index 0000000000..4c5d4c25df
--- /dev/null
+++ b/scripts/check-topic-links.md
@@ -0,0 +1,44 @@
+# check-topic-links.py
+
+Checks that links to topic pages of the Juneau documentation site, found in the source tree, use the slug and title the topic pages define.
+
+## When to use
+
+- You renamed or added a topic page and want to find Javadoc or markdown links that no longer match.
+- It only reports problems; it never changes a file and never fails the build.
+
+## Usage
+
+```bash
+python3 scripts/check-topic-links.py
+```
+
+## Options
+
+None.
+
+## What it does
+
+1. Reads every `*.md` file (except `README.md`) in `docs/pages/topics` and takes `slug` and `title` from the front matter (`extract_topic_info`).
+2. Walks the repository root, skipping dot-directories and `target`, `node_modules`, `build`, `dist` and `scripts`, and reads `.java`, `.md`, `.xml`, `.properties`, `.txt`, `.adoc` and `.rst` files (`find_topic_links`). Code blocks and inline code are removed from markdown files first.
+3. Collects every `https://juneau.apache.org/docs/topics/<slug>">Title</a>` link with its file and line.
+4. Reports each link whose slug is unknown (`UNKNOWN SLUG`) or whose link text differs from the page title (`TITLE MISMATCH`) (`check_links`).
+
+## Output and exit codes
+
+Progress lines, then either `All topic links are correct!` or a list of warnings, each with file, line, slug, expected and actual title, and the link. Everything goes to stdout.
+
+| Code | Meaning |
+|---|---|
+| 0 | The scan finished, with or without warnings. |
+| 1 | No topics were found, for example because `docs/pages/topics` does not exist. |
+
+## Gotchas
+
+- Warnings never change the exit code.
+- The topic pages are read from `docs/` in this checkout, so it needs that directory present.
+- Link text is compared as written, so a different but acceptable label shows up as a mismatch.
+
+## See also
+
+- Docs site: Developer Info, Script: check-topic-links (`04.03.06.ScriptCheckTopicLinks.md` in juneau-docs)
diff --git a/scripts/check-topic-links.py b/scripts/check-topic-links.py
index bbbad01b3a..a7ecfa28a5 100755
--- a/scripts/check-topic-links.py
+++ b/scripts/check-topic-links.py
@@ -12,13 +12,9 @@
# * specific language governing permissions and limitations under the License. *
# ***************************************************************************************************************************
"""
-Script to check for correct topic links in the Juneau source tree.
+Checks that links to topic pages of the Juneau documentation site, found in the source tree, use the slug and title the topic pages define.
-This script:
-1. Scans /docs/pages/topics for all markdown files
-2. Extracts slug names and titles from the frontmatter
-3. Scans the entire source tree for links to https://juneau.apache.org/docs/topics/SLUG">TITLE</a>
-4. Reports any mismatches between expected and actual slug/title combinations
+Full docs: check-topic-links.md
"""
import os
diff --git a/scripts/cleanup-whitespace.md b/scripts/cleanup-whitespace.md
new file mode 100644
index 0000000000..847ead448a
--- /dev/null
+++ b/scripts/cleanup-whitespace.md
@@ -0,0 +1,47 @@
+# cleanup-whitespace.py
+
+Rewrites Java files in place to normalize whitespace: leading spaces to tabs, no trailing spaces, no repeated blank lines, and no final newline.
+
+## When to use
+
+- A one-off whitespace sweep over a source tree.
+- It edits files in place with no dry-run, so commit or stage your work first.
+
+## Usage
+
+```bash
+python3 scripts/cleanup-whitespace.py
+python3 scripts/cleanup-whitespace.py juneau-core/juneau-marshall
+```
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `root-dir` | path (optional positional) | repository root | Directory to scan for `.java` files. Only `sys.argv[1]` is read. |
+
+## What it does
+
+1. Takes the directory from the first argument, or the parent of `scripts/` (`main`).
+2. Finds every `*.java` file under it, skipping paths with a `target` or `.git` component (`find_java_files`).
+3. For each file (`clean_java_file`): converts each group of four leading spaces to a tab, strips trailing whitespace, collapses runs of blank lines to one, drops blank lines before the last `}`, and removes the trailing newline after the final `}`.
+4. Writes the file back only when its content changed.
+5. Prints the totals.
+
+## Output and exit codes
+
+A `Scanning...` line, the file count, a `Cleaned: <path>` line per modified file, and a summary with scanned, modified and unchanged counts.
+
+| Code | Meaning |
+|---|---|
+| 0 | The scan finished. Read errors on single files are printed to stderr and the file is counted as unchanged. |
+
+## Gotchas
+
+- Files written by this script end without a newline, which is the repository convention for Java sources.
+- The `Summary:` heading is printed from inside the loop body, so it appears once per modified file and not at all when nothing changed.
+- It does not honor `.gitignore`; any `*.java` outside `target` and `.git` is rewritten.
+
+## See also
+
+- Docs site: Developer Info, Script: cleanup-whitespace (`04.02.08.ScriptCleanupWhitespace.md` in juneau-docs)
diff --git a/scripts/cleanup-whitespace.py b/scripts/cleanup-whitespace.py
index c415265036..fd93fc2e0d 100755
--- a/scripts/cleanup-whitespace.py
+++ b/scripts/cleanup-whitespace.py
@@ -16,12 +16,9 @@
# limitations under the License.
"""
-Cleans up whitespace inconsistencies in Java files:
-- Converts leading 4-spaces to tabs
-- Removes consecutive blank lines (max 1 blank line allowed)
-- Removes trailing whitespace from lines
-- Removes blank lines before the final closing brace
-- Ensures file ends with "}" with no trailing newline
+Rewrites Java files in place to normalize whitespace: leading spaces to tabs, no trailing spaces, no repeated blank lines, and no final newline.
+
+Full docs: cleanup-whitespace.md
"""
import os
diff --git a/scripts/coverage.md b/scripts/coverage.md
new file mode 100644
index 0000000000..75b303da03
--- /dev/null
+++ b/scripts/coverage.md
@@ -0,0 +1,59 @@
+# coverage.py
+
+Reports JaCoCo branch and instruction coverage for a source file or folder, merged across every module's test data.
+
+## When to use
+
+- You want the coverage of one class or package after changing it, including the uncovered line numbers.
+- You want a per-module summary of the whole repository (`--by-module`).
+- You need to mark a line as intentionally out of scope for coverage (see Gotchas).
+
+## Usage
+
+```bash
+python3 scripts/coverage.py juneau-core/juneau-marshall/src/main/java/org/apache/juneau/marshall/BitSetFormat.java
+python3 scripts/coverage.py juneau-core/juneau-commons/src/main/java/org/apache/juneau/commons/conversion/
+python3 scripts/coverage.py path/to/File.java --run --branches
+python3 scripts/coverage.py --by-module
+```
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `path` | file or folder (positional) | none | A `.java` file or a source folder under `src/main/java` or `src/test/java`. Absolute, or relative to the repository root. Required unless `--by-module` is used. |
+| `--run`, `-r` | none | off | Re-run the owning module's tests (and its sibling `-tests` module, if any) first to refresh its `jacoco.exec`. |
+| `--branches`, `-b` | none | off | Show only lines with missed branches instead of all uncovered lines. |
+| `--by-module` | none | off | Print a per-module summary table sorted by branch percentage, with a TOTAL row, instead of reporting a path. |
+| `--help`, `-h` | none | off | Print the usage text and exit 0. Also printed when no argument is given. |
+
+## What it does
+
+1. With no arguments or a help flag, prints the usage text and exits 0 (`main`).
+2. Finds the nearest Maven module above the path (`find_maven_module`) and derives the JaCoCo package and file name (`path_to_jacoco_package`).
+3. With `--run`, runs `mvn -pl <module>[,<module>-tests] test -Drat.skip=true -q` (`run_tests`). A failure stops the script.
+4. Collects every module's `target/jacoco.exec`, `target/classes` and `src/main/java`, skipping the shaded, distribution, examples, petstore, microservice, `juneau-sc` and test-utils modules (`collect_jacoco_inputs`, `EXCLUDED_MODULE_MARKERS`).
+5. Runs the JaCoCo CLI to write one combined XML report to `target/coverage/jacoco.xml` (`generate_combined_report`). The CLI jar is taken from `~/.m2`, or fetched with Maven if absent (`jacoco_cli_jar`).
+6. Prints per-file and total branch and instruction coverage with bars and the uncovered line numbers (`report`), or the module table (`by_module_report`).
+
+## Output and exit codes
+
+Per file: branch and instruction percentages, missed counts, and the uncovered lines. Files with excluded lines are marked `(HTT-adjusted)` and list the excluded line numbers with their reasons.
+
+| Code | Meaning |
+|---|---|
+| 0 | A report was printed, or help was shown. |
+| 1 | An error: unknown option, no path, a path outside `src/main/java` or `src/test/java`, no `jacoco.exec` or compiled classes found, failed tests with `--run`, or a JaCoCo failure. |
+
+## Gotchas
+
+- Coverage is the union of every module's execution data, so a class tested only by its own module's unit tests is still counted. Build and test the project first so the `jacoco.exec` files exist.
+- `--run` refreshes only the owning module. The other modules' existing data is merged in as is. It needs upstream module artifacts in `~/.m2` from an earlier `mvn install -DskipTests`.
+- `--run` has no effect with `--by-module`, which prints a note. Run the full test suite first, for example `mvn clean test -Drat.skip=true`.
+- Hard-to-test lines: a standalone `HTT` word in a comment on a line (`// HTT`, `x++; // HTT: reason`, `/* HTT */`) removes that line from the uncovered list and subtracts its counts from the totals. `HTTP` does not match, and a match inside a string literal does not count.
+- The JaCoCo version comes from `jacoco.plugin.version` in the root `pom.xml`. The Java launcher is taken from `JAVA_HOME`, then `~/jdk/default`, then `PATH`. `MAVEN_REPO` or `M2_REPO` can point at a different local repository.
+
+## See also
+
+- `sonarqube.py`
+- `test.py`
diff --git a/scripts/coverage.py b/scripts/coverage.py
index 83b367db11..7da6a09bb4 100755
--- a/scripts/coverage.py
+++ b/scripts/coverage.py
@@ -12,6 +12,14 @@
# * specific language governing permissions and limitations under the License.
# ***************************************************************************************************************************
"""
+Reports JaCoCo branch and instruction coverage for a source file or folder, merged across every module's test data.
+
+Full docs: coverage.md
+"""
+
+from __future__ import annotations
+
+USAGE = """
Coverage reporter for Apache Juneau.
Shows JaCoCo branch and instruction coverage for a source file or folder.
@@ -72,8 +80,6 @@ Examples:
./scripts/coverage.py --by-module
"""
-from __future__ import annotations
-
import os
import re
import subprocess
@@ -624,7 +630,7 @@ def by_module_report(xml_path: Path): # NOSONAR python:S3776 -- Cognitive compl
def main(): # NOSONAR: always returns 0 by design — standard POSIX exit code for success
args = sys.argv[1:]
if not args or "--help" in args or "-h" in args:
- print(__doc__)
+ print(USAGE)
return 0
path_arg = None
diff --git a/scripts/create-mvn-site.md b/scripts/create-mvn-site.md
new file mode 100644
index 0000000000..af5f7b8477
--- /dev/null
+++ b/scripts/create-mvn-site.md
@@ -0,0 +1,54 @@
+# create-mvn-site.py
+
+Builds the Maven site, including the Javadocs, and copies it into the documentation tree's static directory so it can be previewed locally.
+
+## When to use
+
+- You changed Javadoc or site content and want to see it rendered, or to check links.
+- You need `docs/static/site` refreshed for a local documentation preview.
+
+## Usage
+
+```bash
+python3 scripts/create-mvn-site.py
+```
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `--help`, `-h` | none | off | Print usage and exit 0 without building or removing anything. |
+| `--dry-run` | none | off | Print what would be removed and run, then exit 0 without doing it. |
+
+Any other argument exits 2 with usage and does nothing.
+
+## What it does
+
+1. Reads the project version with `mvn help:evaluate -Dexpression=project.version`; on failure it prints a warning and continues with `unknown` (`get_project_version`).
+2. Removes `target/site` if it exists.
+3. Runs `mvn clean compile site` in the repository root, streaming the output to the console and to `create-mvn-site.log` in the root (`run_maven_command`).
+4. Fails if `target/site` was not produced.
+5. Deletes `docs/static/site`, recreates it, and copies everything from `target/site` into it.
+6. Prints a success banner with the preview address.
+
+## Output and exit codes
+
+Timestamped progress lines and the full Maven output on stdout. The Maven output is also written to `create-mvn-site.log` in the repository root, overwritten on each run.
+
+| Code | Meaning |
+|---|---|
+| 0 | The site was built and copied, or `--help` was shown. |
+| 2 | An unknown argument was given; nothing was run or removed. |
+| 1 | Maven failed, `target/site` was not created, or the copy failed. |
+
+## Gotchas
+
+- Arguments are parsed first, so `--help` is safe to run; any run without arguments starts the full build.
+- It runs `mvn clean`, so existing build output in the tree is wiped.
+- The copy target is `docs/static/site` in this checkout, and any previous contents are deleted first.
+- The success banner mentions `http://localhost:3000/site/`, which is the Docusaurus dev-server address; the script does not start that server.
+- It needs `mvn` on the `PATH` and takes several minutes.
+
+## See also
+
+- Docs site: Developer Info, Script: create-mvn-site (`04.03.04.ScriptCreateMvnSite.md` in juneau-docs)
diff --git a/scripts/create-mvn-site.py b/scripts/create-mvn-site.py
index dfcc126b72..41274c56f3 100755
--- a/scripts/create-mvn-site.py
+++ b/scripts/create-mvn-site.py
@@ -16,15 +16,12 @@
# limitations under the License.
"""
-Create Maven Site Script
+Builds the Maven site, including the Javadocs, and copies it into the documentation tree's static directory so it can be previewed locally.
-Generates the Maven site with javadocs and copies it to the Docusaurus static directory
-for local testing and broken link validation.
-
-Usage:
- python3 create-mvn-site.py
+Full docs: create-mvn-site.md
"""
+import argparse
import shutil
import subprocess
import sys
@@ -97,13 +94,28 @@ def get_project_version(juneau_root):
return "unknown"
-def main():
+def parse_args(argv):
+ """Parse the command line; runs before anything is removed or built, so --help is side-effect free."""
+ parser = argparse.ArgumentParser(prog="create-mvn-site.py", description=__doc__.strip().split("\n\n")[0])
+ parser.add_argument("--dry-run", action="store_true", help="Print what would be removed and run, then exit without doing it.")
+ return parser.parse_args(argv)
+
+
+def main(argv=None):
+ args = parse_args(sys.argv[1:] if argv is None else argv)
+
# Get directories
script_dir = Path(__file__).parent
juneau_root = script_dir.parent
docs_dir = juneau_root / "docs"
site_dir = juneau_root / "target" / "site"
static_site_dir = docs_dir / "static" / "site"
+
+ if args.dry_run:
+ print(f"Would remove: {site_dir}")
+ print(f"Would run in {juneau_root}: mvn clean compile site")
+ print(f"Would replace {static_site_dir} with a copy of {site_dir}")
+ return 0
print("Creating Maven site with javadocs for local testing...")
print(f"Working from: {juneau_root}")
diff --git a/scripts/current-release.md b/scripts/current-release.md
new file mode 100644
index 0000000000..cabb21f7f5
--- /dev/null
+++ b/scripts/current-release.md
@@ -0,0 +1,45 @@
+# current-release.py
+
+Prints the current Juneau release version taken from the root `pom.xml`, with any `-SNAPSHOT` suffix removed (for example `9.2.0`).
+
+## When to use
+
+- You need the release version as a plain string in a script or when writing release notes.
+- For the installed Maven tool version, use `maven-version.py` instead.
+
+## Usage
+
+```bash
+python3 scripts/current-release.py
+```
+
+## Options
+
+None.
+
+## What it does
+
+1. Locates `pom.xml` in the repository root, the parent of the `scripts/` directory (`get_current_release`).
+2. Runs `mvn help:evaluate -Dexpression=project.version -q -DforceStdout` in the root and, on success, strips `-SNAPSHOT` and returns the result.
+3. If Maven fails for any reason, prints a warning to stderr and parses `pom.xml` directly, reading the project's own `<version>` element.
+4. Prints the version.
+
+## Output and exit codes
+
+Stdout is the version string only. Warnings and errors go to stderr.
+
+| Code | Meaning |
+|---|---|
+| 0 | A version was printed (from Maven or from the XML fallback). |
+| 1 | `pom.xml` does not exist, or neither method produced a version. |
+
+## Gotchas
+
+- The Maven path needs `mvn` on the `PATH` and may download plugins on a fresh machine. The XML fallback reads only the project's own `<version>`, not an inherited parent version.
+- Only the `-SNAPSHOT` suffix is removed; other qualifiers stay in the output.
+
+## See also
+
+- `maven-version.py`
+- `release.py`
+- Docs site: Developer Info, Script: current-release (`04.02.05.ScriptCurrentRelease.md` in juneau-docs)
diff --git a/scripts/current-release.py b/scripts/current-release.py
index f7dd852071..1959a5d4ad 100755
--- a/scripts/current-release.py
+++ b/scripts/current-release.py
@@ -16,16 +16,9 @@
# limitations under the License.
"""
-Get the current release version from pom.xml.
+Prints the current Juneau release version taken from the root `pom.xml`, with any `-SNAPSHOT` suffix removed (for example `9.2.0`).
-This script parses the root pom.xml file and returns the version number
-with the -SNAPSHOT suffix removed (if present).
-
-Usage:
- python3 scripts/current-release.py
-
-Output:
- Prints the version number (e.g., "9.2.0") to stdout.
+Full docs: current-release.md
"""
import subprocess
diff --git a/scripts/eclipse-warnings.md b/scripts/eclipse-warnings.md
new file mode 100644
index 0000000000..a57cc3f19e
--- /dev/null
+++ b/scripts/eclipse-warnings.md
@@ -0,0 +1,56 @@
+# eclipse-warnings.py
+
+Reproduces Eclipse's compiler warnings for a module without a running Eclipse, by compiling its sources with the `ecj` compiler under the module's own preferences.
+
+## When to use
+
+- You need the same warnings Eclipse's Problems view would show, for one module or all of them, and no Eclipse session is available.
+- You want a before-and-after check of a warning fix.
+
+## Usage
+
+```bash
+python3 scripts/eclipse-warnings.py juneau-rest/juneau-rest-server
+python3 scripts/eclipse-warnings.py --all
+```
+
+Run from the repository root.
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `module-path` | path (positional) | none | Module directory relative to the repository root. Exactly one argument is required. |
+| `--all` | none | off | Check every module that `apply-eclipse-prefs.py` would update, source and test modules. Used in place of `module-path`. |
+
+## What it does
+
+1. Requires exactly one argument; otherwise prints the usage text and exits 1 (`main`).
+2. Locates the `ecj` 3.45.0 jar, `~/.m2/repository/org/eclipse/jdt/ecj/3.45.0/ecj-3.45.0.jar` unless `ECJ_JAR` is set, and exits 2 if it is missing (`find_ecj_jar`).
+3. For `--all`, loads `apply-eclipse-prefs.py` and uses its module discovery (`load_apply_prefs_module`).
+4. For each module, runs `mvn -q dependency:build-classpath` from inside the module directory to get its runtime and test classpath (`build_classpath`, 180 second limit).
+5. Skips the module if it has no `.settings/org.eclipse.jdt.core.prefs` or no Java sources (`run_ecj`).
+6. Runs `java -jar ecj.jar -properties <module prefs> -enableJavadoc -warn:+allJavadoc,invalidJavadoc,javadoc -cp <classpath> -d <temp dir> <sources>` over `src/main/java` and `src/test/java` (300 second limit), with the module's `target/classes` and `target/test-classes` appended to the classpath.
+7. Prints ecj's output and its final `n problems (e errors, w warnings)` line (`summarize`).
+
+## Output and exit codes
+
+A banner per module, ecj's raw diagnostics, and a `>>> <module>: <summary>` line.
+
+| Code | Meaning |
+|---|---|
+| 0 | No module reported problems. |
+| 1 | A usage error, or at least one module's summary was not `0 errors, 0 warnings`. |
+| 2 | The `ecj` jar was not found. |
+
+## Gotchas
+
+- `ECJ_JAR` overrides the jar location. `JAVA_HOME` is preferred over the first `java` on the `PATH`.
+- The classpath uses installed jars from `~/.m2` for other reactor modules, not their current sources. A type that is new or renamed in an upstream module shows as "cannot be resolved" until that module is installed again.
+- Javadoc diagnostics come from flags the script always passes, because the tracked preference files set no Javadoc options.
+- Lines such as "invalid Class-Path header in manifest of jar file" come from third-party jars and can be ignored.
+- It needs `mvn` and `java` on the `PATH`. A slow module can hit the time limits and raise an uncaught timeout.
+
+## See also
+
+- `apply-eclipse-prefs.py`
diff --git a/scripts/eclipse-warnings.py b/scripts/eclipse-warnings.py
index c73218aeac..5ce671c6e7 100644
--- a/scripts/eclipse-warnings.py
+++ b/scripts/eclipse-warnings.py
@@ -16,6 +16,12 @@
# limitations under the License.
"""
+Reproduces Eclipse's compiler warnings for a module without a running Eclipse, by compiling its sources with the `ecj` compiler under the module's own preferences.
+
+Full docs: eclipse-warnings.md
+"""
+
+USAGE = """
Headless reproduction of Eclipse JDT compiler warnings via `ecj`.
`ecj` is the exact JDT compiler Eclipse embeds. Driven by a module's own
@@ -171,7 +177,7 @@ def main():
root_dir = script_dir.parent
if len(sys.argv) != 2:
- print(__doc__)
+ print(USAGE)
sys.exit(1)
ecj_jar = find_ecj_jar()
diff --git a/scripts/ensure-playwright.cjs b/scripts/ensure-playwright.cjs
new file mode 100644
index 0000000000..f7b765345e
--- /dev/null
+++ b/scripts/ensure-playwright.cjs
@@ -0,0 +1,105 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+'use strict';
+/*
+ * Makes sure the browser tests' pinned npm packages are in <dir>/node_modules without running npm when it can be avoided:
+ *
+ * node ensure-playwright.cjs <dir> <npm> <na...@version>...
+ *
+ * 1. Every spec already installed at its pinned version: print "already installed" and stop.
+ * 2. Else a per-user cache holds this exact spec set (~/.cache/juneau-js-tests/<hash of the sorted specs>/node_modules):
+ * copy it into <dir>. A full build runs "mvn clean" first, which removes <dir>, so the cache is what makes the
+ * second and later full runs skip npm.
+ * 3. Else run "<npm> install --no-fund --no-audit --prefix <dir> <specs...>" and fill the cache from the result.
+ *
+ * The cache is keyed by the whole spec set, so a petstore entry never satisfies the views module's larger set, or the
+ * reverse. Exits with npm's status when it had to run it.
+ */
+const fs = require('fs');
+const os = require('os');
+const path = require('path');
+const crypto = require('crypto');
+const { spawnSync } = require('child_process');
+
+const [dir, npm, ...specs] = process.argv.slice(2);
+if (!dir || !npm || specs.length === 0) {
+ process.stderr.write('usage: node ensure-playwright.cjs <dir> <npm> <na...@version>...\n');
+ process.exit(2);
+}
+
+/** @return {{name: string, version: string}} A spec such as "playwright@1.49.1" or "@scope/pkg@1.0.0". */
+function parse(spec) {
+ const at = spec.lastIndexOf('@');
+ return { name: spec.slice(0, at), version: spec.slice(at + 1) };
+}
+
+/** @return {boolean} Whether every spec is installed under {@code modules} at exactly its version. */
+function installed(modules) {
+ return specs.every(spec => {
+ const { name, version } = parse(spec);
+ try {
+ return JSON.parse(fs.readFileSync(path.join(modules, name, 'package.json'), 'utf8')).version === version;
+ } catch (e) {
+ return false;
+ }
+ });
+}
+
+const modules = path.join(dir, 'node_modules');
+if (installed(modules)) {
+ process.stdout.write('already installed: ' + specs.join(' ') + '\n');
+ process.exit(0);
+}
+
+const key = crypto.createHash('sha256').update([...specs].sort().join('\n')).digest('hex').slice(0, 16);
+const cacheRoot = process.env.JUNEAU_JS_TESTS_CACHE || path.join(os.homedir(), '.cache', 'juneau-js-tests');
+const cached = path.join(cacheRoot, key, 'node_modules');
+
+if (installed(cached)) {
+ fs.rmSync(modules, { recursive: true, force: true });
+ fs.mkdirSync(dir, { recursive: true });
+ fs.cpSync(cached, modules, { recursive: true, verbatimSymlinks: true });
+ if (installed(modules)) {
+ process.stdout.write('restored from cache ' + cached + ': ' + specs.join(' ') + '\n');
+ process.exit(0);
+ }
+}
+
+const r = spawnSync(npm, ['install', '--no-fund', '--no-audit', '--prefix', dir, ...specs],
+ { stdio: 'inherit', shell: process.platform === 'win32' });
+if (r.status !== 0)
+ process.exit(r.status === null ? 1 : r.status);
+
+// Fill the cache through a sibling that only this process writes, then move it into place. Losing the rename race to
+// another process that filled the same entry first is fine: its copy is as good as ours.
+try {
+ const target = path.join(cacheRoot, key);
+ const tmp = path.join(cacheRoot, key + '.tmp-' + process.pid);
+ fs.mkdirSync(tmp, { recursive: true });
+ fs.cpSync(modules, path.join(tmp, 'node_modules'), { recursive: true, verbatimSymlinks: true });
+ try {
+ fs.renameSync(tmp, target);
+ } catch (e) {
+ fs.rmSync(tmp, { recursive: true, force: true });
+ if (e.code !== 'ENOTEMPTY' && e.code !== 'EEXIST')
+ throw e;
+ }
+} catch (e) {
+ // The cache is an optimisation; the install itself succeeded.
+ process.stderr.write('could not fill the cache: ' + e.message + '\n');
+}
+process.exit(0);
diff --git a/scripts/gen-symbols-key.md b/scripts/gen-symbols-key.md
new file mode 100644
index 0000000000..825e02836f
--- /dev/null
+++ b/scripts/gen-symbols-key.md
@@ -0,0 +1,49 @@
+# gen-symbols-key.py
+
+Regenerates `juneau-symbols-key.svg`, the labelled review sheet of every icon glyph, from the icon sprite `juneau-symbols.svg`.
+
+## When to use
+
+- You added, removed or edited a glyph in the sprite and need the key sheet brought back in step.
+- You want to check, without writing, that the key sheet is current: use `--check`.
+
+## Usage
+
+```bash
+python3 scripts/gen-symbols-key.py
+python3 scripts/gen-symbols-key.py --check
+```
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `--check` | none | off | Do not write. Exit 1 if the key file differs from what would be generated. |
+
+## What it does
+
+1. Locates the repository root by walking up to a directory that holds both `juneau-core` and `juneau-rest` (`repo_root`).
+2. Reads the sprite `juneau-rest/juneau-rest-server-views/src/main/resources/org/apache/juneau/views/juneau-symbols.svg`.
+3. Collects every `<symbol>...</symbol>` verbatim, in order (`symbols`), and reuses the sprite's own XML declaration and license comment as the header (`license_header`).
+4. Lays the glyphs out in a grid of 10 columns with a label under each (`cell`, `render`). The sheet size follows the glyph count.
+5. Writes `juneau-symbols-key.svg` beside the sprite, or with `--check` compares the generated text with the file on disk.
+
+## Output and exit codes
+
+Default run: `wrote <path> (<n> bytes)`. With `--check`: `OK: ... is up to date` on success; on failure a `STALE:` message on stderr with the first differing byte offset, both excerpts, and the command to run.
+
+| Code | Meaning |
+|---|---|
+| 0 | Key file written, or `--check` found it current. |
+| 1 | `--check` found the key file stale. |
+| non-zero (message on stderr) | Repository root or sprite not found, no `<symbol>` in the sprite, or a symbol without `id` or `viewBox`. |
+
+## Gotchas
+
+- The sprite is the single source of truth. Edit it and rerun this script; do not hand-edit the key file.
+- `--check` is stricter than the JUnit staleness test: it also pins the layout arithmetic.
+- The key file is never served; it exists for review and hand authoring of the artwork.
+
+## See also
+
+- `SymbolsKey_Staleness_Test`, the JUnit test that checks symbol bodies and viewBoxes
diff --git a/scripts/gen-symbols-key.py b/scripts/gen-symbols-key.py
index 551717a7f5..f8b3197dcd 100644
--- a/scripts/gen-symbols-key.py
+++ b/scripts/gen-symbols-key.py
@@ -12,32 +12,9 @@
# * specific language governing permissions and limitations under the License. *
# ***************************************************************************************************************************
"""
-Generates juneau-symbols-key.svg from juneau-symbols.svg.
+Regenerates `juneau-symbols-key.svg`, the labelled review sheet of every icon glyph, from the icon sprite `juneau-symbols.svg`.
-The key file is the human-facing authoring sheet for the icon sprite: it inlines every <symbol> definition from
-the sprite into a <defs> block, then lays the glyphs out in a labelled grid so a reviewer can see the whole set
-at once. It is never served (only juneau-symbols.svg is a registered servable resource) - it ships so that the
-artwork can be reviewed and hand-authored.
-
-Why this exists rather than hand-editing the key file:
-
- The key file carries each glyph's viewBox TWICE - once on the inlined <symbol> and again on the per-cell
- wrapper <svg> that <use>s it - so touching one glyph's artwork means four coordinated hand-edits across two
- files, and adding or removing a glyph resizes the whole sheet. Every one of those is a chance for the two
- files to drift apart silently, because nothing in the build reads the key file at all.
-
- Everything in the key file except the layout arithmetic is copied verbatim from the sprite, so the sprite is
- the single source of truth and this script is the copy.
-
-Usage:
-
- python3 scripts/gen-symbols-key.py # rewrite the key file from the sprite
- python3 scripts/gen-symbols-key.py --check # exit 1 if the key file is not what this script would write
-
---check is the staleness gate. It is strictly stronger than the accompanying JUnit test
-(SymbolsKey_Staleness_Test), which asserts only that the two files' <symbol> bodies and their per-cell wrapper
-viewBoxes agree: --check additionally pins the generated layout, so a hand-edit to the grid arithmetic or the
-sheet size fails here while passing there.
+Full docs: gen-symbols-key.md
"""
import argparse
diff --git a/scripts/maven-version.md b/scripts/maven-version.md
new file mode 100644
index 0000000000..30ab17de9c
--- /dev/null
+++ b/scripts/maven-version.md
@@ -0,0 +1,44 @@
+# maven-version.py
+
+Prints the major version number of the Maven tool installed on this machine (for example `3`).
+
+## When to use
+
+- A script or a human needs to know which Maven major version is on the `PATH` before a version-sensitive build step.
+- For the Juneau project version, use `current-release.py` instead; this script reports the Maven tool, not the project.
+
+## Usage
+
+```bash
+python3 scripts/maven-version.py
+```
+
+## Options
+
+None.
+
+## What it does
+
+1. Runs `mvn -version` and reads stdout, or stderr when stdout is empty (`get_maven_version`).
+2. Looks for `Maven <major>.<minor>` in the text.
+3. Falls back to a `<major>.<minor>` at the very start of the text.
+4. Prints the major number.
+
+## Output and exit codes
+
+Stdout is one integer, such as `3`. Errors go to stderr.
+
+| Code | Meaning |
+|---|---|
+| 0 | A major version was found and printed. |
+| 1 | `mvn` is not on the `PATH`, no version could be parsed from its output, or any other exception. |
+
+## Gotchas
+
+- It needs `mvn` on the `PATH`; the Maven wrapper in the repository is not used.
+- It reports only the major number. `3.9.5` prints `3`.
+
+## See also
+
+- `current-release.py`
+- Docs site: Developer Info, Script: maven-version (`04.02.06.ScriptMavenVersion.md` in juneau-docs)
diff --git a/scripts/maven-version.py b/scripts/maven-version.py
index 3785c29255..abecb3f013 100755
--- a/scripts/maven-version.py
+++ b/scripts/maven-version.py
@@ -16,15 +16,9 @@
# limitations under the License.
"""
-Get the Maven version.
+Prints the major version number of the Maven tool installed on this machine (for example `3`).
-This script runs 'mvn -version' and extracts the major version number.
-
-Usage:
- python3 scripts/maven-version.py
-
-Output:
- Prints the major version number (e.g., "3") to stdout.
+Full docs: maven-version.md
"""
import re
diff --git a/scripts/microbench.md b/scripts/microbench.md
new file mode 100644
index 0000000000..6a5682feb2
--- /dev/null
+++ b/scripts/microbench.md
@@ -0,0 +1,65 @@
+# microbench.py
+
+Runs the JMH micro-benchmarks in `juneau-integration-tests` through Maven and checks or reports the results.
+
+## When to use
+
+- You changed a hot path and want to confirm the observability no-op block still allocates nothing (`observability`).
+- You want ops/s and allocation numbers for serialization, parsing, class metadata and a mock REST round trip (`runtime`).
+
+## Usage
+
+```bash
+python3 scripts/microbench.py observability
+python3 scripts/microbench.py observability --threshold 0
+python3 scripts/microbench.py runtime --quick
+python3 scripts/microbench.py runtime --include 'Json.*' --forks 1 --jfr
+python3 scripts/microbench.py --verbose runtime
+```
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `--verbose`, `-v` | none | off | Stream the full Maven output instead of only its tail. Goes before the subcommand. |
+| `observability` | subcommand | required choice | Run `ObservabilityNoopBenchmark` and assert `gc.alloc.rate.norm` is within the threshold. |
+| `runtime` | subcommand | required choice | Run the runtime suite in `org.apache.juneau.microbench.runtime`. |
+| `--threshold` | BYTES (float) | 8.0 | `observability` only. Maximum allowed `gc.alloc.rate.norm` in bytes per operation. |
+| `--include` | REGEX | the whole suite | `runtime` only. JMH include regex. May be repeated. |
+| `--forks` | N (integer) | per-class value | `runtime` only. Override the fork count. |
+| `--warmup` | N (integer) | per-class value | `runtime` only. Override the warmup iteration count. |
+| `--iterations` | N (integer) | per-class value | `runtime` only. Override the measurement iteration count. |
+| `--time` | SECONDS (integer) | per-class value | `runtime` only. Override the warmup and measurement iteration time. |
+| `--quick` | none | off | `runtime` only. Smoke run: 1 fork, 2 measurement iterations, 1 second iterations. |
+| `--jfr` | none | off | `runtime` only. Also record a JFR file per benchmark (JMH `-prof jfr`). |
+| `--out-dir` | DIR | `juneau-integration-tests/target/jmh-results` | `runtime` only. Where results are written. |
+
+## What it does
+
+1. `observability`: runs `mvn -f juneau-integration-tests/pom.xml -Pmicrobench test-compile exec:java` with the benchmark class as `exec.mainClass` (`run_benchmark`, `cmd_observability`).
+2. Reads the newest `juneau-integration-tests/jmh-results/observability-*.json`, extracts the `gc.alloc.rate.norm` secondary metric, and compares it with `--threshold`.
+3. `runtime`: runs `mvn -pl juneau-integration-tests -Pmicrobench-runtime process-test-classes -Dmicrobench.args=...` from the reactor, so it measures this checkout's code and needs no earlier `mvn install` (`cmd_runtime`).
+4. The runtime results go to `<out-dir>/runtime-<date>-<time>.json`, and with `--jfr` the recordings go to `<out-dir>/jfr-<date>-<time>/<benchmark>/`.
+5. Prints ops/s (or time per operation) and `gc.alloc.rate.norm` per benchmark.
+
+## Output and exit codes
+
+A pass or fail line for the allocation check, and the benchmark table for `runtime`. Without `--verbose`, only the tail of the Maven output is shown.
+
+| Code | Meaning |
+|---|---|
+| 0 | The allocation rate was within the threshold, or the runtime suite ran. |
+| 1 | The rate exceeded the threshold, Maven failed, or no result file could be read. |
+
+## Gotchas
+
+- `--verbose` is a top-level option, so it must come before `observability` or `runtime`.
+- The threshold default is 8 bytes per operation; the source notes it can be tightened to 0 once confirmed.
+- `observability` writes under `juneau-integration-tests/jmh-results/`, while `runtime` writes under `target/`, so runtime runs leave no untracked files.
+- `--warmup` also gives `ColdStartBenchmark` warmups, which defeats it. Exclude that benchmark with `--include` when overriding warmup.
+- To record a new baseline, run the `observability` check and update the `[observability]` section of `perf-baseline.txt` at the project root by hand.
+- A benchmark run can take many minutes. Use `--quick` for a smoke run.
+
+## See also
+
+- `perf-required.py`
diff --git a/scripts/microbench.py b/scripts/microbench.py
index 308bae3107..643eb4327c 100755
--- a/scripts/microbench.py
+++ b/scripts/microbench.py
@@ -12,6 +12,12 @@
# * specific language governing permissions and limitations under the License.
# ***************************************************************************************************************************
"""
+Runs the JMH micro-benchmarks in `juneau-integration-tests` through Maven and checks or reports the results.
+
+Full docs: microbench.md
+"""
+
+USAGE = """
JMH micro-benchmark runner for Apache Juneau.
Usage:
@@ -300,7 +306,7 @@ def main() -> int:
parser = argparse.ArgumentParser(
description="JMH micro-benchmark runner for Apache Juneau.",
formatter_class=argparse.RawDescriptionHelpFormatter,
- epilog=__doc__,
+ epilog=USAGE,
)
parser.add_argument(
"--verbose", "-v",
diff --git a/scripts/normalize-generic-commas.md b/scripts/normalize-generic-commas.md
new file mode 100644
index 0000000000..3f5159fbb6
--- /dev/null
+++ b/scripts/normalize-generic-commas.md
@@ -0,0 +1,53 @@
+# normalize-generic-commas.py
+
+Rewrites generic type arguments such as `Map<K, V>` to `Map<K,V>` in Java source, in code and inside comments and Javadoc.
+
+## When to use
+
+- You are applying the project's generic-comma spacing to existing sources. The Eclipse formatter cannot rewrite text inside comments, so this script covers that half.
+- Use `--check` to see which files would change, for example in a review or a script.
+
+## Usage
+
+```bash
+python3 scripts/normalize-generic-commas.py juneau-core
+python3 scripts/normalize-generic-commas.py path/to/File.java other/dir
+python3 scripts/normalize-generic-commas.py --check juneau-core
+```
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `--check` | none | off | Dry run. Report the files that would change and write nothing. Must be the first argument. |
+| `path` | one or more (positional) | none | A `.java` file, or a directory walked recursively. `target/` and `.git/` are skipped. At least one is required. |
+
+## What it does
+
+1. With no path, prints the module's usage text and exits 1 (`main`).
+2. Collects the `.java` files under each path (`find_java_files`).
+3. Segmentation pass: classifies every character as code, line comment, block comment (Javadoc included), string, char or text block, honoring escapes (`transform_source`). Strings, chars and text blocks are copied through untouched.
+4. Generic-span pass: in code and comment segments, finds a `<`, walks forward tracking nesting depth, and accepts only characters that can appear in a type-argument list. The first other character abandons the match and the `<` is left alone.
+5. Inside an accepted span, collapses only a comma followed by spaces or tabs to a bare comma. Bounded wildcards are untouched.
+6. Rewrites each changed file in place, or only reports it with `--check`, and prints a summary.
+
+## Output and exit codes
+
+`Changed: <path> (code=n, comment=m)` per file, or `WOULD CHANGE:` with `--check`, then a summary of files scanned, files changed, and comma sites tightened in code and in comments.
+
+| Code | Meaning |
+|---|---|
+| 0 | Done, or `--check` found nothing to change. |
+| 1 | No path was given, or `--check` found files that would change. |
+
+## Gotchas
+
+- It is a scanner, not a parser, and it only misses sites. It never rewrites a comma outside a validated `<...>` span.
+- Known misses: a span interrupted by a comment, and type-use annotations inside the arguments (`List<@NonNull String>`).
+- A span must contain at least one letter or `?`, so lists of comparison operators like `<123, >=123` are not treated as generics.
+- Spans longer than 2000 characters are not matched (`_MAX_SPAN`).
+- Without `--check` it edits files directly. Review the diff afterwards.
+
+## See also
+
+None.
diff --git a/scripts/normalize-generic-commas.py b/scripts/normalize-generic-commas.py
index bd6148bd39..b5747bd873 100644
--- a/scripts/normalize-generic-commas.py
+++ b/scripts/normalize-generic-commas.py
@@ -16,6 +16,12 @@
# limitations under the License.
"""
+Rewrites generic type arguments such as `Map<K, V>` to `Map<K,V>` in Java source, in code and inside comments and Javadoc.
+
+Full docs: normalize-generic-commas.md
+"""
+
+USAGE = """
Normalize generic type-argument comma spacing.
Rewrites `Foo<A, B>` -> `Foo<A,B>` (at every nesting level, e.g.
@@ -358,7 +364,7 @@ def main():
check_only = True
args = args[1:]
if not args:
- print(__doc__)
+ print(USAGE)
return 1
all_files = []
diff --git a/scripts/perf-required.md b/scripts/perf-required.md
new file mode 100644
index 0000000000..0b8204c84f
--- /dev/null
+++ b/scripts/perf-required.md
@@ -0,0 +1,56 @@
+# perf-required.py
+
+Answers one question for a change: does it touch a performance-sensitive hot-path file?
+
+## When to use
+
+- Your landing notes call for a performance section whenever a change touches the core hot-path classes, and you want to know whether this change does.
+- It checks only that file trigger. A rule that depends on how many tests changed is not covered and stays a manual check.
+
+## Usage
+
+```bash
+python3 scripts/perf-required.py
+python3 scripts/perf-required.py --against origin/master
+python3 scripts/perf-required.py --files path/to/RestContext.java path/to/Other.java
+python3 scripts/perf-required.py --list-hot-paths
+```
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `--against` | ref | auto-detected upstream default branch | Compare against the merge base of `HEAD` and this ref instead. |
+| `--files` | one or more paths | none | Check this explicit file list and do not run git. |
+| `--list-hot-paths` | none | off | Print the hot-path file names, one per line, and exit 0. |
+
+## What it does
+
+1. With `--list-hot-paths`, prints the inventory and exits 0 (`main`).
+2. With `--files`, uses that list. Otherwise computes the touched files with `git diff --name-only <merge-base>`, which includes uncommitted changes (`touched_files`).
+3. The upstream branch is `--against` when given, else the target of `refs/remotes/origin/HEAD`, else the `HEAD branch` reported by `git remote show origin` (`detect_upstream_default_branch`).
+4. With no usable upstream, falls back to `git diff --name-only HEAD`, the uncommitted changes only.
+5. Keeps the files whose base name is one of: `RestContext.java`, `Settings.java`, `VarResolver.java`, `PropertyVar.java`, `BeanContext.java`, `BeanStore.java`, `MockRestClient.java`, `ValueResolver.java`, `FieldInfo.java`, `ParameterInfo.java` (`hot_path_hits`).
+6. Prints `yes` with the matching files, or `no`.
+
+## Output and exit codes
+
+The first line starts with `yes` or `no`, followed by the matching files for `yes` and a reminder that the test-count trigger is a manual check.
+
+| Code | Meaning |
+|---|---|
+| 0 | No hot-path file touched. |
+| 1 | At least one hot-path file touched. |
+| 2 | The diff could not be computed (not a git repository, an unresolvable `--against` ref, or a failed git command). |
+
+## Gotchas
+
+- The exit code is the answer, so it can be used in a shell conditional. Exit 1 means "yes", not an error.
+- New files that git does not yet track are not seen by `git diff`; add them to the index first, or pass them with `--files`.
+- Matching is by base name only, so a file with one of these names anywhere in the tree counts.
+- An explicit `--against` that cannot be resolved is an error, not a silent fallback to `HEAD`.
+- `--help` prints the full description of the rule as its epilog.
+
+## See also
+
+- `test.py`, whose `--enforce-perf` mode compares test timings against a baseline
diff --git a/scripts/prompt-pgp-passphrase.md b/scripts/prompt-pgp-passphrase.md
new file mode 100644
index 0000000000..604ecee1e0
--- /dev/null
+++ b/scripts/prompt-pgp-passphrase.md
@@ -0,0 +1,47 @@
+# prompt-pgp-passphrase.py
+
+Makes a throwaway `gpg --clearsign` call so the PGP passphrase is asked for (and cached by the agent) at the start of a long run, not at the end when signing is needed.
+
+## When to use
+
+- You are about to start a long build or release and want the passphrase prompt out of the way first. `push.py` and `release.py` call it for you.
+- To check that your signing key and passphrase work, run it by hand.
+
+## Usage
+
+```bash
+python3 scripts/prompt-pgp-passphrase.py
+```
+
+## Options
+
+None.
+
+## What it does
+
+1. Prints a prompt banner and writes a small temporary text file containing `dummy` (`prompt_pgp_passphrase`).
+2. Finds `gpg`: first on the `PATH`, then the program named by `git config gpg.program`, then `/opt/homebrew/bin/gpg`, `/usr/local/bin/gpg` and `/usr/bin/gpg` (`_resolve_gpg`).
+3. Runs `gpg --yes --clearsign <tmpfile>` with a 60 second timeout. It is not run in batch mode, so gpg asks for the passphrase on the terminal.
+4. Passes gpg's stderr through, dropping the harmless "error retrieving key fingerprint from card" line.
+5. Deletes the temporary file and its `.asc` signature.
+6. Prints a success line, or a warning that gpg exited non-zero.
+
+## Output and exit codes
+
+Status lines on stdout, gpg's remaining stderr on stderr.
+
+| Code | Meaning |
+|---|---|
+| 0 | gpg signed the file, or the step was skipped harmlessly (gpg not found, timeout, or another setup error). |
+| 1 | gpg ran and exited non-zero, for example a wrong passphrase. |
+
+## Gotchas
+
+- It is best-effort. `push.py` and `release.py` treat a non-zero exit as a warning only and carry on.
+- A missing `gpg` or a timeout still exits 0, so exit 0 does not prove the passphrase was accepted.
+- It needs an interactive terminal for the passphrase prompt.
+
+## See also
+
+- `push.py`, `release.py`
+- Docs site: Developer Info, Script: prompt-pgp-passphrase (`04.02.04.ScriptPromptPgpPassphrase.md` in juneau-docs)
diff --git a/scripts/prompt-pgp-passphrase.py b/scripts/prompt-pgp-passphrase.py
index 37478a906f..a3c323f240 100755
--- a/scripts/prompt-pgp-passphrase.py
+++ b/scripts/prompt-pgp-passphrase.py
@@ -16,13 +16,9 @@
# limitations under the License.
"""
-Prompt for PGP Passphrase Script
+Makes a throwaway `gpg --clearsign` call so the PGP passphrase is asked for (and cached by the agent) at the start of a long run, not at the end when signing is needed.
-This script makes a dummy PGP call to prompt for passphrase early in the execution.
-This ensures the user is prompted for their PGP passphrase at the beginning
-rather than waiting until signing is needed near the end of the process.
-
-Usage: python3 scripts/prompt-pgp-passphrase.py
+Full docs: prompt-pgp-passphrase.md
"""
import os
diff --git a/scripts/push-timings.md b/scripts/push-timings.md
new file mode 100644
index 0000000000..b093de14cc
--- /dev/null
+++ b/scripts/push-timings.md
@@ -0,0 +1,56 @@
+# push-timings.py
+
+Compares the latest test run's timings against the repository's recorded history and reports modules that got slower or faster.
+
+## When to use
+
+- You want to see whether the last run of `test.py` was noticeably slower than usual, per module and test bucket.
+- `push.py` loads this file as a module for its Timing row; you can also run it by hand.
+
+## Usage
+
+```bash
+python3 scripts/push-timings.py
+python3 scripts/push-timings.py --window 10 --report /tmp/timings.txt
+python3 scripts/push-timings.py --history path/to/test.jsonl
+python3 scripts/push-timings.py --repo ~/git/apache/juneau
+```
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `--repo` | directory | this script's repository | Repository whose run history to read. Mutually exclusive with `--history`. |
+| `--history` | file | none | A `test.py` history records file (JSON lines) to read instead of the repository's store. Mutually exclusive with `--repo`. |
+| `--window` | N (integer) | 20 | Number of earlier runs used for the rolling median of each module and bucket. |
+| `--report` | file | `test-timings.txt` beside the history store | Where to write the full table. |
+
+## What it does
+
+1. Reads the `tests` records that `test.py` writes to the run-history store shared by every branch and worktree of the repository, or the file given with `--history` (`load_history`, `rows_from_records`).
+2. Keeps only records with the same command shape as the newest one, so a run with different options never skews the baseline.
+3. Turns each record into rows: one per module and bucket from its Surefire seconds, plus a reactor row from the wall-clock time.
+4. Stops with a one-line message when history is off, there is no history, the latest run failed, or fewer than two runs of this shape exist.
+5. Compares each module and bucket of the latest run with the median of up to `--window` earlier passing runs (`analyse`). It is slower only when it exceeds the median by more than the relative threshold and by more than the minimum number of seconds, and faster by the same two margins (`classify`).
+6. Writes the full table to the report file and prints a summary line plus the slower entries.
+
+## Output and exit codes
+
+One summary line with the counts and the report path, then one `⚠` line per slower entry. The report file lists every entry with a mark.
+
+| Code | Meaning |
+|---|---|
+| 0 | Always, including when there is nothing to compare. |
+
+## Gotchas
+
+- `JUNEAU_PUSH_TIMING_THRESHOLD` sets the relative margin (default `0.20`) and `JUNEAU_PUSH_TIMING_MIN_SECONDS` the absolute one (default `5`). Unparseable values fall back to the defaults.
+- `JUNEAU_RUN_HISTORY=0` turns the history store off; the script then reports that and exits 0.
+- The report file is replaced on every run.
+- The history comes from `test.py` runs, so a repository with no recorded runs reports "no timing history yet".
+
+## See also
+
+- `test.py`, which writes the history
+- `push.py`, whose Timing row uses this script
+- `surefire-rollup.py`
diff --git a/scripts/push-timings.py b/scripts/push-timings.py
index 21f299d168..7f6e18af01 100644
--- a/scripts/push-timings.py
+++ b/scripts/push-timings.py
@@ -16,20 +16,9 @@
# limitations under the License.
#
"""
-Report timing deltas from the repo's run history: test.py's `tests` records (juneau_run's HistoryStore, script
-"test"), shared by every branch and worktree of the repo.
-
-Each record becomes today's rows: one per (module, bucket) of its surefire_s.<bucket> series, and a reactor/suite row
-from its wall_s. Only records with the newest record's sig (its command shape) count toward the baseline; a shape with
-fewer than two runs gets a "no baseline for this command shape" line, not a verdict. An entry (one module and bucket
-of the latest run) is flagged slower only when it is both more than JUNEAU_PUSH_TIMING_THRESHOLD (default 0.20) and
-more than JUNEAU_PUSH_TIMING_MIN_SECONDS (default 5) slower than its rolling median, and faster when it beats the
-median by both margins. The full table goes to test-timings.txt beside test.jsonl in the store directory; the console
-gets one summary line and the slower entries.
-push.py's Timing row loads this file as a module and uses load_history(), report_path(), latest_run(), analyse(),
-write_report() and counts().
-
- push-timings.py [--repo DIR | --history FILE] [--window N] [--report FILE]
+Compares the latest test run's timings against the repository's recorded history and reports modules that got slower or faster.
+
+Full docs: push-timings.md
"""
from __future__ import annotations
diff --git a/scripts/push.md b/scripts/push.md
new file mode 100644
index 0000000000..35612da5a1
--- /dev/null
+++ b/scripts/push.md
@@ -0,0 +1,71 @@
+# push.py
+
+Builds, tests, commits and pushes Juneau (and, when present, the sibling juneau-docs repository) in one command.
+
+## When to use
+
+- You have staged your reviewed changes and want them verified and pushed to the tracked upstream.
+- You want only the build and test gates, with no commit or push: use `--test-only`.
+- You changed only juneau-docs: use `--docs-only`.
+- To run just the tests, use `test.py` directly instead.
+
+## Usage
+
+```bash
+python3 scripts/push.py "Fixed bug in RestClient"
+python3 scripts/push.py "Updated topic page" --docs-only
+python3 scripts/push.py --test-only
+python3 scripts/push.py "Quick fix" --skip-tests --dry-run
+```
+
+The commit message is optional on the command line. It is prompted for when omitted, except with `--test-only`.
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `message` | text (optional positional) | prompted | Git commit message. An empty or missing message aborts with exit 1 (`resolve_commit_message`). |
+| `--test-only` | none | off | Run only the build and test gates. No identity check, commit, push or juneau-docs follow-up. Cannot be combined with `--docs-only`, `--sonarqube`, `--tracker-audit` or `--skip-tests`. |
+| `--skip-tests` | none | off | Skip the test step. The build then runs `mvn clean package install -DskipTests`. |
+| `--dry-run` | none | off | Print the steps that would run and exit 0. |
+| `--docs-only` | none | off | Operate only on juneau-docs: skip all Juneau build, test, commit and push steps. The identity and docs-verification gates still run. `--skip-tests` and `--sonarqube` are ignored with a note. |
+| `--sonarqube`, `--sonar` | none | off | Opt-in gate: run `sonarqube.py` in whole-repository mode and block the push if SonarCloud reports any issue. |
+| `--tracker-audit`, `--todo-audit` | none | off | Opt-in gate for maintainers who keep an external work-tracker checkout: blocks the push if its audit flags a file. |
+| `--js-tests` | none | auto | Force the headless-browser JS harness. Fails if Node or npm is missing. Mutually exclusive with `--no-js-tests`. |
+| `--no-js-tests` | none | off | Never run the JS harness, even when JS, CSS or FTL files changed. |
+| `--console` | `full`, `condensed`, `none` | `condensed` | Maven output on the console. Also `JUNEAU_RUN_CONSOLE`; the flag wins. |
+| `--detail` | `summary`, `actionable`, `modules`, `all` | `actionable` | Console detail. Also `JUNEAU_RUN_DETAIL`. |
+
+## What it does
+
+1. Parses and validates the arguments. A bad combination exits 2 (`parse_args`).
+2. Prechecks, before any build work (`prechecks`): git `user.email` is the committer identity the script requires (`REQUIRED_GIT_EMAIL`), the tree has no unstaged or untracked-and-not-ignored changes, and the branch is not behind its upstream. The same three run for juneau-docs when it is present and has staged changes or unpushed commits.
+3. Optional gates, when asked: SonarQube (`run_sonarqube_gate`), tracker audit (`run_tracker_audit_gate`).
+4. Unless `--skip-tests`: `check-container-tags.py`, `check-bom-completeness.py`, then `test.py --full` (which builds, installs and runs the Java, Python and, when needed, browser tests), then `push-timings.py`.
+5. Commits whatever is already staged and pushes to the tracked upstream (`commit_and_push`). It stages nothing itself.
+6. Repeats the follow-up for juneau-docs when it exists (`run_docs_followup`), after its docs build check.
+
+## Output and exit codes
+
+Progress is a condensed run view. Full logs go under `~/.cache/juneau-run/`, and the session header prints the `tail -f` line for the log. Test timings are appended to `~/.cache/juneau-run/history/<repo>-<sha8>/test.jsonl`.
+
+| Code | Meaning |
+|---|---|
+| 0 | Pushed, nothing to push, `--dry-run`, `--test-only` passed, or `--docs-only` with no changes. |
+| 1 | A gate failed, the prechecks refused, or no commit message was given. |
+| 2 | Invalid argument combination. |
+| 130 | Cancelled with Ctrl-C or, with run markers on, SIGTERM. |
+
+## Gotchas
+
+- It never stages for you. Anything unstaged or untracked-and-not-ignored makes the prechecks refuse, and the check repeats just before the commit.
+- The commit takes the index exactly as it finds it, so stage the whole set you want pushed.
+- `JUNEAU_PUSH_NO_SOUND` silences the completion sound.
+- `--test-only` here means "gates without a commit or push". `test.py --test-only` means "tests without the build".
+- The upstream check does a `git fetch`, so it needs the network. A failed fetch fails closed.
+- With `--sonarqube`, a branch that SonarCloud has not analyzed yet is skipped with a warning, not failed. The result reflects the last CI analysis, not the local diff.
+
+## See also
+
+- `test.py`, `sonarqube.py`, `push-timings.py`, `check-container-tags.py`, `check-bom-completeness.py`
+- Docs site: Developer Info, Script: push (`04.02.02.ScriptPush.md` in juneau-docs)
diff --git a/scripts/push.py b/scripts/push.py
index 25b4f95320..32dd292c7c 100755
--- a/scripts/push.py
+++ b/scripts/push.py
@@ -16,28 +16,9 @@
# limitations under the License.
"""
-Build and Push Script for Juneau
-
-This script automates the build, test, and deployment workflow:
-0. Prechecks: refuses up front unless git has the ASF committer identity, the tree has no unreviewed
- (unstaged/untracked) changes and the branch isn't behind its upstream -- for juneau and, when it exists, the
- sibling juneau-docs -- before any build or test work (commit_and_push() repeats the last two at the end)
-1. Runs all tests
-2. Builds and installs the project
-3. Commits changes to Git
-4. Pushes to remote repository
-
-Usage: python3 push.py "commit message"
- python3 push.py "commit message" --skip-tests
- python3 push.py "commit message" --sonarqube
- python3 push.py "commit message" --tracker-audit
- python3 push.py "commit message" --docs-only
- python3 push.py "commit message" --js-tests (force the headless-browser JS harness)
- python3 push.py "commit message" --no-js-tests (never run it)
- python3 push.py --test-only (build and test gates only; no commit or push)
-
-JS harness (-Pjs-tests): with neither flag, test.py runs it automatically when a .js/.css/.ftl file
-under a src/ tree differs from origin/master, and skips it with a notice if Node/npm are missing.
+Builds, tests, commits and pushes Juneau (and, when present, the sibling juneau-docs repository) in one command.
+
+Full docs: push.md
"""
import argparse
@@ -986,7 +967,8 @@ def run_docs_only(args, juneau_root):
repo = docs_root if docs_root.exists() else juneau_root
return _in_session("Push", lambda s: _docs_only_steps(s, args, console, docs_root, missing),
- header=session_header("Juneau Docs", current_branch(repo), args.message), mode="push", repo=repo,
+ header=session_header("Juneau Docs", current_branch(repo)), lines=session_lines(args.message),
+ mode="push", repo=repo,
plan=[("prechecks", "Prechecks"), ("docs-smoke", DOCS_BUILD_LABEL), ("commit", "Commit"),
("push", "Push")])
@@ -1103,16 +1085,21 @@ def push_plan(args, script_dir, juneau_root, console):
return [(step_id, PLAN_LABELS[step_id]) for step_id in ids]
-def session_header(project, branch, message):
- """The session header's items after the title: the project, the branch, then the commit message when there is one
- (none with --test-only)."""
- return [project, branch] + ([message] if message else [])
+def session_header(project, branch):
+ """The session header's items after the title: the project and the branch."""
+ return [project, branch]
+
+
+def session_lines(message):
+ """The session's header lines under the title: the commit message when there is one (none with --test-only)."""
+ return [f"Commit Message: {message}"] if message else []
-def _in_session(title, body, *, header, mode, repo, plan):
+def _in_session(title, body, *, header, lines=(), mode, repo, plan):
"""Run body(session) -- which returns an exit code -- inside the run session; returns the session's exit code."""
- with toolkit("run").session(title, header=header, bell=True, sound=True, silence_env=(PUSH_NO_SOUND_ENV,),
- mode=mode, project=repo.name, repo=repo, exit=False, plan=plan) as session:
+ with toolkit("run").session(title, header=header, lines=lines, bell=True, sound=True,
+ silence_env=(PUSH_NO_SOUND_ENV,), mode=mode, project=repo.name, repo=repo, exit=False,
+ plan=plan) as session:
code = body(session)
if code:
raise SystemExit(code)
@@ -1331,7 +1318,8 @@ def _main(): # NOSONAR python:S3776 -- Cognitive complexity is acceptable for t
return _in_session("Test" if args.test_only else "Push",
lambda s: _push_steps(s, args, run, console, script_dir, juneau_root),
- header=session_header("Juneau", current_branch(juneau_root), None if args.test_only else args.message),
+ header=session_header("Juneau", current_branch(juneau_root)),
+ lines=session_lines(None if args.test_only else args.message),
mode="test" if args.test_only else "push", repo=juneau_root,
plan=push_plan(args, script_dir, juneau_root, console))
diff --git a/scripts/release.md b/scripts/release.md
new file mode 100644
index 0000000000..bea8ba4d20
--- /dev/null
+++ b/scripts/release.md
@@ -0,0 +1,76 @@
+# release.py
+
+Walks through the Apache Juneau release steps: checks, build, `release:prepare` and `release:perform`, binary artifacts, SVN upload and distribution check.
+
+## When to use
+
+- It is deprecated. At startup it prints a warning that Juneau releases now run from the separate release manager project, and that this script will be removed after the next release.
+- Use it only to continue or revert a release that was started with it.
+
+## Usage
+
+```bash
+python3 scripts/release.py --list-steps
+python3 scripts/release.py
+python3 scripts/release.py --resume
+python3 scripts/release.py --start-step run_release_prepare --skip-step clean_maven_repo
+python3 scripts/release.py --revert
+```
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `--list-steps` | none | off | List the step names in order and exit. |
+| `--start-step` | step name | first step | Start at this step, skipping the ones before it. Mutually exclusive with `--resume`. |
+| `--resume` | none | off | Start after the last step recorded in the checkpoint file. |
+| `--skip-step` | step name | none | Skip this step. May be repeated. |
+| `--revert` | none | off | Revert a release: delete the git tag, reset the Maven versions and remove the RC from SVN. |
+| `--console` | `full`, `condensed`, `none` | `full` | Maven output on the console. Also `JUNEAU_RUN_CONSOLE`; the flag wins. |
+| `--detail` | `summary`, `actionable`, `modules`, `all` | `actionable` | Console detail. Also `JUNEAU_RUN_DETAIL`. |
+
+## What it does
+
+The steps, in order (`RELEASE_STEPS`):
+
+1. `check_prerequisites`: requires `wget`, `gpg`, `svn`, `git`, `java` and `mvn` on the `PATH`.
+2. `check_java_version` and `check_maven_version`: Java 17 or newer, Maven 3 or newer.
+3. `clean_maven_repo`: moves `~/.m2/repository` aside and deletes it in the background, only on the first run for a version.
+4. `make_git_folder`: recreates the staging directory (default `~/tmp/dist-release-juneau`), deleting any existing one.
+5. `clone_juneau` and `configure_git`: clones the project from the Apache git server into staging and sets the committer name and email.
+6. `run_clean_verify`: `mvn clean verify`.
+7. `run_deploy`: `mvn deploy` with MD5, SHA-1 and SHA-512 checksums.
+8. `run_release_prepare`: requires a `-SNAPSHOT` version, then `mvn release:prepare` with the release version, tag and next development version.
+9. `run_git_diff`: shows `git diff <tag>` in a pager so you can review what `release:prepare` changed.
+10. `run_release_perform`: `mvn release:perform`, opens the Nexus staging page, and asks for the staging repository name after you close it by hand.
+11. `create_binary_artifacts`: checks out the SVN dev area, downloads the source and binary zips from the staging repository, renames them to `apache-juneau-<version>-src.zip` and `-bin.zip`, and commits them under the release candidate name.
+12. `verify_distribution`: checks the six expected files exist and are not empty, opens the SVN page, and prints a draft vote email.
+
+Before the steps it prompts for the release candidate number, staging directory, Apache username and email, git branch and `JAVA_HOME`, with the last answers as defaults. `--revert` runs four steps: pull, delete the tag, reset the versions with `release:update-versions`, and remove the RC directories from SVN (`revert_release`).
+
+## Output and exit codes
+
+A configuration banner, then a row per step with the full output written to `~/.juneau-release-logs/<release>-<timestamp>.log`. After a failure it prints the command that resumes the run.
+
+| Code | Meaning |
+|---|---|
+| 0 | Every selected step finished. |
+| 1 | A step failed, or a prerequisite is missing. |
+| 2 | An invalid option or step name. |
+| 130 | Cancelled with Ctrl-C or Ctrl-D at a prompt, or the revert confirmation was declined. |
+
+## Gotchas
+
+- The checkpoint is in `~/.juneau-release-state.json` and the per-version answers in `release-history-<version>.json` beside the script. A corrupt checkpoint stops the run so a release never resumes from a guess.
+- `release:perform` uploads to the public Apache staging repository, and the SVN steps commit to the public dev distribution area. Do not click Release in Nexus until the vote has passed.
+- `make_git_folder` deletes the whole staging directory on every run that includes it. Use `--start-step` or `--resume` to avoid redoing it.
+- `--revert` deletes the remote tag with `git push origin :<tag>` and can commit SVN deletions after asking.
+- Steps need GPG and SVN credentials. The script primes the gpg agent first with `prompt-pgp-passphrase.py`.
+- `RUN_MARKERS` is ignored, with a warning.
+
+## See also
+
+- `prompt-pgp-passphrase.py`
+- `current-release.py`
+- `maven-version.py`
+- Docs site: Developer Info, Script: release (`04.02.03.ScriptRelease.md` in juneau-docs)
diff --git a/scripts/release.py b/scripts/release.py
index 4e916b14a3..ed10c83500 100755
--- a/scripts/release.py
+++ b/scripts/release.py
@@ -12,6 +12,12 @@
# * specific language governing permissions and limitations under the License. *
# ***************************************************************************************************************************
"""
+Walks through the Apache Juneau release steps: checks, build, `release:prepare` and `release:perform`, binary artifacts, SVN upload and distribution check.
+
+Full docs: release.md
+"""
+
+USAGE = """
Apache Juneau Release Script
This script automates the release process for Apache Juneau, including:
@@ -1229,7 +1235,7 @@ def main(argv=None):
parser = argparse.ArgumentParser(
description='Apache Juneau Release Script',
formatter_class=argparse.RawDescriptionHelpFormatter,
- epilog=__doc__
+ epilog=USAGE
)
# --list-steps, --start-step, --skip-step and --resume, validated against the step names
registry = build_registry(RELEASE_STEPS, lambda name: lambda ctx: None) # only parses; never runs a step
diff --git a/scripts/repin-consumers.md b/scripts/repin-consumers.md
new file mode 100644
index 0000000000..219c5afa63
--- /dev/null
+++ b/scripts/repin-consumers.md
@@ -0,0 +1,59 @@
+# repin-consumers.py
+
+Rebuilds Juneau's `HEAD` and installs it into the shared `~/.m2` repository, so downstream projects that resolve the floating `10.0.0-SNAPSHOT` version pick up the new build.
+
+## When to use
+
+- You changed Juneau and a downstream project (Release Manager or another consumer) should build against the change.
+- "Re-pin" means exactly this install. There is no version property to bump and no pin file to edit.
+
+## Usage
+
+```bash
+python3 scripts/repin-consumers.py
+python3 scripts/repin-consumers.py --verify-consumers
+python3 scripts/repin-consumers.py --juneau-dir ~/git/apache/juneau-9.2.1 --verify-consumers
+```
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `--juneau-dir` | path | `~/git/apache/juneau` | Juneau repository root to build and install. |
+| `--verify-consumers` | none | off | After a successful install, run `mvn -B -nsu clean test-compile` in each consumer checkout and report pass or fail. Never runs git in them. |
+| `--release-manager-dir` | path | `~/git/apache/release-manager` | Release Manager checkout, used only with `--verify-consumers`. |
+| `--support-console-dir` | path | `SUPPORT_CONSOLE_DIR_DEFAULT` in the script | Second consumer checkout, used only with `--verify-consumers`. |
+
+## What it does
+
+1. Resolves the Juneau root and exits 2 if it is not a directory or has no `.git` (`main`).
+2. Guard: runs `git status --porcelain` and refuses, exit 1, if any line is returned. Modified, staged and untracked non-ignored files all count (`check_dirty`).
+3. Prints the short hash and subject of `HEAD`, and warns if `HEAD` differs from the locally known `origin/master` (`describe_head`). No `git fetch` is run, so the comparison can be stale.
+4. Sets `JAVA_HOME` to `~/jdk/default` if that exists, otherwise keeps the environment's value (`resolve_java_home`).
+5. Runs `mvn -B -nsu clean install -DskipTests` in the Juneau tree with live output (`run_streaming`). `-nsu` stops Maven from pulling a published snapshot that would shadow the local build.
+6. With `--verify-consumers`, runs `mvn -B -nsu clean test-compile` in each consumer checkout (`verify_consumer`). A missing directory counts as a failure.
+7. Prints a summary: the version read from the root `pom.xml` (`read_pom_version`), the commit, and the install location under `~/.m2/repository/org/apache/juneau/`.
+
+## Output and exit codes
+
+Progress lines, Maven's own output, then a `=== Summary ===` block.
+
+| Code | Meaning |
+|---|---|
+| 0 | The install succeeded and, with `--verify-consumers`, every consumer compiled. |
+| 1 | The dirty-tree guard refused. Nothing was built or installed. |
+| 2 | Usage error, or the Juneau tree or its `pom.xml` could not be found or read. |
+| 3 | The Maven install failed. `~/.m2` may be partly updated. |
+| 4 | The install succeeded but at least one consumer failed to compile. |
+
+## Gotchas
+
+- There is no `--allow-dirty` flag, on purpose. Installing a dirty tree puts unreviewed work into a repository shared by every build on the machine. Commit, stash or discard the changes first.
+- A clean tree is not the same as a pushed one. Unpushed commits are installed, with a warning.
+- Every project on the machine that resolves the snapshot sees the new build on its next build.
+- Consumer checkouts are only built, never modified. Nothing runs git in them.
+
+## See also
+
+- `push.py`, whose unreviewed-changes gate this guard mirrors
+- `release.py`
diff --git a/scripts/repin-consumers.py b/scripts/repin-consumers.py
index 5321ee38a5..57d5a2537f 100755
--- a/scripts/repin-consumers.py
+++ b/scripts/repin-consumers.py
@@ -12,6 +12,14 @@
# * specific language governing permissions and limitations under the License.
# ***************************************************************************************************************************
"""
+Rebuilds Juneau's `HEAD` and installs it into the shared `~/.m2` repository, so downstream projects that resolve the floating `10.0.0-SNAPSHOT` version pick up the new build.
+
+Full docs: repin-consumers.md
+"""
+
+from __future__ import annotations
+
+USAGE = """
Re-pin the downstream consumers (Release Manager, Support Console) to a fresh local Apache
Juneau build.
@@ -81,8 +89,6 @@ Exit status:
The install already happened; this is reported, not treated as an install failure.
"""
-from __future__ import annotations
-
import argparse
import os
import re
@@ -206,7 +212,7 @@ def main() -> int:
"build. Refuses to run if the Juneau tree is not completely clean -- see this "
"script's module docstring for why there is no override flag."
),
- epilog=__doc__,
+ epilog=USAGE,
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument(
diff --git a/scripts/revert-staged.md b/scripts/revert-staged.md
new file mode 100644
index 0000000000..3212c0ffcf
--- /dev/null
+++ b/scripts/revert-staged.md
@@ -0,0 +1,53 @@
+# revert-staged.py
+
+Discards the changes to one git-tracked file by restoring it from `HEAD`, the last commit.
+
+## When to use
+
+- You want to throw away all in-progress edits to a single file and get the committed version back.
+- To keep staged work and drop only later edits, use `revert-unstaged.py` instead.
+
+## Usage
+
+```bash
+python3 scripts/revert-staged.py juneau-core/juneau-marshall/src/main/java/org/apache/juneau/ClassMeta.java
+```
+
+Run it from the repository root; the path is passed to git as given, after a `--` so a path that starts with `-` or matches a ref name is read as a path. For a path that starts with `-`, put `--` before it on the command line too.
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `file_path` | path (positional, required) | none | The tracked file to restore. Exactly one argument is required; any other count prints the usage text to stderr and exits 2. |
+| `-h`, `--help` | none | off | Print the usage text and exit 0 without calling git. |
+
+## What it does
+
+1. Parses the arguments with argparse (`main`), so `--help` and bad arguments never reach git.
+2. Runs `git ls-files --error-unmatch -- <file_path>`; an untracked path stops the script.
+3. Prints a warning and the command it is about to run.
+4. Runs `git restore --source=HEAD -- <file_path>`.
+5. Prints a success line, or git's stderr on failure.
+
+## Output and exit codes
+
+Progress lines go to stdout: the warning, the `git restore` command line, and either a success or an error line. Usage errors from argparse go to stderr.
+
+| Code | Meaning |
+|---|---|
+| 0 | `git restore` succeeded. |
+| 1 | The file is not tracked by git, or `git restore` failed. |
+| 2 | Wrong number of arguments (usage text on stderr). |
+
+## Gotchas
+
+- The banner says "staged and unstaged", but the script runs `git restore --source=HEAD -- <file>` without `--staged`. Git restores the working tree only in that form, so the index is left alone: a change that was already staged stays staged (`git diff --cached` still shows it) and `git status` can show the file as modified against the index. To also unstage, run `git restore --staged <file>` yourself.
+- Run from inside the repository; `--` makes the path literal, so it is not read as a ref or option.
+- There is no confirmation prompt. The working-tree edits are gone once the command runs.
+- Only one file per run. It does not accept directories or globs through its own checks; git decides what a path means.
+
+## See also
+
+- `revert-unstaged.py`
+- Docs site: Developer Info, Script: revert-staged (`04.02.10.ScriptRevertStaged.md` in juneau-docs)
diff --git a/scripts/revert-staged.py b/scripts/revert-staged.py
index 0c9886f444..2bb8035e09 100755
--- a/scripts/revert-staged.py
+++ b/scripts/revert-staged.py
@@ -12,64 +12,44 @@
# * specific language governing permissions and limitations under the License.
# ***************************************************************************************************************************
"""
-Revert staged changes back to HEAD (last commit).
+Discards the changes to one git-tracked file by restoring it from `HEAD`, the last commit.
-This script reverts both staged AND unstaged changes back to the last committed version.
-⚠️ WARNING: This will discard all changes (staged and unstaged) for the specified file.
-
-Usage:
- ./scripts/revert-staged.py <file_path>
-
-Example:
- ./scripts/revert-staged.py juneau-core/juneau-marshall/src/main/java/org/apache/juneau/ClassMeta.java
+Full docs: revert-staged.md
"""
-import sys
+import argparse
import subprocess
-from pathlib import Path
+import sys
+
+
+def git(*args):
+ return subprocess.run(["git", *args], capture_output=True, text=True)
+
-def main():
- if len(sys.argv) != 2:
- print("Error: Exactly one file path argument required")
- print()
- print("Usage: ./scripts/revert-staged.py <file_path>")
- print()
- print("Example:")
- print(" ./scripts/revert-staged.py juneau-core/juneau-marshall/src/main/java/org/apache/juneau/ClassMeta.java")
- sys.exit(1)
-
- file_path = sys.argv[1]
-
- # Verify file exists or is tracked by git
- try:
- subprocess.run(
- ["git", "ls-files", "--error-unmatch", file_path],
- check=True,
- capture_output=True,
- text=True
- )
- except subprocess.CalledProcessError:
+def main(argv=None):
+ parser = argparse.ArgumentParser(
+ description="Discard the changes to one tracked file by restoring it from HEAD.",
+ epilog="Example: ./scripts/revert-staged.py juneau-core/juneau-marshall/src/main/java/org/apache/juneau/ClassMeta.java"
+ )
+ parser.add_argument("file_path", help="the file to restore (use -- before a path that starts with -)")
+ file_path = parser.parse_args(argv).file_path
+
+ if git("ls-files", "--error-unmatch", "--", file_path).returncode != 0:
print(f"Error: File not tracked by git: {file_path}")
- sys.exit(1)
-
- print(f"⚠️ WARNING: This will discard ALL changes (staged and unstaged) for: {file_path}")
- print(f"Command: git restore --source=HEAD {file_path}")
+ return 1
+
+ print(f"WARNING: This will discard ALL changes (staged and unstaged) for: {file_path}")
+ print(f"Command: git restore --source=HEAD -- {file_path}")
print()
-
- try:
- subprocess.run(
- ["git", "restore", "--source=HEAD", file_path],
- check=True,
- capture_output=True,
- text=True
- )
- print(f"✅ Successfully reverted to HEAD for: {file_path}")
- print(" (All staged and unstaged changes discarded)")
- return 0
- except subprocess.CalledProcessError as e:
- print(f"❌ Error reverting file: {e.stderr}")
+
+ result = git("restore", "--source=HEAD", "--", file_path)
+ if result.returncode != 0:
+ print(f"Error reverting file: {result.stderr}")
return 1
+ print(f"Successfully reverted to HEAD for: {file_path}")
+ print(" (All staged and unstaged changes discarded)")
+ return 0
+
if __name__ == "__main__":
sys.exit(main())
-
diff --git a/scripts/revert-unstaged.md b/scripts/revert-unstaged.md
new file mode 100644
index 0000000000..2777ff515b
--- /dev/null
+++ b/scripts/revert-unstaged.md
@@ -0,0 +1,54 @@
+# revert-unstaged.py
+
+Restores one tracked file's working-tree edits to the staged (index) version, keeping what is already staged.
+
+## When to use
+
+- You staged a file, kept editing it, and want to drop only the edits made after staging.
+- To restore the committed version instead, use `revert-staged.py`.
+
+## Usage
+
+```bash
+python3 scripts/revert-unstaged.py juneau-core/juneau-marshall/src/main/java/org/apache/juneau/ClassMeta.java
+```
+
+Run it from the repository root; the path is passed to git as given. For a path that starts with `-`, put `--` first: `python3 scripts/revert-unstaged.py -- -odd.txt`.
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `file_path` | path (positional, required) | none | The file to restore. Exactly one is required; a missing one prints the usage text and exits 2. |
+| `-h`, `--help` | none | off | Print the usage text and exit 0; no git command runs. |
+
+## What it does
+
+1. Parses the arguments (`main`).
+2. Checks that the path exists on disk.
+3. Checks that git tracks the path (`git ls-files --error-unmatch`); an untracked file has no staged version.
+4. Checks for unstaged edits (`git diff --quiet`); if there are none it prints "Nothing to revert" and exits 0.
+5. Prints the command it is about to run.
+6. Runs `git restore --worktree -- <file_path>`, which copies the staged version over the working-tree file.
+7. Prints a success line, or git's stderr on failure.
+
+## Output and exit codes
+
+Progress lines go to stdout: the `git restore` command line, then a success or an error line.
+
+| Code | Meaning |
+|---|---|
+| 0 | The file was restored, or it had no unstaged edits. |
+| 1 | The path does not exist, git does not track it, or `git restore` failed. |
+| 2 | Wrong arguments (argparse usage error). |
+
+## Gotchas
+
+- A file with no staged version (new and untracked) is refused; a tracked file that was never staged is restored to the committed version, since the index holds that.
+- When the restore does work, the edits made since staging are lost with no prompt.
+- Only one file per run.
+
+## See also
+
+- `revert-staged.py`
+- Docs site: Developer Info, Script: revert-unstaged (`04.02.11.ScriptRevertUnstaged.md` in juneau-docs)
diff --git a/scripts/revert-unstaged.py b/scripts/revert-unstaged.py
index 2df940e0f7..6ebcff9c34 100755
--- a/scripts/revert-unstaged.py
+++ b/scripts/revert-unstaged.py
@@ -12,57 +12,53 @@
# * specific language governing permissions and limitations under the License.
# ***************************************************************************************************************************
"""
-Revert unstaged changes back to the staged (INDEX) version.
+Restores one git-tracked file's working-tree edits to the staged (index) version, keeping what is already staged.
-This script safely reverts working directory changes back to what's in the staging area,
-preserving any staged changes that have been tested.
-
-Usage:
- ./scripts/revert-unstaged.py <file_path>
-
-Example:
- ./scripts/revert-unstaged.py juneau-core/juneau-marshall/src/main/java/org/apache/juneau/ClassMeta.java
+Full docs: revert-unstaged.md
"""
-import sys
+import argparse
import subprocess
+import sys
from pathlib import Path
-def main():
- if len(sys.argv) != 2:
- print("Error: Exactly one file path argument required")
- print()
- print("Usage: ./scripts/revert-unstaged.py <file_path>")
- print()
- print("Example:")
- print(" ./scripts/revert-unstaged.py juneau-core/juneau-marshall/src/main/java/org/apache/juneau/ClassMeta.java")
- sys.exit(1)
-
- file_path = sys.argv[1]
-
- # Verify file exists
+
+def git(*args):
+ return subprocess.run(["git", *args], capture_output=True, text=True)
+
+
+def main(argv=None):
+ parser = argparse.ArgumentParser(
+ description="Restore one tracked file's working-tree edits to the staged (index) version; staged changes are kept.",
+ epilog="Example: ./scripts/revert-unstaged.py juneau-core/juneau-marshall/src/main/java/org/apache/juneau/ClassMeta.java"
+ )
+ parser.add_argument("file_path", help="the file to restore (use -- before a path that starts with -)")
+ file_path = parser.parse_args(argv).file_path
+
if not Path(file_path).exists():
print(f"Error: File not found: {file_path}")
- sys.exit(1)
-
+ return 1
+
+ if git("ls-files", "--error-unmatch", "--", file_path).returncode != 0:
+ print(f"Error: File not tracked by git, so it has no staged version: {file_path}")
+ return 1
+
+ if git("diff", "--quiet", "--", file_path).returncode == 0:
+ print(f"Nothing to revert: {file_path} already matches the staged version")
+ return 0
+
print(f"Reverting unstaged changes for: {file_path}")
- print(f"Command: git restore --source=INDEX {file_path}")
+ print(f"Command: git restore --worktree -- {file_path}")
print()
-
- try:
- subprocess.run(
- ["git", "restore", "--source=INDEX", file_path],
- check=True,
- capture_output=True,
- text=True
- )
- print(f"✅ Successfully reverted unstaged changes for: {file_path}")
- print(" (Staged changes preserved)")
- return 0
- except subprocess.CalledProcessError as e:
- print(f"❌ Error reverting file: {e.stderr}")
+
+ result = git("restore", "--worktree", "--", file_path)
+ if result.returncode != 0:
+ print(f"Error reverting file: {result.stderr}")
return 1
+ print(f"Successfully reverted unstaged changes for: {file_path}")
+ print(" (Staged changes preserved)")
+ return 0
+
if __name__ == "__main__":
sys.exit(main())
-
diff --git a/scripts/script-docs-exclude.txt b/scripts/script-docs-exclude.txt
new file mode 100644
index 0000000000..1a5e344e66
--- /dev/null
+++ b/scripts/script-docs-exclude.txt
@@ -0,0 +1,16 @@
+# ***************************************************************************************************************************
+# * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an
+# * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+# * specific language governing permissions and limitations under the License.
+# ***************************************************************************************************************************
+# Exceptions to the script-docs checker (scripts/tests/test_script_docs.py).
+# One per line: [flags:|packages:] <pattern> # <reason> (the reason is required)
+# A pattern that matches nothing fails the test, so remove entries when the script goes away.
+# No exceptions are needed at present.
diff --git a/scripts/slack-notify.md b/scripts/slack-notify.md
new file mode 100644
index 0000000000..33323a4c71
--- /dev/null
+++ b/scripts/slack-notify.md
@@ -0,0 +1,53 @@
+# slack-notify.py
+
+Posts a text message to a Slack Workflow Builder webhook so it arrives as a real push notification, and prints the send time so a caller can find the post.
+
+## When to use
+
+- A long task is done, or needs a decision, and you want a push to your phone.
+- A caller needs to post a message and then read replies in the same thread.
+
+## Usage
+
+```bash
+python3 scripts/slack-notify.py "Build finished."
+echo "Some message" | python3 scripts/slack-notify.py -
+```
+
+Words after the script name are joined with spaces. A lone `-` reads the message from stdin.
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `message` | text (zero or more words; `-` alone for stdin) | none, required in practice | Message text. Empty after trimming exits 3. |
+
+## What it does
+
+1. Builds the message from the arguments, or from stdin when the only argument is `-` (`_read_message`).
+2. Reads `notify_webhook_url` from `.cursor/slack.local.yml` in the repository root with a small built-in parser, so PyYAML is not needed (`_load_webhook_url`).
+3. Requires the URL to start with `https://hooks.slack.com/triggers/`. A `https://slack.com/shortcuts/...` URL means the workflow was created with the wrong trigger type.
+4. Records the time, then POSTs `{"text": "<message>"}` as JSON with a 15 second timeout.
+5. On HTTP 200, prints `T0=<unix seconds>`.
+
+## Output and exit codes
+
+Stdout on success is one line, `T0=<float>`. Errors go to stderr.
+
+| Code | Meaning |
+|---|---|
+| 0 | The webhook accepted the payload. |
+| 1 | `notify_webhook_url` is missing from the config, or does not look like a Workflow Builder webhook URL. |
+| 2 | The webhook returned an HTTP error, a non-200 status, or the connection failed. |
+| 3 | No message was given. |
+
+## Gotchas
+
+- The webhook response carries no message timestamp. To thread-reply or poll for an answer, a caller waits a few seconds, reads the newest messages in the notifications channel, and picks the bot-authored one whose timestamp is at least `T0 - 5`.
+- The workflow posts as the workflow app, not as you, which is why a push is delivered; messages you author yourself do not push to you.
+- In the workflow, the trigger must be "From a webhook" and the "Send a message" step must target the notifications channel, not you. See `.cursor/slack.local.example.yml` if your checkout has it.
+- `.cursor/slack.local.yml` is a local, per-person file and is not committed.
+
+## See also
+
+None.
diff --git a/scripts/slack-notify.py b/scripts/slack-notify.py
index f8e3dda410..071ad0ecc8 100755
--- a/scripts/slack-notify.py
+++ b/scripts/slack-notify.py
@@ -14,79 +14,9 @@
# See the License for the specific language governing permissions and
# limitations under the License.
"""
-Slack notification helper for Apache Juneau.
+Posts a text message to a Slack Workflow Builder webhook so it arrives as a real push notification, and prints the send time so a caller can find the post.
-Posts a text payload to the Slack Workflow Builder webhook configured in
-``.cursor/slack.local.yml`` (key ``notify_webhook_url``). The workflow then
-posts the message into your personal notifications channel (the channel
-configured as the "Send a message" recipient in the workflow — by
-convention the same channel you set as ``default_channel_id``).
-
-Why this exists
-===============
-
-Messages sent through the ``user-slack`` MCP are authored as *you*, and
-Slack deliberately suppresses pushes for messages the recipient sent
-themselves. A Slack Workflow Builder webhook posts from the workflow app's
-identity instead, which does push on iOS/Android — same code path as a
-message from any other human or app you share a channel with.
-
-Because the workflow posts to a real channel (not a DM), the message has a
-``message_ts`` you can thread-reply to. That's what makes "ask me on Slack
-and wait" round-trips work: the push wakes your phone, you tap to open
-Slack, reply in the thread, and Cursor polls that same thread for the
-reply — one message, one thread, one conversation.
-
-This is the backing script for:
-
-- ``/slack notify <message>`` in ``.cursor/commands/slack.md`` (fire-and-
- forget push).
-- The "Notify me on Slack when done" / "Ask me on Slack and wait" trigger
- phrases in ``AGENTS.md``. The wait-loop path combines this script with a
- subsequent ``slack_read_channel`` lookup to resolve the posted message's
- ``message_ts``.
-
-Matching the posted message
-===========================
-
-The Workflow Builder webhook response doesn't include a ``message_ts`` for
-the message the workflow ends up posting, so callers that need the
-timestamp (``/slack notify``, ``/slack ask``, the AGENTS.md wait loop)
-have to locate the post in the channel after the fact. This script prints
-the *pre-send* Unix timestamp on stdout as ``T0=<float>`` on success.
-Callers should:
-
-1. Sleep ~3–5 seconds to let Slack's workflow engine run.
-2. ``slack_read_channel`` on ``default_channel_id`` (limit 5).
-3. Pick the newest message authored by a bot (author id starts with ``B``)
- whose ``message_ts`` is ``>= T0 - 5`` (small jitter tolerance). That's
- the workflow-posted message.
-
-No token is appended to the message body — the post goes out clean.
-
-Usage
-=====
-
-::
-
- python3 scripts/slack-notify.py "Build finished."
- echo "Some message" | python3 scripts/slack-notify.py -
-
-Exit codes
-==========
-
-:0: webhook accepted the payload (HTTP 200); ``T0=<float>`` printed on stdout.
-:1: webhook URL not configured in ``slack.local.yml``.
-:2: HTTP error posting to the webhook.
-:3: missing message argument.
-
-Setup
-=====
-
-See ``.cursor/slack.local.example.yml`` for the full Slack Workflow
-Builder setup (trigger type **must** be "From a webhook"; the "Send a
-message" step recipient **must** be your ``default_channel_id``
-notifications channel, not yourself).
+Full docs: slack-notify.md
"""
from __future__ import annotations
diff --git a/scripts/sonarqube.md b/scripts/sonarqube.md
new file mode 100644
index 0000000000..19420b3571
--- /dev/null
+++ b/scripts/sonarqube.md
@@ -0,0 +1,72 @@
+# sonarqube.py
+
+Shows SonarCloud findings (rule, severity, line, message) for a source file, a package directory, a Maven module, or the whole repository.
+
+## When to use
+
+- You want the live Sonar findings for the code you are changing, with filters by severity, rule or type.
+- You want a repository-wide summary by module and rule, or a pass/fail gate on matching issues (`--fail-on-issues`).
+- You want findings for a branch or pull request analysis instead of the default branch.
+
+## Usage
+
+```bash
+python3 scripts/sonarqube.py juneau-core/juneau-marshall/src/main/java/org/apache/juneau/sse/
+python3 scripts/sonarqube.py juneau-core/juneau-marshall --severity BLOCKER,CRITICAL
+python3 scripts/sonarqube.py path/to/File.java --with-suppress-hint
+python3 scripts/sonarqube.py --all --run
+python3 scripts/sonarqube.py --all --rule java:S3776 --detail
+python3 scripts/sonarqube.py --all --pr 1234 --run --fail-on-issues
+```
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `path` | file, directory or module (positional) | none | A `.java` file, a package directory under `src/main/java` or `src/test/java`, or a module root. Absolute or relative to the repository root. Omitting it selects whole-repository mode. |
+| `--all` | none | implied without a path | Whole-repository mode: totals, a per-module table and the top rules. |
+| `--run`, `-r` | none | off | Fetch the issues from SonarCloud again and overwrite the local cache. |
+| `--severity` | SEV[,SEV...] | all | Filter by severity: `BLOCKER`, `CRITICAL`, `MAJOR`, `MINOR`, `INFO`. Also accepted as `--severity=...`. |
+| `--rule` | rule id | all | Filter by rule id, for example `java:S3776`. May be repeated. Also accepted as `--rule=...`. |
+| `--type` | TYPE[,TYPE...] | all | Filter by type: `CODE_SMELL`, `BUG`, `VULNERABILITY`, `SECURITY_HOTSPOT`. Also accepted as `--type=...`. |
+| `--branch` | name | `master` | SonarCloud branch to query. Cannot be combined with `--pr`. |
+| `--pr` | id | none | SonarCloud pull-request analysis to query. Cannot be combined with `--branch`. |
+| `--with-suppress-hint` | none | off | Append a `@SuppressWarnings` hint to each finding. |
+| `--detail` | none | off | Whole-repository mode only: also print the per-file findings after the summary. |
+| `--max` | N (integer) | 200 | Cap on the number of findings printed. |
+| `--fail-on-issues` | none | off | Exit 2 if any issue remains after the filters. Output is unchanged. |
+| `--help`, `-h` | none | off | Print the usage text and exit 0. Also printed when no argument is given. |
+
+## What it does
+
+1. Parses the options by hand and rejects unknown ones, a second positional, `--branch` together with `--pr`, and bad severity or type values (`parse_args`).
+2. Reads the cache in `target/.sonar-issues.json` unless `--run` is given (`load_cache`).
+3. With no cache, or with `--run`, fetches all issues for project `apache_juneau` in organization `apache` from the SonarCloud web API, 500 per page, up to a 5000 issue cap (`fetch_all_issues`).
+4. For a non-default branch or pull request, first checks that SonarCloud has an analysis for it (`scope_analyzed`). If not, it warns and exits 3.
+5. Applies the severity, rule and type filters, and the path filter (`filter_issues`, `filter_issues_all`).
+6. Prints the findings for the path (`report`), or the totals, module table and top rules (`report_whole_repo`).
+
+## Output and exit codes
+
+Findings grouped by file, each with severity, rule, line and message. Counts by severity are summarized. A line says whether the cache was used and which branch or pull request it holds.
+
+| Code | Meaning |
+|---|---|
+| 0 | Normal, or clean with `--fail-on-issues`, or help was shown. |
+| 1 | An error: bad option or value, missing path, network or authorization failure. |
+| 2 | `--fail-on-issues` was given and matching issues remain. |
+| 3 | The requested branch or pull request has no SonarCloud analysis yet. It is a skip, not a pass. |
+
+## Gotchas
+
+- The cache holds one scope. If it was fetched for a different branch or pull request, the script prints a notice; use `--run` to refresh.
+- It queries the public project anonymously. If `SONAR_TOKEN` is set, it is sent as a bearer token to raise the rate limit and is never printed. A 429 response suggests setting it.
+- Findings come from the last CI analysis, so they lag local edits. A new branch or pull request has nothing to report until CI has analyzed it.
+- `--detail` has an effect only in whole-repository mode.
+- `--max` limits what is printed, not what `--fail-on-issues` counts.
+- Network access to `sonarcloud.io` is required whenever the cache is missing or `--run` is used.
+
+## See also
+
+- `coverage.py`, the sibling tool this one mirrors
+- `view-sonar-category.py`, the legacy offline viewer
diff --git a/scripts/sonarqube.py b/scripts/sonarqube.py
index 6bdb95a6e9..3539b00803 100755
--- a/scripts/sonarqube.py
+++ b/scripts/sonarqube.py
@@ -12,6 +12,14 @@
# * specific language governing permissions and limitations under the License.
# ***************************************************************************************************************************
"""
+Shows SonarCloud findings (rule, severity, line, message) for a source file, a package directory, a Maven module, or the whole repository.
+
+Full docs: sonarqube.md
+"""
+
+from __future__ import annotations
+
+USAGE = """
SonarQube/SonarCloud findings reporter for Apache Juneau.
Shows per-file SonarCloud findings (rule id, severity, line, message) for a
@@ -91,8 +99,6 @@ Examples:
./scripts/sonarqube.py --all --pr 1234 --run --fail-on-issues
"""
-from __future__ import annotations
-
import json
import os
import sys
@@ -687,7 +693,7 @@ def parse_csv_arg(raw: str, valid: list, label: str) -> set:
def parse_args(argv): # NOSONAR java:S3776 -- argparse-by-hand mirror of coverage.py; complexity acceptable
args = argv[1:]
if not args or "--help" in args or "-h" in args:
- print(__doc__)
+ print(USAGE)
sys.exit(0)
path_arg = None
diff --git a/scripts/start-petstore-jetty.md b/scripts/start-petstore-jetty.md
new file mode 100644
index 0000000000..3818811e08
--- /dev/null
+++ b/scripts/start-petstore-jetty.md
@@ -0,0 +1,55 @@
+# start-petstore-jetty.py
+
+Starts the Juneau petstore sample application on the embedded Jetty microservice, with plain `java` and a cached Maven classpath.
+
+## When to use
+
+- You want the Jetty flavor of the petstore running locally, on port 10000 or another port you choose.
+- For the Spring Boot flavor, use `start-petstore-springboot.py`.
+
+## Usage
+
+```bash
+python3 scripts/start-petstore-jetty.py
+python3 scripts/start-petstore-jetty.py --port 10100
+```
+
+Stop the server with Ctrl+C.
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `--port` | N (integer, 1-65535) | 10000 | Port to listen on. Passed to the application as the `petstore.port` system property; the checked-in configuration file is not changed. Out-of-range values are rejected by the argument parser. |
+
+## What it does
+
+1. Parses `--port` (`parse_args`).
+2. Checks whether something already listens on `localhost:<port>` (`port_in_use`). If so, prints the port and, when `lsof` can find it, the holder process, and exits 2 before building or launching anything (`port_busy_message`).
+3. Finds the project root and requires `juneau-petstore/juneau-petstore-jetty` under it.
+4. If the module has no `target/classes`, runs `mvn clean compile` there.
+5. Resolves the runtime classpath with `mvn -q dependency:build-classpath`, caching it in `target/petstore-classpath.txt` and refreshing the cache when the module's `pom.xml` is newer (`resolve_classpath`).
+6. Runs `java -Dpetstore.port=<port> -cp <classes>:<classpath> org.apache.juneau.petstore.jetty.App` in the module directory, using `$JAVA_HOME/bin/java` when `JAVA_HOME` is set (`launch_command`).
+
+## Output and exit codes
+
+Colored status lines, then the application's output. The banner shows `http://localhost:<port>`.
+
+| Code | Meaning |
+|---|---|
+| 0 | Stopped with Ctrl+C. |
+| 1 | The project root or module directory was not found, the build or classpath step failed, or the launch raised an error. |
+| 2 | The port is already in use. |
+| other | The exit code of the `java` process when it ends on its own. |
+
+## Gotchas
+
+- The port is strict. A busy port is never skipped or shared, so a request can never be answered by a stale server.
+- The script uses plain `java` rather than `exec:java` because the Maven exec class loader cannot load some petstore classes.
+- The classes build only runs when `target/classes` is missing. Stale classes are not rebuilt.
+- `lsof` is optional; without it the busy-port message just suggests the command to run.
+
+## See also
+
+- `start-petstore-springboot.py`
+- Docs site: Developer Info, Script: start-petstore-jetty (`04.02.12.ScriptStartPetstoreJetty.md` in juneau-docs)
diff --git a/scripts/start-petstore-jetty.py b/scripts/start-petstore-jetty.py
index 995548f4f0..6e39fb24ab 100755
--- a/scripts/start-petstore-jetty.py
+++ b/scripts/start-petstore-jetty.py
@@ -13,23 +13,9 @@
# ***************************************************************************************************************************
"""
-Start Juneau Petstore (Jetty) Application
+Starts the Juneau petstore sample application on the embedded Jetty microservice, with plain `java` and a cached Maven classpath.
-This script starts the Juneau petstore sample application using the embedded Jetty/Microservice server.
-It runs the org.apache.juneau.petstore.jetty.App main class with plain java. The classpath is resolved once with
-Maven's dependency:build-classpath (cached in target/petstore-classpath.txt); exec:java is not used because its
-class loader fails to load some of the petstore's classes (NoClassDefFoundError on ConfigResource$SectionNotFound).
-
-The port is strict: if it is already bound the script exits non-zero before building or launching anything, naming
-the port and, where it can be found, the holder. A busy port is never skipped past or silently shared, so a request
-to the port can never be answered by some other (possibly stale) server.
-
-Usage:
- python3 scripts/start-petstore-jetty.py [--port N]
-
-Options:
- --port N Port to listen on (default 10000). Passed to Jetty as the petstore.port system property,
- which juneau-petstore-jetty.cfg reads; the checked-in cfg is not modified.
+Full docs: start-petstore-jetty.md
"""
import argparse
diff --git a/scripts/start-petstore-springboot.md b/scripts/start-petstore-springboot.md
new file mode 100644
index 0000000000..70c261ba71
--- /dev/null
+++ b/scripts/start-petstore-springboot.md
@@ -0,0 +1,48 @@
+# start-petstore-springboot.py
+
+Starts the Juneau petstore sample application on Spring Boot, through the Spring Boot Maven plugin.
+
+## When to use
+
+- You want the Spring Boot flavor of the petstore running locally on its default port, 5000.
+- For the Jetty flavor, use `start-petstore-jetty.py`.
+
+## Usage
+
+```bash
+python3 scripts/start-petstore-springboot.py
+```
+
+Stop the server with Ctrl+C.
+
+## Options
+
+None.
+
+## What it does
+
+1. Finds the project root by walking up from `scripts/` to the first directory with a `pom.xml` (`find_project_root`).
+2. Requires `juneau-petstore/juneau-petstore-springboot` under it (`main`).
+3. If that module has no `target/classes`, runs `mvn clean compile` there first.
+4. Runs `mvn spring-boot:run` in the module with its output attached to your terminal. The main class is `org.apache.juneau.petstore.springboot.App`.
+
+## Output and exit codes
+
+Colored status lines, then the Maven and Spring Boot output. The banner shows `http://localhost:5000` as the default address.
+
+| Code | Meaning |
+|---|---|
+| 0 | Stopped with Ctrl+C. |
+| 1 | The project root or module directory was not found, the build failed, or the launch raised an error. |
+| other | The exit code of `mvn spring-boot:run` when it ends on its own. |
+
+## Gotchas
+
+- Unlike the Jetty script, there is no `--port` option and no check that the port is free. The port comes from the application's own configuration.
+- The build step runs only when `target/classes` is missing. Stale classes are not rebuilt.
+- It needs `mvn` on the `PATH`.
+
+## See also
+
+- `start-petstore-jetty.py`
+- Docs site: Developer Info, Script: start-petstore-springboot (`04.02.13.ScriptStartPetstoreSpringboot.md` in juneau-docs)
diff --git a/scripts/start-petstore-springboot.py b/scripts/start-petstore-springboot.py
index 90b4ea38bb..8ffddd1e58 100755
--- a/scripts/start-petstore-springboot.py
+++ b/scripts/start-petstore-springboot.py
@@ -13,13 +13,9 @@
# ***************************************************************************************************************************
"""
-Start Juneau Petstore (Spring Boot) Application
+Starts the Juneau petstore sample application on Spring Boot, through the Spring Boot Maven plugin.
-This script starts the Juneau petstore sample application using Spring Boot.
-It runs the org.apache.juneau.petstore.springboot.App main class via the Spring Boot Maven plugin.
-
-Usage:
- python3 scripts/start-petstore-springboot.py
+Full docs: start-petstore-springboot.md
"""
import sys
diff --git a/scripts/surefire-rollup.md b/scripts/surefire-rollup.md
new file mode 100644
index 0000000000..59a1b18241
--- /dev/null
+++ b/scripts/surefire-rollup.md
@@ -0,0 +1,54 @@
+# surefire-rollup.py
+
+Totals the test counts and run times from a module's Surefire XML reports into core and container buckets.
+
+## When to use
+
+- You want to see how a module's test time splits between plain tests and tests that boot a Spring Boot or Jetty container, without reading the XML by hand.
+
+## Usage
+
+```bash
+python3 scripts/surefire-rollup.py
+python3 scripts/surefire-rollup.py juneau-integration-tests
+```
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `module` | path (optional positional) | `juneau-integration-tests` | Module directory, relative to the repository root, whose `target/surefire-reports` is read. |
+
+## What it does
+
+1. Resolves `<repo>/<module>/target/surefire-reports` (`main`).
+2. If a `core/` or `container/` subdirectory exists there, sums the `tests` and `time` attributes of each `TEST-*.xml` in them (`parse_xml_dir`) and splits the container reports into Spring Boot and Jetty buckets (`parse_container_subbuckets`).
+3. Otherwise treats the reports as one flat directory and assigns each report to a bucket (`parse_flat_reports`). A report is a container test when its source file mentions `SpringbootTest` or `JettyMicroserviceTest`, or when the class name contains `springboot`, `jetty` or `microservice`.
+4. Prints one row per bucket with a bar, the test count and seconds.
+
+## Output and exit codes
+
+```
+juneau-integration-tests/
+ core : ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ (1200 tests, 41.200s)
+ container.springboot: ...
+ container.jetty : ...
+ container : ...
+```
+
+Bars are scaled to the largest bucket, up to 20 blocks. The `container` row is the sum of the two container rows.
+
+| Code | Meaning |
+|---|---|
+| 0 | The report was printed, including when no reports exist (all rows are zero). |
+
+## Gotchas
+
+- Container test sources are looked up under `src/test/java` of the module, derived from the class name. A missing source falls back to the class-name hints.
+- In the split `container/` layout, a report that matches neither framework is counted under `container.jetty`.
+- A malformed report XML raises an uncaught parse error.
+
+## See also
+
+- `test.py`
+- `push-timings.py`
diff --git a/scripts/surefire-rollup.py b/scripts/surefire-rollup.py
index 3e8b738d91..44c36efec8 100644
--- a/scripts/surefire-rollup.py
+++ b/scripts/surefire-rollup.py
@@ -16,7 +16,9 @@
# limitations under the License.
#
"""
-Roll up Surefire XML results into timing buckets.
+Totals the test counts and run times from a module's Surefire XML reports into core and container buckets.
+
+Full docs: surefire-rollup.md
"""
from __future__ import annotations
diff --git a/scripts/test.md b/scripts/test.md
new file mode 100644
index 0000000000..7765deca91
--- /dev/null
+++ b/scripts/test.md
@@ -0,0 +1,83 @@
+# test.py
+
+Builds Juneau, runs the Maven tests, then checks timings, runs the scripts' own pytest suite, and optionally the browser tests.
+
+## When to use
+
+- You want the standard local verification: a clean build plus the full test run.
+- You want only the build (`--build-only`) or only the tests (`--test-only`).
+- You want to profile one module's tests (`--profile`) or refresh the timing baselines (`--rebaseline`).
+- `push.py` runs this script for its build and test steps.
+
+## Usage
+
+```bash
+python3 scripts/test.py
+python3 scripts/test.py --test-only --no-container
+python3 scripts/test.py --build-only
+python3 scripts/test.py --enforce-perf
+python3 scripts/test.py --profile juneau-core/juneau-marshall
+python3 scripts/test.py --rebaseline --window 20 --min-runs 5
+```
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `--build-only`, `-b` | none | off | Only build; skip the tests. |
+| `--test-only`, `-t` | none | off | Only run the tests; no build. Cannot be combined with `--build-only`. |
+| `--full`, `-f` | none | on when no other mode is given | Clean build, then the tests. |
+| `--verbose`, `-v` | none | off | Accepted for compatibility. Maven output always streams live. |
+| `--no-container` | none | off | Exclude tests tagged `container` (`-DexcludedGroups=container`). |
+| `--timing-log` | path | none | Ignored, with a warning. Timings are in the run history; the option goes away in a later release. |
+| `--enforce-perf` | none | off | Fail on a perf-guard breach instead of only warning. |
+| `--rebaseline` | none | off | Rewrite the `suite` line and `[modules]` section of `perf-baseline.txt` from the run history, then exit without building. |
+| `--window` | N (integer) | 20 | With `--rebaseline`: runs per key whose median is taken. A module whose window spreads by more than 50% takes the 75th percentile instead. |
+| `--min-runs` | N (integer) | 5 | With `--rebaseline`: skip keys with fewer runs. |
+| `--js-tests` | none | auto | Also run the headless-browser tests (`mvn -Pjs-tests`). Fails if Node or npm are missing. |
+| `--no-js-tests` | none | off | Never run the browser tests. Cannot be combined with `--js-tests`. |
+| `--console` | `full`, `condensed` or `none` | `full` | How Maven output appears on the console. |
+| `--detail` | `summary`, `actionable`, `modules` or `all` | `actionable` | Console detail level. |
+| `--full-log` | path | none | Also append everything Maven prints to this file. |
+| `--condensed-log` | path | none | Also append the condensed one-line-per-event stream to this file. |
+| `--profile` | module path | none | Run one module's tests under a one-shot JFR profile, then exit. |
+| `--help`, `-h` | none | off | Print the usage text and exit 0. |
+
+## What it does
+
+1. Parses the options. Unknown options print the usage text and exit 1 (`_main`).
+2. Build: `mvn clean install -T1C -DskipTests` (`build`).
+3. Tests: `mvn test -T1C -Drat.skip=true`, plus `-DexcludedGroups=container` with `--no-container` (`test`). Per-module Surefire times and Maven's reactor times are recorded in the run history (`_record_buckets`).
+4. Perf guard: compares the suite wall clock with the `suite` line of `perf-baseline.txt`, and each module's reactor time with its `[modules]` entry. The report is written to `target/perf-guard.txt` (`run_perf_guard`).
+5. Python tests: runs `scripts/tests` with `python3 -m pytest`, or `uv run --native-tls --with pytest pytest` when pytest is not importable (`script_tests`).
+6. Browser tests: with `-Pjs-tests` in the views, petstore-core and terminal modules (`maybe_run_js_tests`). Without `--js-tests` or `--no-js-tests`, they run automatically when a `.js`, `.cjs`, `.css` or `.ftl` file under a `src/` tree differs from `origin/master`.
+7. `--profile` runs `mvn test -pl <module> -Drat.skip=true` with a JFR `argLine` and writes `target/profile-results/<module>-<timestamp>.jfr` (`profile`).
+
+## Output and exit codes
+
+Maven output, a status line per step, and the perf-guard report. The run header and markers come from the shared run toolkit. A standalone run rings the terminal bell at the end; `JUNEAU_PUSH_NO_SOUND=1` silences it.
+
+| Code | Meaning |
+|---|---|
+| 0 | Every requested step passed. |
+| 1 | Conflicting or unknown options, a failed build or test run, or `--rebaseline` with history turned off. |
+| 2 | An invalid `--detail` value. |
+| 130 | Cancelled with Ctrl-C or SIGTERM. |
+| other | The exit code of the failing Maven, perf-guard or pytest step. |
+
+## Gotchas
+
+- The perf guard only warns unless `--enforce-perf` is given. A suite breaches above baseline times `1 + JUNEAU_CI_PERF_THRESHOLD` (default 0.20). A module breaches only when it is over that and more than `JUNEAU_CI_PERF_MIN_SECONDS` (default 3) over its baseline. A module with no baseline entry warns and never fails.
+- `JUNEAU_MVN_WRAPPER` adds a prefix to every `mvn` command, for example a lock script that serializes runs.
+- The pytest step is skipped with a notice when neither pytest nor `uv` is available. Nothing is installed.
+- The browser tests download Playwright and Chromium on the first run, so they need network access.
+- `JUNEAU_RUN_HISTORY=0` turns off the run history, which `--rebaseline` and `push-timings.py` depend on.
+- `--profile` replaces `argLine`, which drops the JaCoCo agent so it does not skew the profile.
+- The default mode runs `mvn clean install`, which replaces the Juneau artifacts in `~/.m2`.
+
+## See also
+
+- `push.py`, which runs this script
+- `push-timings.py`, which reads the run history
+- `perf-required.py`
+- `coverage.py`
diff --git a/scripts/test.py b/scripts/test.py
index 9822792cc7..d91bf38642 100755
--- a/scripts/test.py
+++ b/scripts/test.py
@@ -12,6 +12,12 @@
# * specific language governing permissions and limitations under the License.
# ***************************************************************************************************************************
"""
+Builds Juneau, runs the Maven tests, then checks timings, runs the scripts' own pytest suite, and optionally the browser tests.
+
+Full docs: test.md
+"""
+
+USAGE = """
Build and test helper script for Apache Juneau.
Usage:
@@ -76,6 +82,7 @@ import argparse
import math
import os
import re
+import shlex
import shutil
import statistics
import subprocess
@@ -645,10 +652,12 @@ def script_tests(launcher=None, runner=None):
"""The scripts/tests pytest suite as its own row. Returns 0 on pass or skip, non-zero on failure."""
if os.environ.get(SCRIPT_TESTS_ACTIVE):
say("\n⚠️ Skipping Python Tests: already running inside them (no recursion).", "warn")
+ skip_py_tests_row("already running inside them")
return 0
launcher = launcher or script_tests_launcher()
if launcher is None:
say("\n⚠️ Skipping Python Tests: neither pytest nor uv found (python3 -m pytest / uv run --native-tls --with pytest).", "warn")
+ skip_py_tests_row("neither pytest nor uv found")
return 0
cmd = script_tests_command(launcher)
say("\n🐍 Running Python Tests (scripts/tests)...")
@@ -723,9 +732,59 @@ def js_tests_command():
f"-Dtest='{patterns}' -Dsurefire.failIfNoSpecifiedTests=false")
-def js_tests(installed=False): # NOSONAR python:S1172 -- kept so callers need not change; the scope no longer depends on it
- """The JS harness: the views and petstore browser suites in one run."""
- return run_command(js_tests_command(), "js-tests", "Browser Tests", label="Browser Tests")
+# Forks per module while the modules overlap; a lone petstore run takes the pom's default of 3.
+JS_CONCURRENT_FORKS = 2
+
+
+def js_tests_concurrent_command():
+ """The same suites as js_tests_command, as one mvn per module so they overlap (one reactor runs them in dependency
+ order). Each `mvn -f` resolves its upstream modules from ~/.m2 (hence -o), so this is valid only straight after the
+ install step. Output streams live with a [module] prefix; the shell waits for every module and fails if any did,
+ and a module that left no exit status counts as a failure. Needs bash (PIPESTATUS)."""
+ runs = []
+ for i, (module, pattern) in enumerate(JS_TEST_MODULES):
+ runs.append(f"(mvn -o -Pjs-tests -f {module}/pom.xml test -Drat.skip=true -Dtest='{pattern}' "
+ f"-Dsurefire.failIfNoSpecifiedTests=false -Djuneau.jsTests.forks={JS_CONCURRENT_FORKS} 2>&1 "
+ f"| sed -u 's|^|[{module}] |' | tee target/js-tests-{i}.log; echo ${{PIPESTATUS[0]}} > target/js-tests-{i}.exit) &")
+ codes = " ".join(f"$(cat target/js-tests-{i}.exit 2>/dev/null || echo 1)" for i in range(len(JS_TEST_MODULES)))
+ return ("mkdir -p target && rm -f target/js-tests-*.exit && " + " ".join(runs)
+ + f" wait ; for c in {codes}; do [ \"$c\" = 0 ] || exit 1; done")
+
+
+def js_tests(installed=False):
+ """The JS harness: the views, petstore and terminal browser suites. Straight after the install step the three
+ modules run at once (js_tests_concurrent_command); otherwise one sequential reactor run (js_tests_command), since
+ the upstream modules are then built from the working tree rather than read from ~/.m2."""
+ if not installed:
+ return run_command(js_tests_command(), "js-tests", "Browser Tests", label="Browser Tests")
+ cmd = f"/bin/bash -c {shlex.quote(js_tests_concurrent_command())}"
+ # Wraps the whole compound: run_command only prefixes commands that start with "mvn ".
+ wrapper = os.environ.get("JUNEAU_MVN_WRAPPER", "").strip()
+ if wrapper:
+ cmd = f"{wrapper} {cmd}"
+ return run_command(cmd, "js-tests", "Browser Tests", label="Browser Tests")
+
+
+#: test.py's top-level steps in run order, as (id, label). The labels are push.py's PLAN_LABELS for the same ids:
+#: under push.py this script's pending rows stand in for the parent's.
+PLAN = (("build", "Compile"), ("tests", "Java Tests"), ("perf", "Perf"), ("py-tests", "Python Tests"),
+ ("js-tests", "Browser Tests"))
+
+
+def planned_steps(build_only, test_only, full, no_js_tests):
+ """The steps this run will take, in order, as (id, label) pairs, for the console's pending rows and Overall
+ figure. --no-js-tests leaves Browser Tests out; a run-time skip (no JS changes) stays in the plan, and its skip
+ row is left out of the count."""
+ ids = {"build"} if build_only or full else set()
+ if test_only or full:
+ ids |= {"tests", "perf", "py-tests"} | (set() if no_js_tests else {"js-tests"})
+ return [(step_id, label) for step_id, label in PLAN if step_id in ids]
+
+
+def skip_py_tests_row(reason):
+ """Draw the Python Tests row as skipped, with the reason as its text."""
+ with _juneau.load()[0].row("py-tests", "Python Tests", label="Python Tests") as row:
+ row.skip(reason)
def skip_js_tests_row(reason):
@@ -793,11 +852,11 @@ def _main(): # NOSONAR python:S3776 -- Cognitive complexity is acceptable for t
parser.add_argument("--help", "-h", action="store_true")
args, unknown = parser.parse_known_args()
if args.help:
- print(__doc__)
+ print(USAGE)
return 0
if unknown:
print(f"Unknown option(s): {' '.join(unknown)}")
- print(__doc__)
+ print(USAGE)
return 1
if args.timing_log:
say("⚠ --timing-log is ignored: timings are in the run history now (scripts/push-timings.py reads it). "
@@ -829,6 +888,7 @@ def _main(): # NOSONAR python:S3776 -- Cognitive complexity is acceptable for t
if args.js_tests and args.no_js_tests:
print("Cannot combine --js-tests and --no-js-tests")
return 1
+ _juneau.load()[0].plan(planned_steps(build_only, test_only, full, args.no_js_tests))
exit_code = 0
last_test_output = ""
diff --git a/scripts/tests/README.md b/scripts/tests/README.md
index 0da78314fc..8e1dc280c3 100644
--- a/scripts/tests/README.md
+++ b/scripts/tests/README.md
@@ -11,7 +11,7 @@ remains here is a juneau-only script with no counterpart elsewhere.
## Running
-The suite must also pass under a push-like environment (`JUNEAU_RUN_SESSION`, `JUNEAU_PUSH_ACTIVE`, `JUNEAU_RUN_ACTIVE`, `JUNEAU_RUN_N_BASE` set), because `push.py`'s Python Tests step runs it that way; any test that depends on those variables being unset must clear them itself.
+The suite must also pass under a push-like environment (`JUNEAU_RUN_SESSION`, `JUNEAU_PUSH_ACTIVE`, `JUNEAU_RUN_ACTIVE`, `JUNEAU_RUN_N_BASE` set), because `push.py`'s Python Tests step runs it that way. `conftest.py` strips every live-run `JUNEAU_RUN_*` variable (`LIVE_RUN_ENV`: logs, cancel, header, markers, result, session and pending/overall rows) before each test, so a test never writes into the real run's logs (`test_env_isolation.py`); a test that needs one sets it itself.
`pytest` is not vendored anywhere in this repo (it's pure-stdlib elsewhere in `scripts/` on
purpose). Two easy ways to run this suite without installing anything globally:
@@ -86,3 +86,18 @@ with `argv[0]` set so the writer's script name is `test`; `fake_reports` (F2) bu
repositories for the same reason that one is: the guard's entire risk is real `git status
--porcelain` behavior, which a mocked subprocess would let pass while proving nothing. Never
runs a real `mvn` command.
+
+- `test_script_docs.py` -- the script-docs checker. Every runnable script under `scripts/` (a `.py`
+ file with a `__main__` guard, a `.sh` file, or an extensionless file with a shell or Python
+ shebang) needs a `<name>.md` beside it with the seven standard sections (When to use, Usage,
+ Options, What it does, Output and exit codes, Gotchas, See also), and every flag the script
+ declares must appear in backticks in the doc's Options section. It reads source with `ast` and
+ text matching and never runs a script. Flags come from literal `add_argument` and `add_parser`
+ calls, or from the option `case` block of a shell script; an argparse script with no literal
+ calls fails as `FLAGS-UNVERIFIABLE` unless excluded. A doc listing a flag the script does not
+ declare only warns. Exceptions go in `scripts/script-docs-exclude.txt`, one per line as
+ `pattern # reason` (reason required), with a `flags:` prefix to skip only the flag check and a
+ `packages:` prefix for library directories that need a `README.md` instead; a pattern that
+ matches nothing fails. Tests, vendor, node_modules, `__pycache__` and fixtures directories are
+ never scanned. The checker also unit-tests its own parsers and proves each check fails when
+ broken.
diff --git a/scripts/tests/conftest.py b/scripts/tests/conftest.py
index 3908520092..accf2a1277 100644
--- a/scripts/tests/conftest.py
+++ b/scripts/tests/conftest.py
@@ -60,11 +60,22 @@ def _full_console(monkeypatch):
monkeypatch.setenv("JUNEAU_RUN_CONSOLE", "full")
+# Every JUNEAU_RUN_* variable that points a session at a live parent run (its logs, cancel command, markers, result,
+# header and pending/overall rows, or the fact that one is active). JUNEAU_RUN_CONSOLE, _DETAIL and _HISTORY* are
+# settings, not pointers, and are handled by their own fixtures.
+LIVE_RUN_ENV = (
+ "JUNEAU_RUN_ACTIVE", "JUNEAU_RUN_SESSION", "JUNEAU_RUN_LIVE", "JUNEAU_RUN_N_BASE", "JUNEAU_RUN_LABEL_WIDTH",
+ "JUNEAU_RUN_FULL_LOG", "JUNEAU_RUN_CONDENSED_LOG", "JUNEAU_RUN_CANCEL", "JUNEAU_RUN_HEADER",
+ "JUNEAU_RUN_PENDING", "JUNEAU_RUN_OVERALL", "JUNEAU_RUN_HANDBACK",
+ "JUNEAU_RUN_MARKERS_FILE", "JUNEAU_RUN_RESULT", "JUNEAU_RUN_RESULT_PARTS",
+)
+
+
@pytest.fixture(autouse=True)
def _no_outer_run_files(monkeypatch):
- """A run under JUNEAU_RUN_MARKERS_FILE / JUNEAU_RUN_RESULT exports them to this suite; drop them so the tests see
- the same run state as under a plain run (and don't write the outer run's files)."""
- for name in ("JUNEAU_RUN_MARKERS_FILE", "JUNEAU_RUN_RESULT", "JUNEAU_RUN_RESULT_PARTS"):
+ """A run under watch-run.sh or push.py exports its live-run variables (see LIVE_RUN_ENV) to this suite; drop them so
+ the tests see the same run state as under a plain run (and don't write the outer run's logs or files)."""
+ for name in LIVE_RUN_ENV:
monkeypatch.delenv(name, raising=False)
diff --git a/scripts/tests/test_create_mvn_site_cli.py b/scripts/tests/test_create_mvn_site_cli.py
new file mode 100644
index 0000000000..edb280d178
--- /dev/null
+++ b/scripts/tests/test_create_mvn_site_cli.py
@@ -0,0 +1,84 @@
+# ***************************************************************************************************************************
+# * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file *
+# * distributed with this work for additional information regarding copyright ownership. The ASF licenses this file *
+# * to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance *
+# * with the License. You may obtain a copy of the License at *
+# * *
+# * http://www.apache.org/licenses/LICENSE-2.0 *
+# * *
+# * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an *
+# * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the *
+# * specific language governing permissions and limitations under the License. *
+"""
+Tests for the command line of scripts/create-mvn-site.py: --help and -h print usage and exit 0, and an unknown flag
+exits 2, all before anything is removed or any Maven process is started. Nothing here runs a real mvn.
+"""
+
+from __future__ import annotations
+
+import importlib.util
+import subprocess
+from pathlib import Path
+
+import pytest
+
+SCRIPTS_DIR = Path(__file__).resolve().parent.parent
+
+
+@pytest.fixture
+def module():
+ spec = importlib.util.spec_from_file_location("_undertest_create_mvn_site", SCRIPTS_DIR / "create-mvn-site.py")
+ m = importlib.util.module_from_spec(spec)
+ spec.loader.exec_module(m)
+ return m
+
+
+@pytest.fixture
+def forbidden(module, monkeypatch):
+ """Make any process spawn or directory removal fail the test."""
+
+ def boom(*a, **k):
+ raise AssertionError("nothing may be run or removed while parsing arguments")
+
+ monkeypatch.setattr(module.subprocess, "run", boom)
+ monkeypatch.setattr(module.subprocess, "Popen", boom)
+ monkeypatch.setattr(module.shutil, "rmtree", boom)
+ monkeypatch.setattr(module.shutil, "copytree", boom)
+
+
+@pytest.mark.parametrize("flag", ["--help", "-h"])
+def test_help_exits_zero_and_touches_nothing(module, forbidden, flag, capsys):
+ with pytest.raises(SystemExit) as exc:
+ module.main([flag])
+ assert exc.value.code == 0
+ assert "usage" in capsys.readouterr().out.lower()
+
+
+def test_unknown_flag_exits_two_and_touches_nothing(module, forbidden, capsys):
+ with pytest.raises(SystemExit) as exc:
+ module.main(["--bogus"])
+ assert exc.value.code == 2
+ assert "usage" in capsys.readouterr().err.lower()
+
+
+def test_dry_run_exits_zero_and_touches_nothing(module, forbidden, capsys):
+ assert module.main(["--dry-run"]) == 0
+ assert "mvn clean compile site" in capsys.readouterr().out
+
+
+def test_no_args_still_runs_the_build_sequence(module, tmp_path, monkeypatch):
+ calls = []
+ removed = []
+ site = tmp_path / "target" / "site"
+ site.mkdir(parents=True)
+ (site / "index.html").write_text("x")
+
+ monkeypatch.setattr(module.subprocess, "run", lambda *a, **k: subprocess.CompletedProcess(a, 0, stdout="1.0\n"))
+ monkeypatch.setattr(module, "run_maven_command", lambda command, description, cwd: calls.append(command) or True)
+ monkeypatch.setattr(module.shutil, "rmtree", lambda p: removed.append(Path(p)))
+ monkeypatch.setattr(module, "__file__", str(tmp_path / "scripts" / "create-mvn-site.py"))
+
+ assert module.main([]) == 0
+ assert calls == [["mvn", "clean", "compile", "site"]]
+ assert site in removed
+ assert (tmp_path / "docs" / "static" / "site" / "index.html").exists()
diff --git a/scripts/tests/test_env_isolation.py b/scripts/tests/test_env_isolation.py
new file mode 100644
index 0000000000..627cf481dd
--- /dev/null
+++ b/scripts/tests/test_env_isolation.py
@@ -0,0 +1,43 @@
+# ***************************************************************************************************************************
+# * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an
+# * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+# * specific language governing permissions and limitations under the License.
+# ***************************************************************************************************************************
+"""A live run's JUNEAU_RUN_* variables must not reach the tests, or their sessions write into the real run's logs."""
+
+import os
+import subprocess
+import sys
+from pathlib import Path
+
+from conftest import LIVE_RUN_ENV
+
+_CHECKED = "JUNEAU_TEST_ENV_ISOLATION_INNER"
+
+
+def test_inner_sees_no_live_run_variables():
+ """Runs only in the subprocess below: the suite's fixtures must have dropped every live-run variable."""
+ if not os.environ.get(_CHECKED):
+ return
+ assert [n for n in LIVE_RUN_ENV if n in os.environ] == []
+
+
+def test_live_run_variables_do_not_reach_tests(tmp_path):
+ env = dict(os.environ)
+ env[_CHECKED] = "1"
+ live = tmp_path / "live"
+ for name in LIVE_RUN_ENV:
+ env[name] = str(live / (name + ".txt"))
+ result = subprocess.run(
+ [sys.executable, "-m", "pytest", "-q", "-p", "no:cacheprovider",
+ f"{Path(__file__)}::test_inner_sees_no_live_run_variables"],
+ env=env, capture_output=True, text=True, cwd=Path(__file__).parent)
+ assert result.returncode == 0, result.stdout + result.stderr
+ assert not live.exists()
diff --git a/scripts/tests/test_js_tests_flags.py b/scripts/tests/test_js_tests_flags.py
index 022d7e2924..617f6c7ff4 100644
--- a/scripts/tests/test_js_tests_flags.py
+++ b/scripts/tests/test_js_tests_flags.py
@@ -21,6 +21,7 @@ from __future__ import annotations
import argparse
import importlib.util
+import shlex
import sys
from pathlib import Path
@@ -141,7 +142,7 @@ def test_test_py_rejects_conflicting_flags(test_mod, monkeypatch, capsys):
def test_test_py_help_documents_flags(test_mod):
- assert "--js-tests" in test_mod.__doc__ and "--no-js-tests" in test_mod.__doc__
+ assert "--js-tests" in test_mod.USAGE and "--no-js-tests" in test_mod.USAGE
def test_test_py_accepts_flags(test_mod, monkeypatch):
@@ -151,8 +152,9 @@ def test_test_py_accepts_flags(test_mod, monkeypatch):
assert test_mod.main() == 0
-def test_push_py_doc_documents_flags(push_mod):
- assert "--js-tests" in push_mod.__doc__ and "--no-js-tests" in push_mod.__doc__
+def test_push_py_doc_documents_flags():
+ doc = (SCRIPTS_DIR / "push.md").read_text()
+ assert "`--js-tests`" in doc and "`--no-js-tests`" in doc
def test_push_build_test_command_forwards_flags(push_mod):
@@ -230,3 +232,40 @@ def test_push_always_plans_the_browser_tests_row(push_mod, js_tests):
args = argparse.Namespace(test_only=True, sonarqube=False, tracker_audit=False, skip_tests=False, js_tests=js_tests)
ids = [step_id for step_id, _ in push_mod.push_plan(args, SCRIPTS_DIR, SCRIPTS_DIR.parent, False)]
assert ids.index("js-tests") == ids.index("py-tests") + 1
+
+
+# ---- concurrent module runs --------------------------------------------------------------------------------------------
+def test_concurrent_command_runs_one_f_process_per_module(test_mod):
+ cmd = test_mod.js_tests_concurrent_command()
+ for module, pattern in test_mod.JS_TEST_MODULES:
+ assert f"-f {module}/pom.xml" in cmd and f"-Dtest='{pattern}'" in cmd
+ assert f"[{module}] " in cmd # live output is labelled per module
+ assert cmd.count("mvn -o -Pjs-tests") == len(test_mod.JS_TEST_MODULES) # offline: upstream modules come from ~/.m2
+ assert "-Djuneau.jsTests.forks=2" in cmd
+ assert " wait " in cmd
+
+
+def test_concurrent_command_fails_when_a_module_leaves_no_exit_status(test_mod):
+ cmd = test_mod.js_tests_concurrent_command()
+ # A killed subshell writes no status file; reading none must count as a failure, not a pass.
+ assert cmd.count("|| echo 1") == len(test_mod.JS_TEST_MODULES)
+ assert cmd.rstrip().endswith("do [ \"$c\" = 0 ] || exit 1; done")
+
+
+def test_js_tests_uses_the_reactor_command_unless_just_installed(test_mod, monkeypatch):
+ monkeypatch.delenv("JUNEAU_MVN_WRAPPER", raising=False)
+ seen = []
+ monkeypatch.setattr(test_mod, "run_command", lambda cmd, *a, **k: seen.append(cmd) or (0, ""))
+ test_mod.js_tests(installed=False)
+ test_mod.js_tests(installed=True)
+ assert seen[0] == test_mod.js_tests_command()
+ # The concurrent form needs bash (PIPESTATUS), so it always runs under an explicit bash -c.
+ assert seen[1] == f"/bin/bash -c {shlex.quote(test_mod.js_tests_concurrent_command())}"
+
+
+def test_concurrent_form_keeps_the_mvn_wrapper_around_the_whole_compound(test_mod, monkeypatch):
+ monkeypatch.setenv("JUNEAU_MVN_WRAPPER", "/x/lock.sh")
+ seen = []
+ monkeypatch.setattr(test_mod, "run_command", lambda cmd, *a, **k: seen.append(cmd) or (0, ""))
+ test_mod.js_tests(installed=True)
+ assert seen[0] == f"/x/lock.sh /bin/bash -c {shlex.quote(test_mod.js_tests_concurrent_command())}"
diff --git a/scripts/tests/test_push.py b/scripts/tests/test_push.py
index ba7f769983..4f10dfcab2 100644
--- a/scripts/tests/test_push.py
+++ b/scripts/tests/test_push.py
@@ -1300,6 +1300,24 @@ def flow(push_module, repo, monkeypatch):
return go
+def test_the_session_header_has_no_message_and_the_message_is_a_line(push_module):
+ assert push_module.session_header("Juneau", "master") == ["Juneau", "master"]
+ assert push_module.session_lines("Fix the cap") == ["Commit Message: Fix the cap"]
+ assert push_module.session_lines(None) == []
+ assert push_module.session_lines("") == []
+
+
+@pytest.mark.parametrize("argv,title,lines", [
+ (["Fix the cap"], "Push", ["Commit Message: Fix the cap"]),
+ (["--test-only"], "Test", []),
+])
+def test_the_session_gets_the_message_as_a_header_line(flow, push_module, monkeypatch, argv, title, lines):
+ seen = []
+ monkeypatch.setattr(push_module, "_in_session", lambda t, body, **kw: seen.append((t, kw)) or 0)
+ assert flow(*argv) == 0
+ assert [(t, kw["header"], kw["lines"]) for t, kw in seen] == [(title, ["Juneau", "master"], lines)]
+
+
def _push_from_elsewhere(tmp_path: Path, remote_repo: Path) -> None:
other = tmp_path / "other-clone"
_run(tmp_path, "git", "clone", str(remote_repo), str(other))
diff --git a/scripts/tests/test_release_run.py b/scripts/tests/test_release_run.py
index acddbec2e5..f99c1d96f2 100644
--- a/scripts/tests/test_release_run.py
+++ b/scripts/tests/test_release_run.py
@@ -114,7 +114,7 @@ class FakeRun:
def warn(self, text):
self._record("warn", text)
- def session(self, title, header=(), *, detail=None, sinks=None, repo="."):
+ def session(self, title, header=(), *, lines=(), detail=None, sinks=None, repo="."):
self._record("session", title, list(header))
self.console = True
diff --git a/scripts/tests/test_revert_staged.py b/scripts/tests/test_revert_staged.py
new file mode 100644
index 0000000000..5da18acb0a
--- /dev/null
+++ b/scripts/tests/test_revert_staged.py
@@ -0,0 +1,108 @@
+# ***************************************************************************************************************************
+# * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an
+# * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+# * specific language governing permissions and limitations under the License.
+# ***************************************************************************************************************************
+"""
+revert-staged.py: restores one file from HEAD. Every test runs the script in a throwaway git repo under tmp_path, never in a
+real checkout.
+"""
+
+from __future__ import annotations
+
+import subprocess
+import sys
+from pathlib import Path
+
+import pytest
+
+SCRIPT = Path(__file__).resolve().parent.parent / "revert-staged.py"
+
+
+def git(repo, *args):
+ return subprocess.run(["git", *args], cwd=repo, check=True, capture_output=True, text=True).stdout
+
+
+def run(repo, *args):
+ return subprocess.run([sys.executable, str(SCRIPT), *args], cwd=repo, capture_output=True, text=True)
+
+
+@pytest.fixture
+def repo(tmp_path):
+ git(tmp_path, "init", "-q")
+ git(tmp_path, "config", "user.email", "t@example.com")
+ git(tmp_path, "config", "user.name", "t")
+ (tmp_path / "f.txt").write_text("committed\n")
+ (tmp_path / "-dash.txt").write_text("committed\n")
+ git(tmp_path, "add", "--", "f.txt", "-dash.txt")
+ git(tmp_path, "commit", "-q", "-m", "init")
+ return tmp_path
+
+
+def test_restores_working_tree_from_head(repo):
+ f = repo / "f.txt"
+ f.write_text("edited\n")
+ r = run(repo, "f.txt")
+ assert r.returncode == 0, r.stdout
+ assert f.read_text() == "committed\n"
+
+
+def test_dash_leading_path_after_double_dash(repo):
+ f = repo / "-dash.txt"
+ f.write_text("edited\n")
+ r = run(repo, "--", "-dash.txt")
+ assert r.returncode == 0, r.stdout + r.stderr
+ assert f.read_text() == "committed\n"
+
+
+def test_path_named_like_a_ref(repo):
+ git(repo, "branch", "f.txt")
+ f = repo / "f.txt"
+ f.write_text("edited\n")
+ r = run(repo, "f.txt")
+ assert r.returncode == 0, r.stdout + r.stderr
+ assert f.read_text() == "committed\n"
+
+
+def test_untracked_file_refused(repo):
+ u = repo / "u.txt"
+ u.write_text("keep me\n")
+ r = run(repo, "u.txt")
+ assert r.returncode == 1
+ assert "not tracked" in r.stdout
+ assert u.read_text() == "keep me\n"
+
+
+def test_no_arguments_is_usage_error(repo):
+ r = run(repo)
+ assert r.returncode == 2
+ assert "usage" in r.stderr.lower()
+
+
+def test_help_makes_no_git_call(monkeypatch, capsys):
+ import importlib.util
+ spec = importlib.util.spec_from_file_location("revert_staged", SCRIPT)
+ mod = importlib.util.module_from_spec(spec)
+ spec.loader.exec_module(mod)
+
+ def boom(*a, **k):
+ raise AssertionError("subprocess.run called")
+
+ monkeypatch.setattr(subprocess, "run", boom)
+ with pytest.raises(SystemExit) as e:
+ mod.main(["--help"])
+ assert e.value.code == 0
+ assert "usage" in capsys.readouterr().out.lower()
+
+
+def test_help_exit_code_via_cli(repo):
+ r = run(repo, "-h")
+ assert r.returncode == 0
+ assert "usage" in r.stdout.lower()
diff --git a/scripts/tests/test_revert_unstaged.py b/scripts/tests/test_revert_unstaged.py
new file mode 100644
index 0000000000..13f40a9657
--- /dev/null
+++ b/scripts/tests/test_revert_unstaged.py
@@ -0,0 +1,112 @@
+# ***************************************************************************************************************************
+# * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an
+# * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+# * specific language governing permissions and limitations under the License.
+# ***************************************************************************************************************************
+"""
+revert-unstaged.py: restores one file's working-tree edits to the staged version and keeps what is staged. Every test runs
+the script in a throwaway git repo under tmp_path, never in a real checkout.
+"""
+
+from __future__ import annotations
+
+import subprocess
+import sys
+from pathlib import Path
+
+import pytest
+
+SCRIPT = Path(__file__).resolve().parent.parent / "revert-unstaged.py"
+
+
+def git(repo, *args):
+ return subprocess.run(["git", *args], cwd=repo, check=True, capture_output=True, text=True).stdout
+
+
+def run(repo, *args):
+ return subprocess.run([sys.executable, str(SCRIPT), *args], cwd=repo, capture_output=True, text=True)
+
+
+@pytest.fixture
+def repo(tmp_path):
+ git(tmp_path, "init", "-q")
+ git(tmp_path, "config", "user.email", "t@example.com")
+ git(tmp_path, "config", "user.name", "t")
+ (tmp_path / "f.txt").write_text("committed\n")
+ git(tmp_path, "add", "f.txt")
+ git(tmp_path, "commit", "-q", "-m", "init")
+ return tmp_path
+
+
+def test_restores_staged_version_and_keeps_index(repo):
+ f = repo / "f.txt"
+ f.write_text("A\n")
+ git(repo, "add", "f.txt")
+ f.write_text("B\n")
+ r = run(repo, "f.txt")
+ assert r.returncode == 0, r.stdout
+ assert f.read_text() == "A\n"
+ assert git(repo, "diff", "--cached", "--stat").strip() != ""
+ assert git(repo, "show", ":f.txt") == "A\n"
+ assert git(repo, "diff").strip() == ""
+
+
+def test_nothing_to_revert(repo):
+ r = run(repo, "f.txt")
+ assert r.returncode == 0
+ assert "nothing to revert" in r.stdout.lower()
+ assert (repo / "f.txt").read_text() == "committed\n"
+
+
+def test_untracked_file_refused(repo):
+ u = repo / "u.txt"
+ u.write_text("keep me\n")
+ r = run(repo, "u.txt")
+ assert r.returncode == 1
+ assert "not tracked" in r.stdout
+ assert u.read_text() == "keep me\n"
+
+
+def test_nonexistent_path(repo):
+ r = run(repo, "missing.txt")
+ assert r.returncode == 1
+ assert "not found" in r.stdout.lower()
+
+
+def test_dash_leading_path_after_double_dash(repo):
+ f = repo / "-dash.txt"
+ f.write_text("A\n")
+ git(repo, "add", "--", "-dash.txt")
+ f.write_text("B\n")
+ r = run(repo, "--", "-dash.txt")
+ assert r.returncode == 0, r.stdout + r.stderr
+ assert f.read_text() == "A\n"
+
+
+def test_help_makes_no_git_call(monkeypatch, capsys):
+ import importlib.util
+ spec = importlib.util.spec_from_file_location("revert_unstaged", SCRIPT)
+ mod = importlib.util.module_from_spec(spec)
+ spec.loader.exec_module(mod)
+
+ def boom(*a, **k):
+ raise AssertionError("subprocess.run called")
+
+ monkeypatch.setattr(subprocess, "run", boom)
+ with pytest.raises(SystemExit) as e:
+ mod.main(["--help"])
+ assert e.value.code == 0
+ assert "usage" in capsys.readouterr().out.lower()
+
+
+def test_help_exit_code_via_cli(repo):
+ r = run(repo, "-h")
+ assert r.returncode == 0
+ assert "usage" in r.stdout.lower()
diff --git a/scripts/tests/test_run_adoption.py b/scripts/tests/test_run_adoption.py
index 3ed5c251b2..449498ec50 100644
--- a/scripts/tests/test_run_adoption.py
+++ b/scripts/tests/test_run_adoption.py
@@ -667,7 +667,8 @@ def test_push_identity_is_a_row_not_a_header_mark(push_mod, gates, monkeypatch,
assert push_mod.main() == 0
[session] = events.of("session")
assert session["title"] == "Push"
- assert session["header"] == ["Juneau", "main", "Quick fix"]
+ assert session["header"] == ["Juneau", "main"]
+ assert session["lines"] == ["Commit Message: Quick fix"]
assert not [line for line in session["header"] if "✓" in line]
assert [f["id"] for f in events.of("step_start")][:1] == ["prechecks"]
out = capsys.readouterr().out
@@ -783,7 +784,8 @@ def test_docs_only_under_a_console_has_one_session_and_one_done(push_mod, docs_t
assert push_mod.main() == 0
[session] = events.of("session")
assert session["title"] == "Push"
- assert session["header"] == ["Juneau Docs", "main", "Docs fix"]
+ assert session["header"] == ["Juneau Docs", "main"]
+ assert session["lines"] == ["Commit Message: Docs fix"]
assert [f["status"] for f in events.of("done")] == ["ok"]
assert [p["ids"] for p in events.of("plan")] == [["prechecks", "docs-smoke", "commit", "push"]]
diff --git a/scripts/tests/test_script_docs.py b/scripts/tests/test_script_docs.py
new file mode 100644
index 0000000000..83981a94ad
--- /dev/null
+++ b/scripts/tests/test_script_docs.py
@@ -0,0 +1,598 @@
+# ***************************************************************************************************************************
+# * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an
+# * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
+# * specific language governing permissions and limitations under the License.
+# ***************************************************************************************************************************
+"""
+Script docs checker: every runnable script under scripts/ has a <name>.md beside it, with the seven standard sections,
+and every flag the script declares is named in that doc's Options section.
+
+Source copy: this file in apache/juneau (scripts/tests/test_script_docs.py), version 1. Other repositories carry the
+same file; the only intended difference is SKIP_DIRS. Change the shared logic everywhere, not in one copy.
+
+Read-only and hermetic: it parses source with ast and text matching, never runs or imports a script, and writes nothing.
+Exceptions live in scripts/script-docs-exclude.txt (format in scripts/tests/README.md).
+"""
+
+from __future__ import annotations
+
+import ast
+import fnmatch
+import re
+import warnings
+from pathlib import Path
+
+import pytest
+
+# scripts/ is resolved from this file, so the result does not depend on the current directory.
+SCRIPTS_DIR = Path(__file__).resolve().parents[1]
+EXCLUDE_FILE = "script-docs-exclude.txt"
+
+# Directory names never scanned for entry points or docs.
+SKIP_DIRS = ("tests", "vendor", "node_modules", "__pycache__", "fixtures")
+
+REQUIRED_HEADINGS = ["When to use", "Usage", "Options", "What it does", "Output and exit codes", "Gotchas", "See also"]
+
+SCRIPT_EXTENSIONS = (".py", ".sh")
+SHEBANG = re.compile(r"^#!.*\b(sh|bash|zsh|python[0-9.]*)\b")
+DOC_TITLE = re.compile(r"^# (?P<name>[\w.\-]+\.(?:py|sh|bash|zsh))\s*$")
+HELP_FLAGS = ("-h", "--help")
+
+# Option variables a shell `case` may switch on (as $1, $opt and so on); any other case subject is not an option parser.
+SHELL_OPTION_SUBJECTS = ("1", "opt", "arg", "a", "option", "flag", "key", "OPT", "ARG")
+SHELL_CASE = re.compile(r"^\s*case\s+\"?\$\{?(?P<subject>\w+)\}?\"?\s+in\b")
+SHELL_ESAC = re.compile(r"^\s*esac\b")
+SHELL_PATTERN = re.compile(r"^\s*(?P<alts>-[^)\s]*(?:\s*\|\s*-[^)\s]*)*)\)")
+SHELL_FLAG = re.compile(r"^(--?[A-Za-z0-9][\w-]*)")
+USAGE_FLAG = re.compile(r"(?<![\w-])(--?[A-Za-z][\w-]*)")
+
+
+# --------------------------------------------------------------------------------------------------------------------
+# Discovery
+# --------------------------------------------------------------------------------------------------------------------
+
+def _skipped(relative: Path) -> bool:
+ """True for a path inside a skipped or hidden directory."""
+ return any(part in SKIP_DIRS or part.startswith(".") for part in relative.parts[:-1])
+
+
+def _read(path: Path) -> str:
+ return path.read_text(encoding="utf-8", errors="replace")
+
+
+def has_main_guard(source: str) -> bool:
+ """True when the module has a top-level `if __name__ == "__main__":` (either operand order, either quote)."""
+ try:
+ tree = ast.parse(source)
+ except SyntaxError:
+ return re.search(r"^if\s+__name__\s*==\s*['\"]__main__['\"]", source, re.MULTILINE) is not None
+ for node in tree.body:
+ if not isinstance(node, ast.If) or not isinstance(node.test, ast.Compare):
+ continue
+ test = node.test
+ if len(test.ops) != 1 or not isinstance(test.ops[0], ast.Eq):
+ continue
+ sides = [test.left, *test.comparators]
+ names = [s for s in sides if isinstance(s, ast.Name) and s.id == "__name__"]
+ consts = [s for s in sides if isinstance(s, ast.Constant) and s.value == "__main__"]
+ if names and consts:
+ return True
+ return False
+
+
+def is_entry_point(path: Path) -> bool:
+ """A .py file with a __main__ guard, a .sh file, or an extensionless file with a shell or Python shebang."""
+ if path.suffix == ".py":
+ return has_main_guard(_read(path))
+ if path.suffix == ".sh":
+ return True
+ if path.suffix == "":
+ try:
+ first_line = _read(path).splitlines()[0]
+ except (IndexError, OSError):
+ return False
+ return SHEBANG.match(first_line) is not None
+ return False
+
+
+def find_entry_points(scripts_dir: Path) -> list[Path]:
+ """Every entry point under scripts_dir, as sorted absolute paths."""
+ found = []
+ for path in sorted(scripts_dir.rglob("*")):
+ if path.is_file() and not _skipped(path.relative_to(scripts_dir)) and is_entry_point(path):
+ found.append(path)
+ return found
+
+
+def doc_path(script: Path) -> Path:
+ """The doc beside a script: <stem>.md in the same directory."""
+ return script.with_name(script.stem + ".md")
+
+
+def find_script_docs(scripts_dir: Path) -> list[Path]:
+ """Every .md under scripts_dir outside the skipped directories."""
+ return [p for p in sorted(scripts_dir.rglob("*.md")) if not _skipped(p.relative_to(scripts_dir))]
+
+
+# --------------------------------------------------------------------------------------------------------------------
+# Exclude file
+# --------------------------------------------------------------------------------------------------------------------
+
+class Excludes:
+ """The parsed exclude file: whole-file excludes, flag-check excludes, package directories and format errors."""
+
+ def __init__(self):
+ self.skip = [] # (pattern, reason)
+ self.flags = [] # (pattern, reason)
+ self.packages = [] # (directory, reason)
+ self.errors = [] # human-readable format problems
+
+
+def parse_excludes(text: str) -> Excludes:
+ """Parse the exclude file text. A line is `[flags:|packages:] <pattern> # <reason>`; the reason is mandatory."""
+ result = Excludes()
+ for number, raw in enumerate(text.splitlines(), 1):
+ line = raw.strip()
+ if not line or line.startswith("#"):
+ continue
+ body, hash_mark, reason = line.partition(" #")
+ body, reason = body.strip(), reason.strip()
+ if not hash_mark or not reason:
+ result.errors.append(f"line {number}: no '# reason' after '{body}'")
+ continue
+ if body.startswith("flags:"):
+ result.flags.append((body[len("flags:"):].strip(), reason))
+ elif body.startswith("packages:"):
+ for name in body[len("packages:"):].split(","):
+ if name.strip():
+ result.packages.append((name.strip().rstrip("/"), reason))
+ else:
+ result.skip.append((body, reason))
+ return result
+
+
+def load_excludes(scripts_dir: Path) -> Excludes:
+ path = scripts_dir / EXCLUDE_FILE
+ return parse_excludes(_read(path)) if path.is_file() else Excludes()
+
+
+def _matches(relative: str, patterns: list[tuple[str, str]]) -> bool:
+ return any(fnmatch.fnmatchcase(relative, pattern) for pattern, _ in patterns)
+
+
+# --------------------------------------------------------------------------------------------------------------------
+# Flag extraction
+# --------------------------------------------------------------------------------------------------------------------
+
+class Declared:
+ """What a script declares: flags, positionals, sub-command names, and whether it is argparse-based at all."""
+
+ def __init__(self):
+ self.flags = []
+ self.positionals = []
+ self.subcommands = []
+ self.uses_argparse = False
+ self.add_calls = 0
+
+
+def python_declared(source: str) -> Declared:
+ """Flags from add_argument calls on any receiver (parser, group, sub-parser) and sub-commands from add_parser."""
+ declared = Declared()
+ try:
+ tree = ast.parse(source)
+ except SyntaxError:
+ return declared
+ for node in ast.walk(tree):
+ if isinstance(node, ast.Import):
+ declared.uses_argparse |= any(alias.name == "argparse" for alias in node.names)
+ elif isinstance(node, ast.ImportFrom):
+ declared.uses_argparse |= node.module == "argparse"
+ elif isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute):
+ strings = [a.value for a in node.args if isinstance(a, ast.Constant) and isinstance(a.value, str)]
+ if node.func.attr == "add_argument":
+ declared.add_calls += 1
+ for value in strings:
+ if value.startswith("-"):
+ if value not in HELP_FLAGS and value not in declared.flags:
+ declared.flags.append(value)
+ elif value not in declared.positionals:
+ declared.positionals.append(value)
+ elif node.func.attr == "add_parser" and strings:
+ declared.add_calls += 1
+ if strings[0] not in declared.subcommands:
+ declared.subcommands.append(strings[0])
+ return declared
+
+
+def shell_flags(source: str) -> list[str]:
+ """
+ Flags of a shell script. Option patterns (`--long|-s)`) inside a `case` over an option variable; when there are
+ none, the flags named on `Usage:` lines of the leading comment block.
+ """
+ flags = []
+ in_option_case = False
+ for line in source.splitlines():
+ case = SHELL_CASE.match(line)
+ if case:
+ in_option_case = case.group("subject") in SHELL_OPTION_SUBJECTS
+ continue
+ if SHELL_ESAC.match(line):
+ in_option_case = False
+ continue
+ if in_option_case:
+ pattern = SHELL_PATTERN.match(line)
+ if pattern:
+ for alt in pattern.group("alts").split("|"):
+ match = SHELL_FLAG.match(alt.strip())
+ if match and match.group(1) not in HELP_FLAGS and match.group(1) not in flags:
+ flags.append(match.group(1))
+ if flags:
+ return flags
+ for line in source.splitlines():
+ if line.strip() and not line.startswith("#"):
+ break
+ if line.startswith("#") and "Usage:" in line:
+ for token in USAGE_FLAG.findall(line.split("Usage:", 1)[1]):
+ if token not in HELP_FLAGS and token not in flags:
+ flags.append(token)
+ return flags
+
+
+# --------------------------------------------------------------------------------------------------------------------
+# Doc parsing
+# --------------------------------------------------------------------------------------------------------------------
+
+def h2_sections(text: str) -> list[tuple[str, str]]:
+ """(heading, body) for each `## ` heading outside code fences, in order."""
+ sections = []
+ fenced = False
+ for line in text.splitlines():
+ if line.lstrip().startswith("```"):
+ fenced = not fenced
+ if not fenced and line.startswith("## "):
+ sections.append((line[3:].strip(), []))
+ elif sections:
+ sections[-1][1].append(line)
+ return [(heading, "\n".join(body)) for heading, body in sections]
+
+
+def documented_tokens(options_text: str) -> set[str]:
+ """Every word inside a backtick span in the Options section, split on whitespace and commas."""
+ tokens = set()
+ for span in re.findall(r"`([^`]+)`", options_text):
+ for word in re.split(r"[\s,|/]+", span):
+ if word:
+ tokens.add(word)
+ tokens.add(word.split("=", 1)[0])
+ return tokens
+
+
+def options_section(text: str) -> str | None:
+ for heading, body in h2_sections(text):
+ if heading == "Options":
+ return body
+ return None
+
+
+# --------------------------------------------------------------------------------------------------------------------
+# The checks. Each returns a list of failure strings (empty means pass).
+# --------------------------------------------------------------------------------------------------------------------
+
+def rel(scripts_dir: Path, path: Path) -> str:
+ return path.relative_to(scripts_dir).as_posix()
+
+
+def check_missing_docs(scripts_dir: Path, excludes: Excludes) -> list[str]:
+ """Check 1: an entry point has no <name>.md beside it."""
+ failures = []
+ for script in find_entry_points(scripts_dir):
+ if _matches(rel(scripts_dir, script), excludes.skip):
+ continue
+ if not doc_path(script).is_file():
+ failures.append(f"{rel(scripts_dir, script)}: no doc; create {rel(scripts_dir, doc_path(script))}")
+ return failures
+
+
+def check_orphan_docs(scripts_dir: Path) -> list[str]:
+ """Check 2: a script doc (first line `# <file name>.py`) whose script is missing or named differently."""
+ failures = []
+ for doc in find_script_docs(scripts_dir):
+ lines = _read(doc).splitlines()
+ title = DOC_TITLE.match(lines[0]) if lines else None
+ if not title:
+ continue
+ name = title.group("name")
+ if not (doc.parent / name).is_file():
+ failures.append(f"{rel(scripts_dir, doc)}: its title names {name}, but there is no such script beside it")
+ elif Path(name).stem != doc.stem:
+ failures.append(f"{rel(scripts_dir, doc)}: named for {doc.stem} but its title names {name}")
+ return failures
+
+
+def check_headings(scripts_dir: Path, excludes: Excludes) -> list[str]:
+ """Check 3: every script doc has exactly the seven H2 sections, in order."""
+ failures = []
+ docs = [doc_path(s) for s in find_entry_points(scripts_dir)]
+ docs += [d for d in find_script_docs(scripts_dir) if DOC_TITLE.match((_read(d).splitlines() or [""])[0])]
+ for doc in sorted(set(docs)):
+ if not doc.is_file():
+ continue
+ text = _read(doc)
+ if not text.startswith("# "):
+ failures.append(f"{rel(scripts_dir, doc)}: first line must be the H1 '# <script file name>'")
+ headings = [heading for heading, _ in h2_sections(text)]
+ if headings != REQUIRED_HEADINGS:
+ missing = [h for h in REQUIRED_HEADINGS if h not in headings]
+ detail = f"missing {missing}" if missing else f"found {headings}, expected this order"
+ failures.append(f"{rel(scripts_dir, doc)}: headings are not the required seven ({detail})")
+ return failures
+
+
+def declared_for(script: Path) -> Declared | None:
+ """Declared flags for a script, or None when its flags cannot be determined without running it."""
+ source = _read(script)
+ if script.suffix == ".py":
+ declared = python_declared(source)
+ if declared.uses_argparse and declared.add_calls == 0:
+ return None
+ return declared
+ declared = Declared()
+ declared.flags = shell_flags(source)
+ return declared
+
+
+def check_flags(scripts_dir: Path, excludes: Excludes) -> tuple[list[str], list[str]]:
+ """Check 4: every declared flag is in backticks in the Options section. Returns (failures, reverse warnings)."""
+ failures, warns = [], []
+ for script in find_entry_points(scripts_dir):
+ name = rel(scripts_dir, script)
+ if _matches(name, excludes.skip) or _matches(name, excludes.flags):
+ continue
+ doc = doc_path(script)
+ if not doc.is_file():
+ continue # check 1 reports it
+ declared = declared_for(script)
+ if declared is None:
+ failures.append(f"{name}: FLAGS-UNVERIFIABLE (argparse without literal add_argument calls); document it by "
+ f"hand and add 'flags: {name} # <reason>' to {EXCLUDE_FILE}")
+ continue
+ options = options_section(_read(doc))
+ if options is None:
+ continue # check 3 reports it
+ tokens = documented_tokens(options)
+ for flag in [*declared.flags, *declared.subcommands]:
+ if flag not in tokens:
+ failures.append(f"{name}: flag {flag} is not in backticks in the Options section of {doc.name}")
+ for positional in declared.positionals:
+ if positional not in tokens:
+ warns.append(f"{name}: positional {positional} is not in backticks in the Options section of {doc.name}")
+ if declared.flags or declared.subcommands:
+ known = {*declared.flags, *declared.subcommands, *HELP_FLAGS}
+ for token in sorted(t for t in tokens if t.startswith("-") and t not in known):
+ warns.append(f"{name}: the Options section of {doc.name} lists {token}, which the script does not declare")
+ return failures, warns
+
+
+def check_excludes(scripts_dir: Path, excludes: Excludes) -> list[str]:
+ """The exclude file: every line has a reason, and no pattern is stale."""
+ failures = list(excludes.errors)
+ entries = [rel(scripts_dir, s) for s in find_entry_points(scripts_dir)]
+ for kind, patterns in (("exclude", excludes.skip), ("flags:", excludes.flags)):
+ for pattern, _ in patterns:
+ if not any(fnmatch.fnmatchcase(entry, pattern) for entry in entries):
+ failures.append(f"stale {kind} pattern '{pattern}' matches no entry point")
+ return failures
+
+
+def check_packages(scripts_dir: Path, excludes: Excludes) -> list[str]:
+ """Each directory listed with `packages:` exists, holds a .py module, and has a README.md (H1 plus a paragraph)."""
+ failures = []
+ for name, _ in excludes.packages:
+ directory = scripts_dir / name
+ if not directory.is_dir():
+ failures.append(f"package {name}: no such directory")
+ elif not list(directory.glob("*.py")):
+ failures.append(f"package {name}: no .py module in it")
+ elif not (directory / "README.md").is_file():
+ failures.append(f"package {name}: no README.md")
+ else:
+ lines = [ln for ln in _read(directory / "README.md").splitlines() if ln.strip()]
+ if not lines or not lines[0].startswith("# ") or len(lines) < 2 or lines[1].startswith("#"):
+ failures.append(f"package {name}: README.md needs an H1 followed by at least one paragraph")
+ return failures
+
+
+def _fail(failures: list[str]) -> None:
+ if failures:
+ pytest.fail("\n" + "\n".join(f" {f}" for f in failures), pytrace=False)
+
+
+# --------------------------------------------------------------------------------------------------------------------
+# The checks against this repository
+# --------------------------------------------------------------------------------------------------------------------
+
+def test_every_entry_point_has_a_doc():
+ _fail(check_missing_docs(SCRIPTS_DIR, load_excludes(SCRIPTS_DIR)))
+
+
+def test_every_script_doc_has_a_script():
+ _fail(check_orphan_docs(SCRIPTS_DIR))
+
+
+def test_every_script_doc_has_the_seven_headings():
+ _fail(check_headings(SCRIPTS_DIR, load_excludes(SCRIPTS_DIR)))
+
+
+def test_every_declared_flag_is_documented():
+ failures, warns = check_flags(SCRIPTS_DIR, load_excludes(SCRIPTS_DIR))
+ for message in warns:
+ warnings.warn(message, stacklevel=1)
+ _fail(failures)
+
+
+def test_exclude_file_is_valid_and_not_stale():
+ _fail(check_excludes(SCRIPTS_DIR, load_excludes(SCRIPTS_DIR)))
+
+
+def test_listed_packages_have_a_readme():
+ _fail(check_packages(SCRIPTS_DIR, load_excludes(SCRIPTS_DIR)))
+
+
+# --------------------------------------------------------------------------------------------------------------------
+# Unit tests for the parsers and the checks, on inline fixtures
+# --------------------------------------------------------------------------------------------------------------------
+
+GOOD_DOC = "# tool.py\n\nSummary.\n\n" + "".join(f"## {h}\n\nNone.\n\n" for h in REQUIRED_HEADINGS)
+GOOD_DOC = GOOD_DOC.replace("## Options\n\nNone.", "## Options\n\n| Option |\n|---|\n| `--alpha`, `-a` |\n| `--beta` |")
+
+ARGPARSE_SCRIPT = '''
+import argparse
+p = argparse.ArgumentParser()
+p.add_argument("--alpha", "-a")
+g = p.add_mutually_exclusive_group()
+g.add_argument("--beta", action="store_true")
+sub = p.add_subparsers()
+run = sub.add_parser("run")
+run.add_argument("--gamma")
+p.add_argument("target")
+if __name__ == "__main__":
+ pass
+'''
+
+
+def _scripts(tmp_path: Path, files: dict[str, str]) -> Path:
+ scripts = tmp_path / "scripts"
+ for name, content in files.items():
+ target = scripts / name
+ target.parent.mkdir(parents=True, exist_ok=True)
+ target.write_text(content, encoding="utf-8")
+ return scripts
+
+
+def test_python_flags_with_aliases_groups_and_subparsers():
+ declared = python_declared(ARGPARSE_SCRIPT)
+ assert declared.flags == ["--alpha", "-a", "--beta", "--gamma"]
+ assert declared.subcommands == ["run"]
+ assert declared.positionals == ["target"]
+ assert declared.uses_argparse
+
+
+def test_python_help_flags_are_ignored():
+ assert python_declared('import argparse\np.add_argument("--help", "-h")').flags == []
+
+
+def test_python_dynamic_flags_are_unverifiable():
+ source = "import argparse\nfor flag in FLAGS:\n p.add_argument(flag)\n"
+ declared = python_declared(source)
+ assert declared.uses_argparse and declared.flags == []
+ assert declared.add_calls == 1 and python_declared("import argparse\nparser = None\n").add_calls == 0
+
+
+def test_main_guard_detection():
+ assert has_main_guard('if __name__ == "__main__":\n pass\n')
+ assert has_main_guard("if '__main__' == __name__:\n pass\n")
+ assert not has_main_guard("def main():\n if __name__ == '__main__':\n pass\n")
+ assert not has_main_guard("def helper():\n pass\n")
+
+
+def test_shell_flags_from_a_case_loop():
+ source = '#!/bin/bash\nwhile [ $# -gt 0 ]; do\n case "$1" in\n --out|-o) OUT=$2; shift;;\n' \
+ ' --verbose) V=1;;\n -h|--help) usage;;\n *) break;;\n esac\ndone\n'
+ assert shell_flags(source) == ["--out", "-o", "--verbose"]
+
+
+def test_shell_case_over_something_else_is_not_an_option_parser():
+ source = '#!/bin/bash\ncase "$(uname)" in\n -x) echo x;;\nesac\ncase "$OSTYPE" in\n --odd) :;;\nesac\n'
+ assert shell_flags(source) == []
+
+
+def test_shell_usage_comment_fallback():
+ assert shell_flags("#!/bin/sh\n# Usage: tool.sh [--dry-run] [-n NAME]\nrun\n") == ["--dry-run", "-n"]
+ assert shell_flags("#!/bin/sh\nrun\n# Usage: tool.sh --late\n") == []
+
+
+def test_entry_points_cover_py_sh_and_shebang_files(tmp_path):
+ scripts = _scripts(tmp_path, {
+ "a.py": "if __name__ == '__main__':\n pass\n", "lib.py": "def f():\n pass\n", "b.sh": "echo hi\n",
+ "c": "#!/usr/bin/env bash\necho hi\n", "d": "plain text\n", "tests/e.py": "if __name__ == '__main__':\n pass\n",
+ "vendor/f.sh": "echo\n", "fixtures/g.sh": "echo\n", "sub/h.py": "if __name__ == '__main__':\n pass\n"})
+ assert [rel(scripts, p) for p in find_entry_points(scripts)] == ["a.py", "b.sh", "c", "sub/h.py"]
+
+
+def test_exclude_parser_prefixes_and_reasons():
+ parsed = parse_excludes("# c\n\nfoo/*.sh # fixture\nflags: bar.py # dynamic\npackages: a, b # libs\nbad.py\nalso.py #\n")
+ assert parsed.skip == [("foo/*.sh", "fixture")]
+ assert parsed.flags == [("bar.py", "dynamic")]
+ assert parsed.packages == [("a", "libs"), ("b", "libs")]
+ assert len(parsed.errors) == 2
+
+
+# Each check is broken once, to prove it fails for the thing it guards.
+
+def test_check1_fails_without_a_doc(tmp_path):
+ scripts = _scripts(tmp_path, {"tool.py": "if __name__ == '__main__':\n pass\n"})
+ assert check_missing_docs(scripts, Excludes()) != []
+ assert check_missing_docs(scripts, parse_excludes("tool.py # test\n")) == []
+
+
+def test_check2_fails_for_a_doc_without_a_script(tmp_path):
+ scripts = _scripts(tmp_path, {"tool.md": GOOD_DOC, "README.md": "# Notes\n\nProse.\n"})
+ assert len(check_orphan_docs(scripts)) == 1
+ (scripts / "tool.py").write_text("x = 1\n")
+ assert check_orphan_docs(scripts) == []
+
+
+def test_check3_fails_for_a_missing_heading(tmp_path):
+ scripts = _scripts(tmp_path, {"tool.py": "if __name__ == '__main__':\n pass\n", "tool.md": GOOD_DOC})
+ assert check_headings(scripts, Excludes()) == []
+ (scripts / "tool.md").write_text(GOOD_DOC.replace("## Gotchas\n\nNone.\n\n", ""))
+ assert len(check_headings(scripts, Excludes())) == 1
+
+
+def test_check3_ignores_headings_inside_code_fences(tmp_path):
+ fenced = GOOD_DOC.replace("## Usage\n\nNone.", "## Usage\n\n```\n## Not a heading\n```")
+ scripts = _scripts(tmp_path, {"tool.py": "if __name__ == '__main__':\n pass\n", "tool.md": fenced})
+ assert check_headings(scripts, Excludes()) == []
+
+
+def test_check4_fails_for_an_undocumented_flag(tmp_path):
+ scripts = _scripts(tmp_path, {"tool.py": ARGPARSE_SCRIPT, "tool.md": GOOD_DOC})
+ failures, _ = check_flags(scripts, Excludes())
+ assert any("--gamma" in f for f in failures) and any("run" in f for f in failures)
+ assert not any("--alpha" in f or "-a" in f.split(":")[-1] for f in failures)
+
+
+def test_check4_unverifiable_flags_need_an_exclude(tmp_path):
+ scripts = _scripts(tmp_path, {"tool.py": "import argparse\nif __name__ == '__main__':\n pass\n", "tool.md": GOOD_DOC})
+ failures, _ = check_flags(scripts, Excludes())
+ assert "FLAGS-UNVERIFIABLE" in failures[0]
+ assert check_flags(scripts, parse_excludes("flags: tool.py # built by a helper\n"))[0] == []
+
+
+def test_check4_reverse_check_only_warns(tmp_path):
+ source = 'import argparse\np.add_argument("--alpha")\nif __name__ == "__main__":\n pass\n'
+ scripts = _scripts(tmp_path, {"tool.py": source, "tool.md": GOOD_DOC})
+ failures, warns = check_flags(scripts, Excludes())
+ assert failures == [] and any("--beta" in w for w in warns)
+
+
+def test_stale_exclude_patterns_fail(tmp_path):
+ scripts = _scripts(tmp_path, {"tool.py": "if __name__ == '__main__':\n pass\n"})
+ assert check_excludes(scripts, parse_excludes("gone.py # was removed\n")) != []
+ assert check_excludes(scripts, parse_excludes("tool.py # kept\nflags: tool.py # kept\n")) == []
+ assert check_excludes(scripts, parse_excludes("tool.py\n")) != []
+
+
+def test_packages_need_a_readme(tmp_path):
+ scripts = _scripts(tmp_path, {"lib/mod.py": "x = 1\n"})
+ excludes = parse_excludes("packages: lib # library\n")
+ assert check_packages(scripts, excludes) != []
+ (scripts / "lib" / "README.md").write_text("# lib\n\nWhat it holds.\n")
+ assert check_packages(scripts, excludes) == []
diff --git a/scripts/tests/test_slack_notify_cli.py b/scripts/tests/test_slack_notify_cli.py
new file mode 100644
index 0000000000..664ff7a9ff
--- /dev/null
+++ b/scripts/tests/test_slack_notify_cli.py
@@ -0,0 +1,45 @@
+# ***************************************************************************************************************************
+# * Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file *
+# * distributed with this work for additional information regarding copyright ownership. The ASF licenses this file *
+# * to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance *
+# * with the License. You may obtain a copy of the License at *
+# * *
+# * http://www.apache.org/licenses/LICENSE-2.0 *
+# * *
+# * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an *
+# * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the *
+# * specific language governing permissions and limitations under the License. *
+"""
+Tests for the command line of scripts/slack-notify.py: --help prints usage and exits 0 before the config is read or
+any network call is made. Nothing here touches the network.
+"""
+
+from __future__ import annotations
+
+import importlib.util
+from pathlib import Path
+
+import pytest
+
+SCRIPTS_DIR = Path(__file__).resolve().parent.parent
+
+
+@pytest.fixture
+def module():
+ spec = importlib.util.spec_from_file_location("_undertest_slack_notify", SCRIPTS_DIR / "slack-notify.py")
+ m = importlib.util.module_from_spec(spec)
+ spec.loader.exec_module(m)
+ return m
+
+
+@pytest.mark.parametrize("flag", ["--help", "-h"])
+def test_help_exits_zero_without_config_or_network(module, flag, monkeypatch, capsys):
+ def boom(*a, **k):
+ raise AssertionError("--help must not read the config or touch the network")
+
+ monkeypatch.setattr(module, "_load_webhook_url", boom)
+ monkeypatch.setattr(module.urllib.request, "urlopen", boom)
+ with pytest.raises(SystemExit) as exc:
+ module.main(["slack-notify.py", flag])
+ assert exc.value.code == 0
+ assert "usage" in capsys.readouterr().out.lower()
diff --git a/scripts/tests/test_test_py_console.py b/scripts/tests/test_test_py_console.py
index 173cde51a5..6b85697f5d 100644
--- a/scripts/tests/test_test_py_console.py
+++ b/scripts/tests/test_test_py_console.py
@@ -15,6 +15,7 @@
from __future__ import annotations
import importlib.util
+import json
import os
import signal
import sys
@@ -156,17 +157,21 @@ def test_js_tests_cover_the_views_and_petstore_modules(test_mod, monkeypatch):
assert modules["juneau-petstore/juneau-petstore-core"] == "org.apache.juneau.petstore.**.*_BrowserTest"
assert modules["juneau-rest/juneau-rest-server-terminal"] == "org.apache.juneau.rest.server.terminal.*_BrowserTest"
calls = []
+ monkeypatch.delenv("JUNEAU_MVN_WRAPPER", raising=False)
monkeypatch.setattr(test_mod, "run_command", lambda cmd, *a, **k: calls.append(cmd) or (0, ""))
- test_mod.js_tests(installed=True)
test_mod.js_tests(installed=False)
+ test_mod.js_tests(installed=True)
cmd = calls[0]
- assert calls[0] == calls[1]
+ # Not just installed: one reactor run (upstream modules built from the working tree by --also-make).
+ assert calls[0] != calls[1]
assert cmd.startswith("mvn -Pjs-tests -pl juneau-rest/juneau-rest-server-views,juneau-petstore/juneau-petstore-core,"
"juneau-rest/juneau-rest-server-terminal test ")
assert "-f " not in cmd
assert ("-Dtest='org.apache.juneau.rest.server.views.*_BrowserTest,"
"org.apache.juneau.petstore.**.*_BrowserTest,org.apache.juneau.rest.server.terminal.*_BrowserTest'") in cmd
assert "-Dsurefire.failIfNoSpecifiedTests=false" in cmd
+ # Just installed: one mvn -f per module, all at once.
+ assert all(f"-f {module}/pom.xml" in calls[1] for module in modules)
@pytest.mark.parametrize("argv,installed", [([], True), (["--test-only"], False)])
@@ -266,3 +271,72 @@ def test_test_py_plain_ctrl_c_is_a_cancel(test_mod, monkeypatch, capsys):
captured = capsys.readouterr()
assert "❌ Cancelled." in captured.out
assert "##run" not in captured.out
+
+
+# ---- the plan ----------------------------------------------------------------------------------------------------------
+def _stub_steps(test_mod, monkeypatch):
+ monkeypatch.setattr(test_mod, "build", lambda: (0, ""))
+ monkeypatch.setattr(test_mod, "test", lambda no_container=False: (0, ""))
+ monkeypatch.setattr(test_mod, "run_perf_guard", lambda *a, **k: 0)
+ monkeypatch.setattr(test_mod, "script_tests", lambda *a, **k: 0)
+ monkeypatch.setattr(test_mod, "changed_files_vs_origin", lambda: [])
+ monkeypatch.setattr(test_mod, "maybe_run_js_tests", lambda *a, **k: 0)
+
+
+@pytest.mark.parametrize("argv,ids", [
+ ([], ["build", "tests", "perf", "py-tests", "js-tests"]),
+ (["--full"], ["build", "tests", "perf", "py-tests", "js-tests"]),
+ (["--build-only"], ["build"]),
+ (["--test-only"], ["tests", "perf", "py-tests", "js-tests"]),
+ (["--test-only", "--no-js-tests"], ["tests", "perf", "py-tests"]),
+ (["--no-js-tests"], ["build", "tests", "perf", "py-tests"]),
+ (["--js-tests"], ["build", "tests", "perf", "py-tests", "js-tests"]),
+])
+def test_main_declares_the_steps_its_own_flags_will_run(test_mod, bus, monkeypatch, argv, ids):
+ _stub_steps(test_mod, monkeypatch)
+ monkeypatch.setattr(sys, "argv", ["test.py", *argv])
+ assert test_mod._main() == 0
+ plans = bus.of("plan")
+ assert [p["ids"] for p in plans] == [ids]
+ assert set(plans[0]["labels"]) == set(ids) # every step is labelled, so each draws a pending row
+
+
+@pytest.mark.parametrize("argv", [["--build-only", "--test-only"], ["--js-tests", "--no-js-tests"], ["--profile", "m"]])
+def test_a_refused_combination_or_a_profile_run_declares_no_plan(test_mod, bus, monkeypatch, argv):
+ _stub_steps(test_mod, monkeypatch)
+ monkeypatch.setattr(test_mod, "profile", lambda module: (0, ""))
+ monkeypatch.setattr(sys, "argv", ["test.py", *argv])
+ test_mod._main()
+ assert bus.of("plan") == []
+
+
+def test_the_plan_labels_are_push_pys(test_mod):
+ """Under push.py the child's pending rows replace the parent's, so a label must read the same in both."""
+ spec = importlib.util.spec_from_file_location("_push_labels", SCRIPTS_DIR / "push.py")
+ push = importlib.util.module_from_spec(spec)
+ spec.loader.exec_module(push)
+ assert {i: label for i, label in test_mod.PLAN} == {i: push.PLAN_LABELS[i] for i, _ in test_mod.PLAN}
+
+
+def test_a_nested_run_under_a_parents_overall_has_an_overall_figure(test_mod, monkeypatch):
+ run = test_mod._juneau.load()[0]
+ monkeypatch.setenv("JUNEAU_RUN_OVERALL", json.dumps({"elapsed": 60, "done": 2, "pending": [
+ ["tests", 120], ["perf", None], ["py-tests", 20], ["js-tests", 40], ["install", 10]]}))
+ board = run.Board(nested=True)
+ feed = run.subscribe(lambda kind, fields: board.apply(kind, fields))
+ try:
+ _stub_steps(test_mod, monkeypatch)
+ monkeypatch.setattr(sys, "argv", ["test.py", "--test-only"])
+ assert board.overall() is None # no plan yet: no Overall (today's nested test.py)
+ assert test_mod._main() == 0
+ assert board.plan == ["tests", "perf", "py-tests", "js-tests"]
+ assert board.overall() is not None # the view's region now ends with ▶ Overall
+ finally:
+ run.unsubscribe(feed)
+
+
+def test_skipped_python_tests_still_get_their_row(test_mod, bus, monkeypatch):
+ monkeypatch.setenv(test_mod.SCRIPT_TESTS_ACTIVE, "1")
+ assert test_mod.script_tests() == 0
+ ends = [f for f in bus.of("step_end") if f["id"] == "py-tests"]
+ assert [e["status"] for e in ends] == ["skip"]
diff --git a/scripts/view-sonar-category.md b/scripts/view-sonar-category.md
new file mode 100644
index 0000000000..ef9c416e40
--- /dev/null
+++ b/scripts/view-sonar-category.md
@@ -0,0 +1,50 @@
+# view-sonar-category.py
+
+Legacy viewer for a locally saved, categorized SonarQube issue export; it does not talk to SonarCloud.
+
+## When to use
+
+- You still have the categorized JSON file from an older offline triage and want to browse one category.
+- For live findings, use `sonarqube.py` instead; it queries SonarCloud and supports rule and severity filters.
+
+## Usage
+
+```bash
+python3 scripts/view-sonar-category.py
+python3 scripts/view-sonar-category.py "<category-name>"
+python3 scripts/view-sonar-category.py "<category-name>" --limit 50
+```
+
+With no argument it lists the categories. Use a name exactly as the listing prints it.
+
+## Options
+
+| Option | Argument | Default | Meaning |
+|---|---|---|---|
+| `category-name` | text (optional positional) | none | The category to show. Omitted: print every category with its issue count, largest first. |
+| `--limit` | N (integer) | 20 | How many sample issues to print. Honored only as the second argument after the category name (`<category> --limit N`). |
+
+## What it does
+
+1. Loads `SonarQubeIssues.categorized.json` from a fixed path in one user's Downloads folder (`main`).
+2. With no category argument, prints the numbered category list and exits 0.
+3. Otherwise looks the category up; an unknown name prints the sorted list of names and exits 1.
+4. Prints the issue total, the number of affected files, the first N issues (description, file and location, id), and the 20 files with the most issues.
+
+## Output and exit codes
+
+All output goes to stdout.
+
+| Code | Meaning |
+|---|---|
+| 0 | The category list or a category report was printed. |
+| 1 | The JSON file is missing, or the category name is not in it. |
+
+## Gotchas
+
+- The JSON path is hard-coded to one user's Downloads folder, so the script does nothing useful elsewhere. The file was written by a categorizing script that is no longer in the repository.
+- `--limit` is read by position. `--limit 50` before the category name, or any other layout, is ignored and the default of 20 applies; a non-integer value raises an uncaught `ValueError`.
+
+## See also
+
+- `sonarqube.py`
diff --git a/scripts/view-sonar-category.py b/scripts/view-sonar-category.py
index a148576954..8f5d7b4f23 100755
--- a/scripts/view-sonar-category.py
+++ b/scripts/view-sonar-category.py
@@ -12,8 +12,9 @@
# * specific language governing permissions and limitations under the License. *
# ***************************************************************************************************************************
"""
-Helper script to view SonarQube issue categories without interactive input.
-Usage: view-sonar-category.py <category-name> [--limit N]
+Legacy viewer for a locally saved, categorized SonarQube issue export; it does not talk to SonarCloud.
+
+Full docs: view-sonar-category.md
"""
import json