You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by lh...@apache.org on 2026/09/20 20:04:51 UTC

(pulsar) 06/09: [improve][test] Retain post-measurement-start profile events

This is an automated email from the ASF dual-hosted git repository.

lhotari pushed a commit to branch lh-improve-perf-measurement-jfr
in repository https://gitbox.apache.org/repos/asf/pulsar.git

commit 3423e3bda6f03b90f4d7ec22c10042f318c08771
Author: Lari Hotari <lh...@apache.org>
AuthorDate: Sun Sep 20 20:42:25 2026 +0300

    [improve][test] Retain post-measurement-start profile events
---
 tests/performance/README.md                        | 15 ++++++-----
 tests/performance/iot-telemetry.md                 | 12 +++++----
 .../pulsar/tests/performance/launcher/JfrCut.java  | 31 +++++++++++++++++-----
 .../launcher/JfrRecordingProcessor.java            |  4 +--
 .../performance/launcher/PerformanceLauncher.java  |  3 +--
 .../tests/performance/launcher/JfrCutTest.java     |  8 +++---
 6 files changed, 49 insertions(+), 24 deletions(-)

diff --git a/tests/performance/README.md b/tests/performance/README.md
index 40888f4fde64..532c74b2d21e 100644
--- a/tests/performance/README.md
+++ b/tests/performance/README.md
@@ -84,8 +84,9 @@ The `iotTelemetry` workload can run traffic before measurements begin. Use `warm
 or use `warmupMessages` when `rate: 0`; the two settings are mutually exclusive. `warmupRounds` repeats that
 traffic, and `warmupRoundDelaySeconds` adds an idle stabilization period after each fully drained round, including
 the final round. The default is one round with no delay. Warmup traffic remains part of
-delivery and ordering validation. Producer throughput and the epoch-millisecond JFR measurement boundaries in
-`producer-summary.json` cover only the configured measurement messages.
+delivery and ordering validation. Producer throughput and the epoch-millisecond measurement boundaries in
+`producer-summary.json` cover only the configured measurement messages. The launcher uses the start boundary to
+remove warmup from the derived JFR while retaining everything recorded afterward.
 
 Use a top-level `extends` entry to inherit one file or an ordered list of files:
 
@@ -110,8 +111,9 @@ merge recursively, while scalar values and lists replace earlier values. An expl
 an inherited entry. Cycles, missing files, non-mapping roots and invalid `extends` entries are rejected.
 
 Profiled standalone runs retain the complete JFR and also create a sibling whose name ends in
-`.measurement.jfr`. The measurement recording contains events that overlap the producer's recorded measurement
-interval, excluding startup and warmup work. One-time JVM, host, recording setting and runtime configuration events
+`.measurement.jfr`. The measurement recording contains events from the producer's recorded measurement start through
+the end of each process recording, excluding startup and warmup work without dropping asynchronous completion,
+consumer validation, or shutdown activity after the producer finishes sending. One-time JVM, host, recording setting and runtime configuration events
 are copied from the beginning of the complete recording so JDK Mission Control can describe the source JVM. Set
 `profiling.retainOriginalRecording: false` to remove the complete
 recording after a successful cut, or `profiling.createMeasurementRecording: false` to keep only the complete
@@ -130,8 +132,9 @@ task requires JDK 19 or newer because it uses the public JFR recording writer ad
   --args='--input /tmp/full.jfr --from 5s --to 2m --output /tmp/measurement.jfr --info'
 ```
 
-Java code can call `JfrCut.cut(Path input, Instant from, Instant to, Path output)` directly without invoking the
-command-line entry point. `JfrCut.cutUsingTimeExpressions(...)` provides the relative and omitted-boundary syntax,
+Java code can call `JfrCut.cut(Path input, Instant from, Instant to, Path output)` or
+`JfrCut.cutFrom(Path input, Instant from, Path output)` directly without invoking the command-line entry point.
+`JfrCut.cutUsingTimeExpressions(...)` provides the relative and omitted-boundary syntax,
 and `JfrCut.recordingInfo(...)` returns the event range. Events overlapping the half-open interval `[from, to)`
 are retained.
 
diff --git a/tests/performance/iot-telemetry.md b/tests/performance/iot-telemetry.md
index d290f47a9e98..c421a007a87a 100644
--- a/tests/performance/iot-telemetry.md
+++ b/tests/performance/iot-telemetry.md
@@ -69,8 +69,9 @@ The value applies to each of `warmupRounds`. Every round drains its asynchronous
 `warmupRoundDelaySeconds` begins. The delay after the final round gives background JIT compilation and other
 startup work time to settle before the producer records the measurement boundary. This is a stabilization control,
 not a guarantee that the JVM has completed compilation.
-`producer-summary.json` records the warmup and measurement counts and epoch-millisecond measurement boundaries so
-the same window can be selected from broker and client JFRs.
+`producer-summary.json` records the warmup and measurement counts and epoch-millisecond measurement boundaries.
+The launcher uses the start boundary to remove warmup from broker and client JFRs and retains the rest of each
+recording. Both boundaries remain available for throughput calculations and manually selecting an exact interval.
 
 The base scenario also keeps incidental storage maintenance outside normal measurement windows. Its managed-ledger
 entry, size and time limits allow the topic and cursor ledgers to remain open throughout ordinary runs. BookKeeper
@@ -102,9 +103,10 @@ The launcher owns each `file=` option so recordings remain inside the run direct
 component unprofiled. The ordinary `run` task rejects profiling-enabled YAML rather than silently running with
 an image that lacks the native agent.
 
-After every profiled process exits, the launcher uses the measurement boundaries from `producer-summary.json` to
-write a sibling `.measurement.jfr` containing events that overlap the measured interval. The complete recording
-is retained by default. The cut recording also retains the one-time JVM, host, recording setting and runtime
+After every profiled process exits, the launcher uses the measurement start from `producer-summary.json` to
+write a sibling `.measurement.jfr` containing the remainder of each process recording. This removes startup and
+warmup while retaining asynchronous completion, consumer validation, and shutdown activity after the producer
+finishes sending. The complete recording is retained by default. The cut recording also retains the one-time JVM, host, recording setting and runtime
 configuration events needed to describe the source JVM in JDK Mission Control. Set
 `profiling.retainOriginalRecording: false` to keep only the measurement recording, or
 `profiling.createMeasurementRecording: false` to keep only the complete recording. If cutting fails, the complete
diff --git a/tests/performance/launcher/src/main/java/org/apache/pulsar/tests/performance/launcher/JfrCut.java b/tests/performance/launcher/src/main/java/org/apache/pulsar/tests/performance/launcher/JfrCut.java
index e6050e5d7644..ef53ad68f825 100644
--- a/tests/performance/launcher/src/main/java/org/apache/pulsar/tests/performance/launcher/JfrCut.java
+++ b/tests/performance/launcher/src/main/java/org/apache/pulsar/tests/performance/launcher/JfrCut.java
@@ -54,10 +54,11 @@ import picocli.CommandLine.Option;
  * when investigating startup or shutdown behavior.
  *
  * <p>Boundaries can be supplied as absolute {@link Instant} values through {@link #cut(Path, Instant, Instant, Path)}.
- * {@link #cutUsingTimeExpressions(Path, String, String, Path)} additionally accepts ISO-8601 timestamps, epoch
- * milliseconds and offsets from the recording start such as {@code 5s}. A missing boundary selects the corresponding
- * beginning or end of the recording. {@link #recordingInfo(Path)} reads the source timestamps directly from its JFR
- * chunk headers.
+ * {@link #cutFrom(Path, Instant, Path)} removes only the prefix before a boundary, which is appropriate when work
+ * triggered during a measurement can finish later. {@link #cutUsingTimeExpressions(Path, String, String, Path)}
+ * additionally accepts ISO-8601 timestamps, epoch milliseconds and offsets from the recording start such as
+ * {@code 5s}. A missing boundary selects the corresponding beginning or end of the recording.
+ * {@link #recordingInfo(Path)} reads the source timestamps directly from its JFR chunk headers.
  *
  * <p>Applications can call these methods directly. For one-off use, invoke this class through the {@code runJfrCut}
  * Gradle task.
@@ -175,6 +176,25 @@ public final class JfrCut implements Callable<Integer> {
         if (!from.isBefore(to)) {
             throw new IllegalArgumentException("JFR cut start must be before its end");
         }
+        cut(input, output, event -> isJvmContextEvent(event)
+                || event.getStartTime().isBefore(to) && !event.getEndTime().isBefore(from));
+    }
+
+    /**
+     * Removes events that finished before {@code from} while retaining the remainder of the recording. One-time JVM,
+     * host, recording setting, and runtime configuration events are retained even when they precede the boundary.
+     * This is useful for removing benchmark startup and warmup without discarding asynchronous work that finishes
+     * after the measured producer activity has ended.
+     *
+     * @param input source JFR recording
+     * @param from inclusive start boundary
+     * @param output destination JFR recording, which must differ from {@code input}
+     */
+    public static void cutFrom(Path input, Instant from, Path output) throws IOException {
+        cut(input, output, event -> isJvmContextEvent(event) || !event.getEndTime().isBefore(from));
+    }
+
+    private static void cut(Path input, Path output, Predicate<RecordedEvent> filter) throws IOException {
         Path normalizedInput = input.toAbsolutePath().normalize();
         Path normalizedOutput = output.toAbsolutePath().normalize();
         if (normalizedInput.equals(normalizedOutput)) {
@@ -184,8 +204,7 @@ public final class JfrCut implements Callable<Integer> {
         Path temporary = normalizedOutput.resolveSibling(normalizedOutput.getFileName() + ".tmp");
         Files.deleteIfExists(temporary);
         try (RecordingFile recording = new RecordingFile(normalizedInput)) {
-            write(recording, temporary, event -> isJvmContextEvent(event)
-                    || event.getStartTime().isBefore(to) && !event.getEndTime().isBefore(from));
+            write(recording, temporary, filter);
         }
         try {
             Files.move(temporary, normalizedOutput, StandardCopyOption.ATOMIC_MOVE,
diff --git a/tests/performance/launcher/src/main/java/org/apache/pulsar/tests/performance/launcher/JfrRecordingProcessor.java b/tests/performance/launcher/src/main/java/org/apache/pulsar/tests/performance/launcher/JfrRecordingProcessor.java
index 856689964674..84a66d6aeac1 100644
--- a/tests/performance/launcher/src/main/java/org/apache/pulsar/tests/performance/launcher/JfrRecordingProcessor.java
+++ b/tests/performance/launcher/src/main/java/org/apache/pulsar/tests/performance/launcher/JfrRecordingProcessor.java
@@ -48,13 +48,13 @@ final class JfrRecordingProcessor {
         }
     }
 
-    static void process(Set<Path> recordings, Instant from, Instant to,
+    static void process(Set<Path> recordings, Instant from,
                         boolean retainOriginal, boolean createMeasurementRecording) throws IOException {
         IOException failure = null;
         for (Path recording : recordings) {
             try {
                 if (createMeasurementRecording) {
-                    JfrCut.cut(recording, from, to, measurementPath(recording));
+                    JfrCut.cutFrom(recording, from, measurementPath(recording));
                 }
                 if (!retainOriginal) {
                     Files.delete(recording);
diff --git a/tests/performance/launcher/src/main/java/org/apache/pulsar/tests/performance/launcher/PerformanceLauncher.java b/tests/performance/launcher/src/main/java/org/apache/pulsar/tests/performance/launcher/PerformanceLauncher.java
index 2517f0fdb9e8..4ece3c2af4eb 100644
--- a/tests/performance/launcher/src/main/java/org/apache/pulsar/tests/performance/launcher/PerformanceLauncher.java
+++ b/tests/performance/launcher/src/main/java/org/apache/pulsar/tests/performance/launcher/PerformanceLauncher.java
@@ -176,13 +176,12 @@ public class PerformanceLauncher implements Callable<Integer> {
         if (profilingEnabled) {
             JsonNode summary = loader.mapper().readTree(runOutput.resolve("producer/producer-summary.json").toFile());
             Instant measurementStart = Instant.ofEpochMilli(requiredLong(summary, "measurementStartEpochMs"));
-            Instant measurementEnd = Instant.ofEpochMilli(requiredLong(summary, "measurementEndEpochMs"));
             Set<Path> recordings = JfrRecordingProcessor.findOriginalRecordings(runOutput);
             recordings.removeAll(recordingsBeforeRun);
             if (recordings.isEmpty()) {
                 throw new IllegalStateException("Profiling completed without producing a JFR recording");
             }
-            JfrRecordingProcessor.process(recordings, measurementStart, measurementEnd,
+            JfrRecordingProcessor.process(recordings, measurementStart,
                     retainOriginalRecording, createMeasurementRecording);
         }
         return 0;
diff --git a/tests/performance/launcher/src/test/java/org/apache/pulsar/tests/performance/launcher/JfrCutTest.java b/tests/performance/launcher/src/test/java/org/apache/pulsar/tests/performance/launcher/JfrCutTest.java
index bcca2c73fc00..ac867a17820a 100644
--- a/tests/performance/launcher/src/test/java/org/apache/pulsar/tests/performance/launcher/JfrCutTest.java
+++ b/tests/performance/launcher/src/test/java/org/apache/pulsar/tests/performance/launcher/JfrCutTest.java
@@ -69,14 +69,16 @@ public class JfrCutTest {
         Path directory = Files.createTempDirectory("jfr-retention-test");
         try {
             Path input = directory.resolve("profile.jfr");
-            createRecording(input);
+            Instant[] interval = createRecording(input);
             Path measurement = JfrRecordingProcessor.measurementPath(input);
 
-            JfrRecordingProcessor.process(Set.of(input), Instant.EPOCH,
-                    Instant.now().plus(1, ChronoUnit.HOURS), retainOriginal, createMeasurement);
+            JfrRecordingProcessor.process(Set.of(input), interval[0], retainOriginal, createMeasurement);
 
             assertEquals(Files.exists(input), retainOriginal);
             assertEquals(Files.exists(measurement), createMeasurement);
+            if (createMeasurement) {
+                assertEquals(markers(measurement), List.of("measurement", "after"));
+            }
         } finally {
             deleteDirectory(directory);
         }