You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "lhotari (via GitHub)" <gi...@apache.org> on 2026/06/20 18:31:47 UTC
[I] [enhancement] Reduce pulsar-admin / CLI startup time with Class Data Sharing (CDS) via -XX:+AutoCreateSharedArchive [pulsar]
lhotari opened a new issue, #26069:
URL: https://github.com/apache/pulsar/issues/26069
### Motivation
Every invocation of `pulsar-admin` (and the other CLI tools — `pulsar-client`, `pulsar-perf`, `pulsar-shell`) pays the full JVM startup + class‑loading cost. In #22318, `pulsar-admin tenants list` was measured at **~2.4s** of wall‑clock, most of it JVM warmup and class loading. For scripting and interactive use this is a noticeable tax.
Building a GraalVM **native image** (#22318) cuts that ~9× (≈0.26s), but it requires per‑platform builds, ongoing reflection‑config maintenance, and extra CI cost, and the broader‑stack reflection (Jackson model serialization, auth plugins, Netty) makes a complete native `pulsar-admin` hard — as discussed in that issue.
**Class Data Sharing (CDS)** is the cheaper, lower‑risk win already suggested in #22318. It works with the existing jars, has no closed‑world/reflection constraints, and delivers a meaningful startup reduction "for relatively little work" (Brian Goetz, quoted in #22318). Since **JDK 19**, `-XX:+AutoCreateSharedArchive` (JDK‑8261455) makes it almost free to wire up: a single flag that **auto‑creates** the dynamic CDS archive on first run and **auto‑regenerates** it when the archive is missing/invalid or after a JDK upgrade — no two‑step dump and no manual archive lifecycle to manage.
This is a natural follow‑up to the CDS idea raised in #22318 (where @nodece offered "Let me try CDS"), and complements, rather than competes with, the native‑image effort.
### Solution
Wire CDS into the CLI launcher scripts, gated on the detected JDK version. The common script `bin/pulsar-admin-common.sh` already detects `JAVA_MAJOR_VERSION` and assembles `OPTS`, so this is the natural hook (with the equivalent in the `.cmd` scripts):
- **JDK ≥ 19** (covers Pulsar's supported Java 21): single‑step auto‑create dynamic CDS
```
-XX:+AutoCreateSharedArchive -XX:SharedArchiveFile=<archive>
```
relying on the default `-Xshare:auto`.
- **JDK 17–18** (still supported for the CLI): either the older two‑step dynamic CDS
(`-XX:ArchiveClassesAtExit=<archive>` to seed, then `-XX:SharedArchiveFile=<archive>` to load),
or simply skip CDS. `-XX:+AutoCreateSharedArchive` does **not** exist before JDK 19.
Details:
- **Archive location:** a writable, per‑user path keyed per tool, e.g. `${PULSAR_CDS_DIR:-$PULSAR_HOME/cds}/pulsar-admin.jsa` (or an XDG/`$TMPDIR` cache dir). The JVM handles JDK‑version mismatch automatically with `AutoCreateSharedArchive`; using distinct files per tool avoids classpath‑mismatch invalidation between tools.
- **Opt‑out:** an env toggle (e.g. `PULSAR_CDS_ENABLED=false`) so users on read‑only/locked‑down environments can disable it; users can always add flags via `PULSAR_EXTRA_OPTS`.
- **Safety:** keep `-Xshare:auto` (never `-Xshare:on`) so a read‑only filesystem or a failed archive map falls back silently instead of crashing.
- **Default on for the CLI tools** (short‑lived, frequently invoked — the prime CDS target). Extending CDS to the broker/bookie launchers (`conf/pulsar_env.sh`) can be a follow‑up; for long‑lived servers the relative startup benefit is smaller.
### Alternatives
- **GraalVM native image** (#22318) — much larger startup win (~9×) but high effort: per‑platform builds, reflection‑config maintenance, CI cost; gated on broader‑stack reflection.
- **Two‑step AppCDS only** — drop the JDK‑19 path and always use `-XX:ArchiveClassesAtExit` + `-XX:SharedArchiveFile`. More portable (works on 17/18) but needs explicit archive‑lifecycle logic in the scripts and doesn't self‑heal on JDK upgrade.
- **Do nothing** — keep paying full startup cost per invocation.
### Anything else?
Things to validate during implementation:
- **JDK‑version gate is essential.** `-XX:+AutoCreateSharedArchive` is **JDK 19+** (JDK‑8261455); it is absent in JDK 17/18. Pulsar's CLI runs on Java 17 or 21, so the script must branch on `JAVA_MAJOR_VERSION`.
- **Classpath gotcha.** Pulsar's CLI classpath includes the `conf` directory and `lib/*`. CDS archiving has known trouble with **non‑empty directories on the classpath** (JDK‑8329980) — verify archive creation succeeds with Pulsar's classpath and adjust if needed.
- **Classpath must match** between create and load; Pulsar's is stable per install, and `-Xshare:auto` + `AutoCreateSharedArchive` self‑heal on mismatch/upgrade. The **first** invocation builds the archive on exit and sees no benefit; subsequent runs do.
- **Read‑only containers:** the archive path must be writable, otherwise CDS silently no‑ops (acceptable with `-Xshare:auto`).
- **Expected gain is real but app‑dependent and must be measured**, not assumed. App‑CDS typically yields roughly **20–35%** off startup (Spring's CDS report shows ~30–45% in some scenarios, with caveats). It will not match native‑image's ~9×, but it's far cheaper and broadly applicable. Worth benchmarking `pulsar-admin` startup before/after.
- **Prior art / references:** #22318 (native image + the original CDS suggestion), #25883 (embedded GraalVM native‑image config in the client), #17211 and #4194 (earlier native‑image explorations).
- **Docs:** [Oracle JDK 21 — Class Data Sharing](https://docs.oracle.com/en/java/javase/21/vm/class-data-sharing.html) · [`AutoCreateSharedArchive` in the `java` man page](https://docs.oracle.com/en/java/javase/21/docs/specs/man/java.html#creating-dynamic-cds-archive-file-with--xxautocreatesharedarchive) · [JDK‑8261455](https://bugs.openjdk.org/browse/JDK-8261455) · [Spring CDS blog](https://spring.io/blog/2023/12/04/cds-with-spring-framework-6-1).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [enhancement] Reduce pulsar-admin / CLI startup time with Class Data Sharing (CDS) via -XX:+AutoCreateSharedArchive [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat commented on issue #26069:
URL: https://github.com/apache/pulsar/issues/26069#issuecomment-6388662529
@hozumi's findings change the plan. `-XX:+AutoCreateSharedArchive` alone does nothing in the Docker image, because the jlink'd JDK has no base CDS archive, and one-off `docker run` calls never reuse an archive. A PR would need two parts: (1) keep the CLI classpath jar-only by passing the log4j2 config by path, which any CDS approach requires, and (2) bake a static `pulsar-admin` archive at image build time, with the `AutoCreateSharedArchive` path in `bin/pulsar-admin-common.sh` as the option for tarball installs on JDK 19 or later. Please include startup timings from before and after the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [enhancement] Reduce pulsar-admin / CLI startup time with Class Data Sharing (CDS) via -XX:+AutoCreateSharedArchive [pulsar]
Posted by "hozumi (via GitHub)" <gi...@apache.org>.
hozumi commented on issue #26069:
URL: https://github.com/apache/pulsar/issues/26069#issuecomment-4831479499
I experimented with CDS for `pulsar-admin` on the Docker image based on this — sharing a few findings in case they're useful.
**`-XX:+AutoCreateSharedArchive` doesn't work as-is on the Docker image.** The jlink'd JDK in the image ships no default CDS archive (`$JAVA_HOME/lib/server/classes.jsa` is absent and nothing runs `-Xshare:dump`), and a dynamic archive needs a base archive loaded — so auto-create fails on every run:
```
[warning][cds] -XX:ArchiveClassesAtExit is unsupported when base CDS archive is not loaded.
[info][cds] Unable to map shared spaces
```
Also, since `AutoCreateSharedArchive` only pays off from the *second* invocation and only if the archive persists, one-shot `docker run … pulsar-admin …` invocations (and CI) never benefit — only a long-lived container or a persistent `cds/` volume does.
**What worked on the image:** bake a single *self-contained* static archive for `pulsar-admin` at image-build time — a `-XX:DumpLoadedClassList` training run against an unreachable `--admin-url` (loads the REST client stack, no broker needed), then a static `-Xshare:dump` — placed under `/pulsar/cds` rather than `/opt/jvm`. Keeping it out of the default location means only `pulsar-admin` loads it and the servers are untouched; a shared `/opt/jvm` base would also hit a per-launcher `--add-opens` mismatch and disable CDS "optimized module handling" for every launcher except the one the base was dumped for (JDK 25+). Being self-contained, it needs no base archive and every invocation benefits immediately:
| setup | `clusters list` |
|---|---|
| as shipped | ~6.1s |
| + AppCDS | ~3.9s |
| + AppCDS & `-XX:TieredStopAtLevel=1` | ~3.0s |
(median of 30, Corretto 25, local `master` build).
On the `conf/` non-empty-directory classpath gotcha you noted: passing the log4j2 config by absolute path (instead of prepending `conf/` to the classpath and resolving by basename) keeps the classpath jar-only and unblocks the dump — that part is needed for any CDS approach here.
For reference, here's how I did it: https://github.com/hozumi/pulsar/pull/1/changes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [enhancement] Reduce pulsar-admin / CLI startup time with Class Data Sharing (CDS) via -XX:+AutoCreateSharedArchive [pulsar]
Posted by "hozumi (via GitHub)" <gi...@apache.org>.
hozumi commented on issue #26069:
URL: https://github.com/apache/pulsar/issues/26069#issuecomment-6451794056
Here's where the CDS work landed — one commit with the full implementation (classpath prep + Docker
static archive + tarball AutoCreate path): https://github.com/hozumi/pulsar/commit/f24d7b26419109dfabfaef5f22a53ea69bece3a6
Building it out and then having it adversarially reviewed (a few rounds, with Claude Opus 5.5) kept
surfacing corner cases that need care. Sharing before taking it further.
It works. Measured on `topics list` — a command the archive is *not* trained on (training is
`clusters list`), so a realistic floor rather than a best case — against a running broker, median of 15
(Apple M2):
- Docker image, static archive: ~1.77s → ~1.19s (~1.49×)
- tarball install, dynamic archive (auto-created on first run): ~0.90s → ~0.67s (~1.34×)
(Higher absolute times in the image are the containerized jlink'd JDK; the ratio is the point. The same
archive also helps `pulsar-shell` (~1.4×) and `pulsar-client` (~0.3s off startup; its ratio is diluted by
`produce`'s network time).)
But CDS is more environment- and JDK-sensitive than I'd assumed (details in the commit):
- **classpath order**: `java -cp .../lib/*` expands in filesystem order, so a prebuilt archive silently
no-ops on a host that enumerates `lib/` differently (e.g. a k8s node vs the build host). It needs a
sorted classpath to be portable — and this hid itself in local testing, where the filesystem never
changed.
- **sharp edges**: `-XX:+AutoCreateSharedArchive` aborts the JVM (exit 1) on a read-only cds dir; a benign
classpath mismatch logs `[error][cds]` to stdout, breaking `pulsar-admin … | jq`, unless silenced.
- **JDK-version-dependent**: AutoCreate needs JDK 19+ and a *usable* base archive. The jlink'd image JDK
ships none; and on JDK 25/26, where Pulsar forces `-XX:+UseCompactObjectHeaders`, it needs the
compact-headers base variant (`classes_coh.jsa`) specifically — standard JDKs ship it, but minimal or
jlink'd builds may not. In all these cases it degrades to a silent no-op (no crash, just no speedup).
Against a real but modest ~0.5s/invocation, that's +63 MB image size, ~12s added to the build for the
train+dump step (native, Apple M2; likely more where a multi-arch build emulates the non-native arch under
QEMU), and a fragile path to maintain.
Also — I hadn't known about `pulsar-shell`. Where faster startup would actually help me is scripting a
local test setup (standalone + a batch of tenant/namespace/topic creates), and `pulsar-shell -f` already
runs all of that in one JVM — far better than CDS can. (An inline multi-command form for `pulsar-shell`, not
just `-f <file>`, would be handy.)
So I'm not sure the full CDS machinery is worth its weight. A few questions:
1. Is it worth pursuing at all, given `pulsar-shell` covers the batch case and CDS's remaining win is
~0.5s on single invocations?
2. If yes, how would you prefer it split? The commit separates cleanly into: (a) classpath cleanup
(log4j-by-path + deterministic sorted classpath), (b) the Docker static archive, (c) the tarball
`AutoCreateSharedArchive` path.
3. Either way, (a) looks worth doing on its own — a deterministic classpath order (instead of
filesystem-dependent) is a reasonable cleanup regardless of CDS, and it lets anyone build their own CDS
archive. Happy to send just that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org