(causeway) 03/14: CAUSEWAY-3975: proposes Boot 2.7 tracing compatibility spike

[da...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

danhaywood pushed a commit to branch CAUSEWAY-3975
in repository https://gitbox.apache.org/repos/asf/causeway.git

commit 191e8d225b3a55c2643b464b97b4c6c1e42d1892
Author: Dan Haywood <da...@haywood-associates.co.uk>
AuthorDate: Mon Sep 21 16:59:01 2026 +0100

    CAUSEWAY-3975: proposes Boot 2.7 tracing compatibility spike
---
 .../.openspec.yaml                                 |   2 +
 .../verify-boot27-micrometer-tracing/design.md     | 115 +++++++++++++++++++++
 .../verify-boot27-micrometer-tracing/proposal.md   |  31 ++++++
 .../spec.md                                        |  43 ++++++++
 .../verify-boot27-micrometer-tracing/tasks.md      |  25 +++++
 .../planned-changes/micrometer-tracing-backport.md |  79 ++++++++++++++
 6 files changed, 295 insertions(+)

diff --git a/openspec/changes/verify-boot27-micrometer-tracing/.openspec.yaml b/openspec/changes/verify-boot27-micrometer-tracing/.openspec.yaml
new file mode 100644
index 00000000000..563fab5abae
--- /dev/null
+++ b/openspec/changes/verify-boot27-micrometer-tracing/.openspec.yaml
@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-09-21
diff --git a/openspec/changes/verify-boot27-micrometer-tracing/design.md b/openspec/changes/verify-boot27-micrometer-tracing/design.md
new file mode 100644
index 00000000000..0c6d54988e9
--- /dev/null
+++ b/openspec/changes/verify-boot27-micrometer-tracing/design.md
@@ -0,0 +1,115 @@
+## Context
+
+Causeway 4 instruments framework operations through Micrometer Observation and relies on modern Spring Boot auto-configuration to connect observations to OpenTelemetry.
+This maintenance branch uses Spring Boot 2.7.18, Java 11, and Boot-managed Micrometer Core 1.9.17, so it has neither the required Observation generation nor the newer tracing auto-configuration.
+
+Micrometer Observation is distributed as a separate artifact from Micrometer 1.10 onward, and Micrometer Tracing 1.0 uses that API.
+The initial compatibility hypothesis is that `micrometer-observation` 1.10.x and `micrometer-tracing` 1.0.x can coexist with Boot-managed `micrometer-core` 1.9.17 when the global Micrometer version is not overridden.
+The transcript motivating this change identifies `micrometer-observation` 1.10.13 and `micrometer-tracing` 1.0.12 as an example aligned pair, but the spike must verify and record the complete resolved dependency set rather than assuming compatibility.
+
+The intended deployment uses the OpenTelemetry Java agent for Servlet and JDBC instrumentation and for OTLP export.
+Causeway's future semantic observations must join the agent's process-wide context so that agent-created JDBC spans become descendants of Causeway-created spans.
+
+This phase creates evidence only.
+It does not change production Causeway instrumentation.
+
+## Goals / Non-Goals
+
+**Goals:**
+
+- Prove that Micrometer Observation and Tracing can run on the repository's Spring Boot 2.7.18 and Java 11 baseline without replacing Boot-managed Micrometer Core 1.9.17.
+- Prove that a Micrometer observation backed by `micrometer-tracing-bridge-otel` can use the Java agent's `GlobalOpenTelemetry` context.
+- Prove that a JDBC span created by Java-agent instrumentation is a direct or expected descendant of the custom Micrometer span surrounding the JDBC operation.
+- Produce reproducible evidence containing the selected versions, resolved dependency graph, execution command, and observed trace identifiers.
+- Verify that the same compatibility harness remains safe when launched without the Java agent.
+
+**Non-Goals:**
+
+- Add production observation configuration or framework instrumentation.
+- Add root-interaction, action, Wicket, transaction, property-edit, execution-publishing, or JPA semantic spans.
+- Introduce Spring Cloud Sleuth.
+- Override Boot's `micrometer.version` or upgrade `micrometer-core` globally.
+- Construct an application-owned `OpenTelemetrySdk` or configure an application-owned OTLP exporter.
+- Establish final production sampling, baggage, privacy, or dynamic-enable policies.
+
+## Decisions
+
+### Keep the proof isolated from production modules
+
+The compatibility harness will live in regression-test or equivalent test-only code and will use test-scoped dependencies.
+No production Causeway module will acquire Micrometer Observation, Micrometer Tracing, bridge, SDK, or exporter dependencies during this phase.
+
+This isolates dependency experiments from the decommissioned maintenance branch's production artifacts and makes a failed compatibility result cheap to remove.
+
+**Alternative considered:** add the dependencies directly to `core/config` and validate them through a production bean.
+This is rejected for the spike because it changes published production artifacts before compatibility has been demonstrated.
+
+### Retain Boot-managed Micrometer Core
+
+The harness will add explicit, mutually compatible Observation and Tracing artifacts while leaving Boot's Micrometer Core dependency management unchanged.
+It will not set or override the global `micrometer.version` property.
+
+The initial candidate line is Micrometer Observation 1.10.x with Micrometer Tracing 1.0.x.
+The implementation must inspect the resolved tree and pin all versions required to make the result reproducible.
+
+**Alternative considered:** upgrade the complete Micrometer family to 1.10.x.
+This is rejected because Spring Boot 2.7.18 was built and tested against Micrometer Core 1.9.17.
+
+### Build the bridge from GlobalOpenTelemetry
+
+The harness will construct Micrometer's OpenTelemetry-backed tracer from `GlobalOpenTelemetry.getTracer(...)` and install the corresponding tracing observation handler in an `ObservationRegistry`.
+It will not call `OpenTelemetrySdk.builder()` or otherwise create another SDK.
+
+This allows Micrometer-created spans to share the context and exporter installed by the Java agent.
+It also models the boundary intended for the later production integration: Causeway code uses Micrometer while Boot 2.7-specific integration code understands the OpenTelemetry bridge.
+
+**Alternative considered:** use the OpenTelemetry API directly from Causeway instrumentation.
+This is rejected because it would diverge from the Causeway 4 Micrometer abstraction and make a later Spring Boot upgrade harder.
+
+**Alternative considered:** configure a standalone OpenTelemetry SDK and exporter inside the test application.
+This is rejected because it would not prove interoperability with the Java agent and could conceal a split-context or duplicate-exporter problem.
+
+### Use a forked JVM with the real Java agent
+
+The compatibility test will launch a small Java 11 child process with the actual OpenTelemetry Java agent, create a Micrometer observation around a real JDBC operation, and capture exported spans.
+The preferred initial capture mechanism is the agent's logging trace exporter because it avoids requiring Docker, an external collector, or an application-owned SDK.
+The test will identify the custom and JDBC spans and compare their trace and parent identifiers.
+
+The same helper process will also be launched without `-javaagent` to verify that the wiring degrades safely rather than preventing startup or JDBC execution.
+
+**Alternative considered:** create OTel spans directly in a unit test to simulate JDBC instrumentation.
+This is rejected as the sole proof because it would not exercise the Java agent's instrumentation and context behavior.
+
+**Alternative considered:** require Jaeger, Tempo, or an OpenTelemetry Collector through Testcontainers.
+This is deferred because it would add Docker and external-image requirements to a narrow compatibility spike.
+
+### Treat the result as a gate for later phases
+
+A successful build alone is insufficient.
+The phase succeeds only when the evidence demonstrates both dependency compatibility and the expected span parentage.
+If the bridge and agent produce disconnected traces, subsequent production integration must not proceed until the context-sharing design is revised.
+
+## Risks / Trade-offs
+
+- **[Risk] Micrometer 1.10-era support artifacts may pull versions that conflict with Boot-managed Micrometer Core 1.9.17.** → Inspect and record the complete dependency tree, prohibit the global Micrometer override, and treat convergence failures as a no-go result rather than forcing exclusions without explanation.
+- **[Risk] Micrometer Tracing bridge constructor signatures differ across patch versions.** → Pin one verified version set and keep all version-specific construction inside the isolated harness and, later, the Boot 2.7 integration boundary.
+- **[Risk] A modern Java agent may use a different OpenTelemetry API generation from the bridge.** → Pin and report the agent version, execute the actual forked-JVM test, and rely on observed parentage rather than API compatibility assumptions.
+- **[Risk] The logging exporter format may be inconvenient or unstable to parse.** → Keep span capture behind a test helper; if necessary, replace it with a local OTLP capture endpoint without changing the compatibility requirement.
+- **[Risk] A no-agent run can return a no-op global implementation and therefore cannot prove export.** → Use it only to prove safe degradation; require the agent-enabled run for parentage evidence.
+- **[Trade-off] The spike does not prove the complete future HTTP-to-JDBC trace.** → It proves the critical Micrometer-to-agent JDBC boundary first; later phases will add production interaction/action observations and end-to-end HTTP coverage.
+
+## Migration Plan
+
+1. Add only test-scoped dependency and harness changes.
+2. Run dependency convergence and the compatibility test on Java 11.
+3. Record the selected versions, commands, and span-parentage evidence.
+4. If the proof succeeds, use its pinned recipe as input to the planned observation-substrate change.
+5. If it fails, remove the isolated harness or retain it as failure evidence and reconsider the bridge, agent version, or lower-level Micrometer Tracing approach.
+
+Rollback consists of removing the test-only harness and dependencies because this phase changes no production runtime behavior or public API.
+
+## Open Questions
+
+- Which exact OpenTelemetry Java agent version provides the best supported intersection with the selected Micrometer Tracing 1.0.x bridge while retaining Java 11 compatibility?
+- Does the agent's logging exporter expose sufficiently stable trace and parent identifiers for an automated assertion, or will a small local OTLP capture endpoint be required?
+- Which existing regression-test module provides the smallest reliable Spring Boot 2.7 and JDBC fixture, or is a dedicated compatibility-test module clearer?
diff --git a/openspec/changes/verify-boot27-micrometer-tracing/proposal.md b/openspec/changes/verify-boot27-micrometer-tracing/proposal.md
new file mode 100644
index 00000000000..bb0d8a789f0
--- /dev/null
+++ b/openspec/changes/verify-boot27-micrometer-tracing/proposal.md
@@ -0,0 +1,31 @@
+## Why
+
+Apache Causeway 2.x uses Spring Boot 2.7.18 and its managed Micrometer 1.9 generation, while the Causeway 4 tracing implementation relies on Micrometer Observation and Tracing facilities supplied by newer Spring Boot generations.
+Before changing production framework code, the project needs an executable compatibility proof that a separately versioned Micrometer tracing stack can join the trace context owned by an OpenTelemetry Java agent without replacing Boot's managed Micrometer Core or creating a second OpenTelemetry SDK.
+
+## What Changes
+
+- Add an isolated compatibility proof for Micrometer Observation and Tracing on the existing Spring Boot 2.7 and Java 11 baseline.
+- Pin and document a compatible set of Micrometer Observation, Micrometer Tracing, Micrometer OpenTelemetry bridge, and OpenTelemetry API versions without overriding Boot's managed `micrometer-core` version.
+- Wire the Micrometer OpenTelemetry bridge to the agent-provided `GlobalOpenTelemetry` instance.
+- Verify that a custom Micrometer span around a JDBC operation and the agent-generated JDBC span belong to one trace with the expected parent-child relationship.
+- Verify safe no-op behavior when tracing is disabled or the OpenTelemetry Java agent is absent.
+- Record the validated dependency and wiring decisions for the subsequent production backport phases.
+- Make no production Causeway interaction, action, Wicket, transaction, or persistence instrumentation changes in this phase.
+
+## Capabilities
+
+### New Capabilities
+
+- `boot27-micrometer-tracing-compatibility`: Defines the compatibility evidence required before Micrometer-based tracing is integrated into Causeway 2.x production modules.
+
+### Modified Capabilities
+
+None.
+
+## Impact
+
+The change affects only compatibility-test or spike code, its test dependencies, and supporting documentation.
+It evaluates new Micrometer Observation, Micrometer Tracing, Micrometer OpenTelemetry bridge, and OpenTelemetry API dependencies while retaining Spring Boot 2.7.18's managed Micrometer Core 1.9.17.
+It must not add an application-owned OpenTelemetry SDK or exporter, change public Causeway APIs, or alter production request and interaction behavior.
+The result is a go/no-go decision and a pinned integration recipe for the later framework-substrate and semantic-instrumentation changes described in `openspec/planned-changes/micrometer-tracing-backport.md`.
diff --git a/openspec/changes/verify-boot27-micrometer-tracing/specs/boot27-micrometer-tracing-compatibility/spec.md b/openspec/changes/verify-boot27-micrometer-tracing/specs/boot27-micrometer-tracing-compatibility/spec.md
new file mode 100644
index 00000000000..30d93c40f88
--- /dev/null
+++ b/openspec/changes/verify-boot27-micrometer-tracing/specs/boot27-micrometer-tracing-compatibility/spec.md
@@ -0,0 +1,43 @@
+## ADDED Requirements
+
+### Requirement: Preserve the Spring Boot Micrometer baseline
+The compatibility proof SHALL retain Spring Boot 2.7.18's managed Micrometer Core 1.9.17 while resolving an explicitly pinned and internally compatible Micrometer Observation and Tracing stack.
+The proof MUST NOT override the global `micrometer.version` property.
+
+#### Scenario: Resolve the compatibility dependency graph
+- **WHEN** the compatibility module's dependency graph is resolved
+- **THEN** `micrometer-core` resolves to the Spring Boot 2.7.18-managed version and the separately pinned Observation, Tracing, and OpenTelemetry bridge artifacts resolve without unexplained convergence conflicts
+
+### Requirement: Use the Java agent's OpenTelemetry runtime
+The compatibility proof SHALL construct the Micrometer OpenTelemetry bridge from the process-wide `GlobalOpenTelemetry` instance supplied by the OpenTelemetry Java agent.
+It MUST NOT construct a second `OpenTelemetrySdk` or configure a second trace exporter in application code.
+
+#### Scenario: Run with the OpenTelemetry Java agent
+- **WHEN** the compatibility harness is launched with the pinned OpenTelemetry Java agent
+- **THEN** Micrometer observations use the agent-owned OpenTelemetry context and exporter
+
+### Requirement: Prove Micrometer-to-agent span parentage
+The compatibility proof SHALL execute a real JDBC operation inside a custom Micrometer observation and SHALL verify that the custom span and agent-generated JDBC span belong to the same trace with the expected parent-child relationship.
+
+#### Scenario: Observe a JDBC operation
+- **WHEN** the agent-enabled compatibility harness executes JDBC work inside the custom Micrometer observation
+- **THEN** the exported JDBC span has the same trace identifier as the custom span and identifies the custom span as its parent or expected semantic ancestor
+
+### Requirement: Degrade safely without the Java agent
+The compatibility harness SHALL remain executable when the OpenTelemetry Java agent is absent.
+Absence of the agent MUST NOT prevent application startup or completion of the JDBC operation.
+
+#### Scenario: Run without the OpenTelemetry Java agent
+- **WHEN** the compatibility harness is launched without `-javaagent`
+- **THEN** the harness starts and completes its JDBC operation without requiring an application-owned OpenTelemetry SDK or exporter
+
+### Requirement: Produce reproducible compatibility evidence
+The compatibility proof SHALL record the Java version, Spring Boot version, Micrometer versions, OpenTelemetry bridge and agent versions, resolved dependency graph, execution command, and span-parentage result.
+
+#### Scenario: Review successful proof evidence
+- **WHEN** the compatibility proof passes
+- **THEN** a maintainer can reproduce the proof and identify the exact validated dependency and runtime configuration from the recorded evidence
+
+#### Scenario: Review failed proof evidence
+- **WHEN** dependency compatibility or span parentage cannot be demonstrated
+- **THEN** the evidence identifies the failed gate and the project does not treat the tracing backport as ready for production integration
diff --git a/openspec/changes/verify-boot27-micrometer-tracing/tasks.md b/openspec/changes/verify-boot27-micrometer-tracing/tasks.md
new file mode 100644
index 00000000000..7bbd5f515de
--- /dev/null
+++ b/openspec/changes/verify-boot27-micrometer-tracing/tasks.md
@@ -0,0 +1,25 @@
+## 1. Establish the Compatibility Fixture
+
+- [ ] 1.1 Select the smallest existing regression-test module, or create a dedicated test-only module, that can run a Java 11 Spring Boot 2.7.18 application with a real JDBC operation.
+- [ ] 1.2 Add test-scoped, explicitly pinned candidate dependencies for Micrometer Observation, Micrometer Tracing, the Micrometer OpenTelemetry bridge, the OpenTelemetry API, and the OpenTelemetry Java agent without overriding `micrometer.version`.
+- [ ] 1.3 Capture and review the resolved dependency tree, confirming that `micrometer-core` remains at Spring Boot 2.7.18's managed version and documenting every required exclusion or alignment decision.
+
+## 2. Build the Agent Interoperability Harness
+
+- [ ] 2.1 Add a Java 11-compatible child-process fixture that performs a real JDBC operation inside a custom Micrometer observation.
+- [ ] 2.2 Configure the fixture's `ObservationRegistry` and tracing observation handler with a Micrometer `OtelTracer` built from `GlobalOpenTelemetry.getTracer(...)` without constructing an `OpenTelemetrySdk`.
+- [ ] 2.3 Add a parent-process test harness that launches the fixture with the pinned OpenTelemetry Java agent and captures the custom and agent-generated JDBC spans through the agent-owned exporter.
+- [ ] 2.4 Add span parsing and assertions that verify the custom and JDBC spans share a trace identifier and that the JDBC span names the custom span as its parent or expected semantic ancestor.
+
+## 3. Verify Safe Degradation and Build Compatibility
+
+- [ ] 3.1 Launch the same fixture without `-javaagent` and verify that startup and JDBC execution complete without an application-owned SDK or exporter.
+- [ ] 3.2 Run the compatibility tests using the repository's Java 11 compilation target and relevant regression-test Maven profile.
+- [ ] 3.3 Run dependency convergence and targeted module verification, resolving failures without changing Spring Boot's managed Micrometer Core version.
+
+## 4. Record the Gate Result
+
+- [ ] 4.1 Record the validated Java, Spring Boot, Micrometer, OpenTelemetry bridge, and Java-agent versions together with the resolved dependency graph and reproduction commands.
+- [ ] 4.2 Record the custom-span and JDBC-span trace identifiers and parentage evidence, or document the failed compatibility gate if the expected relationship cannot be demonstrated.
+- [ ] 4.3 Summarize the go/no-go result and the pinned integration recipe that the planned Boot 2.7 observation-substrate change must use.
+- [ ] 4.4 Validate the completed OpenSpec change and confirm that no production Causeway module or public API was changed by the compatibility phase.
diff --git a/openspec/planned-changes/micrometer-tracing-backport.md b/openspec/planned-changes/micrometer-tracing-backport.md
new file mode 100644
index 00000000000..2c082eb3652
--- /dev/null
+++ b/openspec/planned-changes/micrometer-tracing-backport.md
@@ -0,0 +1,79 @@
+# Micrometer Tracing Backport Roadmap
+
+## Objective
+
+Backport a narrow, low-risk subset of Causeway 4's Micrometer-based tracing to the Spring Boot 2.7 maintenance branch.
+The target trace initially covers an agent-created HTTP span, Causeway root-interaction and action spans, and an agent-created JDBC span.
+
+```text
+OpenTelemetry agent: HTTP request
+└── Micrometer: Causeway root interaction
+    └── Micrometer: action invocation
+        └── OpenTelemetry agent: JDBC
+```
+
+The OpenTelemetry Java agent owns automatic HTTP and JDBC instrumentation, the OpenTelemetry SDK, sampling, and exporting.
+Causeway owns only its semantic spans and must join the agent's process-wide trace context through `GlobalOpenTelemetry`.
+
+## Guiding Constraints
+
+- Retain Spring Boot 2.7.18 and its managed `micrometer-core` 1.9.17.
+- Do not override Boot's global Micrometer version.
+- Keep the implementation compatible with Java 11.
+- Do not create an application-owned `OpenTelemetrySdk` or a second OTLP exporter.
+- Keep tracing disabled or no-op unless explicitly activated.
+- Avoid the broad interaction-model refactoring included in the Causeway 4 `CAUSEWAY-3975` commit series.
+- Exclude sensitive or unbounded tags such as action arguments, usernames, tenancy tokens, and bookmarks from the initial slice.
+
+## Phase 1: Verify Boot 2.7 Compatibility
+
+**OpenSpec change:** `verify-boot27-micrometer-tracing`
+
+Establish an executable compatibility proof before changing production framework behavior.
+Pin a compatible Micrometer Observation, Micrometer Tracing, OpenTelemetry bridge, and OpenTelemetry API version set without replacing Boot's Micrometer Core.
+Construct the Micrometer bridge from the Java agent's `GlobalOpenTelemetry` instance.
+Prove that a custom Micrometer span becomes the parent of an agent-generated JDBC span.
+Verify safe behavior when tracing or the Java agent is absent.
+
+**Exit gate:** dependency convergence passes and the exported trace proves the expected custom-span-to-JDBC-span parentage.
+
+## Phase 2: Add the Boot 2.7 Observation Substrate
+
+Add Java 11-compatible production integration around `ObservationRegistry` and Micrometer Tracing.
+Provide a no-op default and opt-in activation using the `observation` Spring profile, matching the Causeway 4 convention where practical.
+Keep the OpenTelemetry-specific bridge wiring at the integration boundary so core instrumentation depends on Micrometer rather than directly on OpenTelemetry.
+Add lifecycle and failure tests for observation scope creation and cleanup.
+
+**Exit gate:** Causeway applications behave unchanged when observation is inactive, while an activated application can create and export a framework-owned test observation through the agent.
+
+## Phase 3: Instrument Root Interactions and Actions
+
+Create one observation for each top-level Causeway interaction using the existing `InteractionServiceDefault` lifecycle without refactoring the interaction model.
+Create one child observation around `MemberExecutorServiceDefault.invokeAction(...)`.
+Use stable, low-cardinality observation names and put the action identifier and initiation mode in explicit tags.
+Record failures on the active observation and close scopes reliably on success and failure.
+Do not add custom Wicket, transaction, property-edit, execution-publishing, or JPA semantic observations in this phase.
+
+**Exit gate:** an integration trace demonstrates `HTTP → root interaction → action → JDBC` as one correctly parented trace.
+
+## Phase 4: Document and Operationalize
+
+Document Java-agent installation, `JAVA_TOOL_OPTIONS`, service naming, OTLP endpoint configuration, sampling, and the `observation` profile.
+Document that the agent owns the SDK and exporter and that applications must not configure a duplicate in-process exporter.
+Document the initial span names, tags, privacy policy, troubleshooting steps, and the behavior when the agent is absent.
+
+**Exit gate:** an application team can enable, disable, and diagnose the narrow tracing slice without changing Causeway framework code.
+
+## Deferred Extensions
+
+The following remain separate decisions after the narrow slice is proven in production:
+
+- Wicket request-cycle observations beyond the agent's Servlet span.
+- Nested Causeway interaction observations.
+- Transaction observations.
+- Property-edit and execution-publishing observations.
+- JPA semantic observations and duration-based filtering.
+- Runtime enablement changes without an application restart.
+- Baggage propagation.
+- Metrics derived from the same observations.
+- Additional exporters or an application-owned OpenTelemetry SDK.