(causeway) 01/41: CAUSEWAY-4060: proposes metamodel introspection diagnostics

[da...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

danhaywood pushed a commit to branch CAUSEWAY-4066
in repository https://gitbox.apache.org/repos/asf/causeway.git

commit 8bfbbbbd2b61f854b3e9c74dc385a869b52a4664
Author: Dan Haywood <da...@haywood-associates.co.uk>
AuthorDate: Wed Sep 23 12:00:47 2026 +0100

    CAUSEWAY-4060: proposes metamodel introspection diagnostics
---
 .../.openspec.yaml                                 |  2 +
 .../design.md                                      | 99 ++++++++++++++++++++++
 .../proposal.md                                    | 28 ++++++
 .../metamodel-introspection-diagnostics/spec.md    | 59 +++++++++++++
 .../tasks.md                                       | 25 ++++++
 5 files changed, 213 insertions(+)

diff --git a/openspec/changes/diagnose-metamodel-introspection-recursion/.openspec.yaml b/openspec/changes/diagnose-metamodel-introspection-recursion/.openspec.yaml
new file mode 100644
index 00000000000..265da3d94ed
--- /dev/null
+++ b/openspec/changes/diagnose-metamodel-introspection-recursion/.openspec.yaml
@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-09-23
diff --git a/openspec/changes/diagnose-metamodel-introspection-recursion/design.md b/openspec/changes/diagnose-metamodel-introspection-recursion/design.md
new file mode 100644
index 00000000000..08d1994b347
--- /dev/null
+++ b/openspec/changes/diagnose-metamodel-introspection-recursion/design.md
@@ -0,0 +1,99 @@
+## Context
+
+Metamodel creation performs demand-driven, nested specification introspection.
+When command-log recording support is enabled, `SynthesizeNavigationActionsPostProcessor` streams associations including mixed-in members, which can fully introspect mixin specifications.
+Other postprocessors, notably `DescribedAsFromTypePostProcessor`, can then resolve action element types and trigger further specification loads.
+
+The observed failure repeats this path until the JVM stack is exhausted, but the exception contains method names rather than the domain and mixin classes being traversed.
+In addition, `CausewayBeanTypeRegistryDefault` stores mixin metadata in a `HashMap`, and `streamMixinTypes()` exposes that unspecified iteration order to both mixed-in action and association discovery.
+This can change the nesting depth and the point of failure between otherwise equivalent JVM runs.
+
+The relevant CAUSEWAY-4039 regression tests cover a small cyclic graph but do not expose a long chain involving numerous mixins and action element-type post-processing.
+No metamodel classes on the failing path changed after `maintenance-branch`, so the immediate objective is to expose and stabilize the latent path before selecting a corrective redesign.
+
+## Goals / Non-Goals
+
+**Goals:**
+
+- Report the actual domain and mixin type chain involved when nested specification introspection overflows the JVM stack.
+- Include enough state and caller information to distinguish navigation synthesis, mixed-in member discovery, and action element-type resolution.
+- Make mixin traversal repeatable for a fixed set of registered types.
+- Add focused regression coverage for diagnostics, ordering, and the suspected cross-postprocessor path.
+
+**Non-Goals:**
+
+- Redesign metamodel introspection or eliminate all recursive loading in this change.
+- Change mixin applicability, member ordering, or the public domain programming model.
+- Introduce a general-purpose tracing subsystem or a new dependency.
+- Treat a larger JVM stack as a correction for the underlying dependency chain.
+
+## Decisions
+
+### Maintain a thread-local introspection chain at the specification-loader boundary
+
+`SpecificationLoaderDefault` is the common boundary through which nested type loads pass, so diagnostics will maintain a per-thread stack around `_loadSpecification` or the narrowest equivalent internal method.
+Each entry will contain the requested Java type, requested introspection state, current specification state when available, nesting depth, and a compact caller identifier.
+The scope will be removed in a `finally` block during normal execution.
+
+This captures the type information absent from a Java stack trace while preserving isolation between the concurrent metamodel bootstrap tasks.
+A global stack was rejected because concurrent task output would be interleaved and misleading.
+Adding type logging independently at every postprocessor was rejected because it would duplicate instrumentation and still leave gaps between callers.
+
+### Emit the retained chain when a stack overflow crosses the loader boundary
+
+The innermost loader boundary that observes a `StackOverflowError` will emit the chain once and rethrow the original error unchanged.
+A per-thread reported flag will suppress duplicate reports as the same error unwinds through outer loader calls.
+The report will use compact, depth-ordered lines so it remains usable even for a long chain.
+
+Optional trace-level entry and exit messages may reuse the same data, but normal successful startup will not produce diagnostic noise at standard log levels.
+Catching and replacing the error was rejected because it would obscure the original failure and could require unsafe allocation while the stack is exhausted.
+Logging only at the outer task-list boundary was rejected because nested loader scopes have already unwound there.
+
+### Derive caller information without changing public loader APIs
+
+The diagnostic entry will identify the nearest relevant framework caller, using a small internal helper and Java's built-in stack walking facilities or an equivalently bounded internal mechanism.
+Framework plumbing frames from the specification loader itself will be skipped.
+This avoids propagating a diagnostic-reason parameter through public and widely used `SpecificationLoader` methods.
+
+Explicit reason parameters at every call site would provide stronger semantics, but they create a broader, riskier change in a decommissioning branch.
+If caller derivation proves too costly or unreliable, the minimum acceptable fallback is the type and introspection-state chain, because that is sufficient to reconstruct the domain dependency path.
+
+### Sort mixin types by stable class name at the registry stream boundary
+
+`CausewayBeanTypeRegistry.streamMixinTypes()` will return mixin classes ordered by their fully qualified class name.
+Centralizing the ordering there applies consistently to mixed-in actions and mixed-in associations and leaves the underlying lookup maps unchanged.
+
+Replacing all registry maps with sorted maps was rejected because only mixin traversal is implicated and broader map-order changes could affect unrelated bootstrap behavior.
+Sorting separately in each metamodel consumer was rejected because duplicated ordering logic could drift.
+
+### Test the combined path rather than only a two-node collection cycle
+
+Tests will cover stable class-name ordering directly and will repeat metamodel construction to verify a stable traversal sequence.
+A full-boot or suitably production-equivalent regression fixture will combine multiple mixins, mixed-in associations, navigation-action synthesis, and actions whose element types trigger further specification introspection.
+The fixture will verify that the diagnostic chain identifies participating types if overflow is deliberately reproduced, or otherwise verify the captured nested sequence through a test seam.
+
+The test need not prescribe a permanent failing order.
+Its purpose is to prove observability and reproducibility before a follow-up change removes or flattens the problematic recursion.
+
+## Risks / Trade-offs
+
+- [Risk] Capturing caller information for every specification load adds startup overhead. → Keep the helper bounded, avoid full stack materialization, and measure or disable detailed caller capture unless diagnostics require it.
+- [Risk] Logging while handling `StackOverflowError` can itself fail because stack and allocation headroom are limited. → Keep the catch block shallow, retain compact pre-existing entries, emit once, and always rethrow the original error.
+- [Risk] Class-name ordering changes which member clash or validation issue is encountered first. → Preserve all existing filtering and member ordering semantics and test representative mixed-in actions and associations.
+- [Risk] A deterministic order may make the application fail consistently rather than intermittently. → This is an intentional diagnostic outcome; rollback is the isolated ordering change, while the emitted chain supplies the evidence for the corrective follow-up.
+- [Risk] The representative fixture may still be smaller than the production graph. → Include the interaction between all three observed triggers and allow production logs to provide the exact chain.
+
+## Migration Plan
+
+No data or configuration migration is required.
+Deploy the diagnostic and ordering changes together to an environment that has exhibited the intermittent failure, retaining the existing command-log recording-support setting and JVM stack size.
+Compare repeated starts and collect the emitted type chain if a failure occurs.
+Rollback consists of reverting the change; no persisted state is altered.
+Use the resulting chain to decide whether a follow-up should defer navigation synthesis, avoid fully introspecting inapplicable mixins, or otherwise flatten the dependency traversal.
+
+## Open Questions
+
+- What JVM stack size was used by the failing process, and does it match the integration-test runtime?
+- Should successful nested-chain tracing be available only through the existing metamodel logger, or also through a dedicated diagnostic logger category?
+- Is fully qualified class-name ordering sufficient in environments that load same-named classes through different class loaders?
+- Which production domain and mixin types form the failing chain once diagnostics are available?
diff --git a/openspec/changes/diagnose-metamodel-introspection-recursion/proposal.md b/openspec/changes/diagnose-metamodel-introspection-recursion/proposal.md
new file mode 100644
index 00000000000..695aa858804
--- /dev/null
+++ b/openspec/changes/diagnose-metamodel-introspection-recursion/proposal.md
@@ -0,0 +1,28 @@
+## Why
+
+Metamodel creation can intermittently fail with a `StackOverflowError` while navigation-action synthesis, mixed-in association discovery, and action element-type post-processing recursively trigger full introspection of further specifications.
+The current stack trace does not identify the domain and mixin types in the nested chain, while `HashMap`-backed mixin iteration makes the traversal order vary between JVM runs and prevents reliable diagnosis.
+
+## What Changes
+
+- Add focused metamodel introspection diagnostics that record the specification type, requested introspection state, current introspection state, and the nested reason or caller responsible for loading it.
+- Ensure diagnostics make the active cross-specification introspection chain visible when metamodel creation fails without enabling excessively broad framework logging.
+- Traverse registered mixin types in a deterministic class-name order so equivalent application inputs produce a repeatable metamodel traversal order.
+- Add regression coverage proving deterministic mixin traversal and demonstrating the recursive chain with a representative combination of mixed-in associations, navigation-action synthesis, and action element-type post-processing.
+- Treat this change as a diagnostic and reproducibility improvement rather than claiming to remove every deep or cyclic introspection path.
+
+## Capabilities
+
+### New Capabilities
+
+- `metamodel-introspection-diagnostics`: Provides actionable nested type diagnostics and deterministic mixin traversal for investigating metamodel creation failures.
+
+### Modified Capabilities
+
+None.
+
+## Impact
+
+The change affects metamodel specification loading, mixed-in action and association discovery, diagnostic logging, and regression tests around command-log navigation-action synthesis.
+It introduces no new external dependency and no intended public API or domain-programming-model change.
+Mixin processing order becomes stable by class name, which may expose code that accidentally depends on the previous unspecified `HashMap` iteration order.
diff --git a/openspec/changes/diagnose-metamodel-introspection-recursion/specs/metamodel-introspection-diagnostics/spec.md b/openspec/changes/diagnose-metamodel-introspection-recursion/specs/metamodel-introspection-diagnostics/spec.md
new file mode 100644
index 00000000000..ccd74ae243c
--- /dev/null
+++ b/openspec/changes/diagnose-metamodel-introspection-recursion/specs/metamodel-introspection-diagnostics/spec.md
@@ -0,0 +1,59 @@
+## ADDED Requirements
+
+### Requirement: Stack-overflow introspection chain
+
+When nested specification loading results in a `StackOverflowError`, the framework SHALL report the active metamodel introspection chain before propagating the original error.
+The report SHALL identify each requested Java type in nesting order and SHALL include its requested introspection state and current introspection state when available.
+The report SHALL include compact caller information sufficient to distinguish relevant metamodel loading paths when that information can be obtained safely.
+
+#### Scenario: Nested specification loading exhausts the JVM stack
+
+- **WHEN** metamodel creation encounters a `StackOverflowError` during nested specification loading
+- **THEN** one diagnostic report lists the active specification types in nesting order
+- **THEN** each listed entry includes the requested introspection state and any available current state
+- **THEN** the original `StackOverflowError` continues through the existing failure handling
+
+#### Scenario: The same error unwinds through multiple loader invocations
+
+- **WHEN** one `StackOverflowError` crosses multiple nested specification-loader boundaries
+- **THEN** the framework emits no more than one introspection-chain report for that error on that thread
+
+#### Scenario: Metamodel creation succeeds
+
+- **WHEN** metamodel creation completes without a stack overflow
+- **THEN** no stack-overflow introspection-chain report is emitted at standard log levels
+
+### Requirement: Thread-isolated diagnostics
+
+The framework SHALL track nested introspection independently for each metamodel bootstrap thread.
+
+#### Scenario: Concurrent tasks load specifications
+
+- **WHEN** two metamodel bootstrap threads perform nested specification loads concurrently
+- **THEN** each diagnostic chain contains only entries created by its own thread
+
+### Requirement: Deterministic mixin traversal
+
+The framework SHALL traverse registered mixin types in ascending fully qualified class-name order whenever the metamodel discovers mixed-in actions or associations.
+The deterministic traversal SHALL NOT change mixin applicability or the framework's explicit member-ordering rules.
+
+#### Scenario: Registry insertion order differs
+
+- **WHEN** equivalent registries contain the same mixin classes inserted in different orders
+- **THEN** `streamMixinTypes()` returns the same ascending fully qualified class-name sequence for both registries
+
+#### Scenario: Mixed-in actions and associations are discovered
+
+- **WHEN** metamodel introspection discovers mixed-in actions and mixed-in associations from the registry
+- **THEN** both discovery paths consume the same deterministic mixin sequence
+- **THEN** the resulting members retain their existing applicability and explicit ordering semantics
+
+### Requirement: Representative recursion regression coverage
+
+The test suite SHALL exercise the combined metamodel path involving navigation-action synthesis, mixed-in association discovery, and action element-type introspection across multiple specifications.
+
+#### Scenario: Representative nested graph is introspected repeatedly
+
+- **WHEN** the representative metamodel fixture is bootstrapped repeatedly with command-log recording support enabled
+- **THEN** each run follows the same mixin traversal order
+- **THEN** the diagnostic mechanism can identify the participating specification types in their nested order
diff --git a/openspec/changes/diagnose-metamodel-introspection-recursion/tasks.md b/openspec/changes/diagnose-metamodel-introspection-recursion/tasks.md
new file mode 100644
index 00000000000..a8fb9e56c96
--- /dev/null
+++ b/openspec/changes/diagnose-metamodel-introspection-recursion/tasks.md
@@ -0,0 +1,25 @@
+## 1. Characterize the recursive path
+
+- [ ] 1.1 Add a representative fixture with multiple mixins, mixed-in associations, navigation-action synthesis, and action element types that trigger nested specification loading.
+- [ ] 1.2 Add a test seam that records the nested specification types and reasons without depending on a production `StackOverflowError` for ordinary assertions.
+- [ ] 1.3 Confirm the fixture exercises the same key framework methods seen in the reported stack trace.
+
+## 2. Deterministic mixin traversal
+
+- [ ] 2.1 Change the central mixin-type stream to order classes by fully qualified class name without changing the registry's lookup maps.
+- [ ] 2.2 Add unit coverage proving identical traversal for registries populated in different orders.
+- [ ] 2.3 Add coverage proving mixed-in action and association discovery retain existing applicability and explicit member-order behavior.
+
+## 3. Introspection-chain diagnostics
+
+- [ ] 3.1 Add an internal thread-local diagnostic scope around nested specification loading with type, requested state, available current state, depth, and compact caller information.
+- [ ] 3.2 Report the retained chain once when a `StackOverflowError` crosses the loader boundary, then rethrow the original error unchanged.
+- [ ] 3.3 Add tests for ordered chain content, per-thread isolation, cleanup after successful loading, and duplicate-report suppression while an error unwinds.
+- [ ] 3.4 Verify successful metamodel creation produces no new diagnostics at standard log levels.
+
+## 4. Regression and operational validation
+
+- [ ] 4.1 Extend full-boot command-recording regression coverage with the representative nested graph and verify repeated boots use the same mixin traversal order.
+- [ ] 4.2 Run targeted metamodel and recording-navigation tests using the production JVM stack-size setting.
+- [ ] 4.3 Exercise the previously failing application or closest available fixture, capture the reported type chain, and document whether deterministic ordering makes the outcome repeatable.
+- [ ] 4.4 Record follow-up recommendations for flattening or removing the identified recursive path without expanding this diagnostic change into the final correction.