(causeway) 07/22: CAUSEWAY-3910: plan command result metadata reconciliation

[da...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

danhaywood pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/causeway.git

commit 2d386a8c4db939e38143319853e68652058b4a9e
Author: Dan Haywood <da...@haywood-associates.co.uk>
AuthorDate: Tue Aug 4 11:15:56 2026 +0100

    CAUSEWAY-3910: plan command result metadata reconciliation
---
 .../.openspec.yaml                                 |   2 +
 .../reconcile-command-result-metadata/design.md    |  95 +++++++++++++++
 .../reconcile-command-result-metadata/proposal.md  |  32 ++++++
 .../specs/command-result-metadata/spec.md          | 128 +++++++++++++++++++++
 .../reconcile-command-result-metadata/tasks.md     |  21 ++++
 5 files changed, 278 insertions(+)

diff --git a/openspec/changes/reconcile-command-result-metadata/.openspec.yaml b/openspec/changes/reconcile-command-result-metadata/.openspec.yaml
new file mode 100644
index 00000000000..1b062d3a7c7
--- /dev/null
+++ b/openspec/changes/reconcile-command-result-metadata/.openspec.yaml
@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-08-04
diff --git a/openspec/changes/reconcile-command-result-metadata/design.md b/openspec/changes/reconcile-command-result-metadata/design.md
new file mode 100644
index 00000000000..a3ff3e3c8e1
--- /dev/null
+++ b/openspec/changes/reconcile-command-result-metadata/design.md
@@ -0,0 +1,95 @@
+## Context
+
+The maintenance reconciliation graph identifies C4a result capture and D1 portable result metadata as the next foundations after C1.
+Causeway 4 currently records a command result only when the direct action result is persistable, and it ignores bookmarkable view models and single-element result containers.
+`CommandDtoUtils` currently supports the Causeway 4 legacy `CommandDto` YAML APIs but has no result-bearing transfer envelope, bookmark DTO, imported-command carrier, or deep-copy operation.
+The maintenance implementation is authoritative for observable semantics, while the implementation must retain Causeway 4's current executor construction, Jakarta APIs, `CommandDtoJacksonSupport`, and YAML utility conventions.
+
+## Goals / Non-Goals
+
+**Goals:**
+
+- Capture one bookmarkable direct action result without restricting it to persistent entities.
+- Capture the contained bookmarkable object from a framework-supported result container only when the container has exactly one non-empty element.
+- Preserve an existing command result and leave ambiguous or non-bookmarkable results unset.
+- Add public result-bearing transfer values compatible with maintenance's `command`, `result`, `type`, and `id` shape.
+- Add independent deep-copy support for `CommandDto`.
+- Serialize result-bearing command values as multi-document YAML without changing existing Causeway 4 `CommandDto` YAML methods.
+- Establish contracts and tests that later replay, projection, reachability, and export/import changes can consume.
+
+**Non-Goals:**
+
+- Enabling command publishing for safe actions or property edits.
+- Synthesizing or executing navigation actions.
+- Importing YAML into commandlog entries or changing command manager workflows.
+- Implementing replay input/result mapping.
+- Enriching `ReplayableCommand` or introducing the unified `CommandManager`.
+- Validating whether later action targets are reachable from earlier results.
+
+## Decisions
+
+### Capture every bookmarkable single result
+
+Result capture will use the adapted result's bookmark as the capability boundary rather than requiring a persistable entity.
+This includes bookmarkable view models because recording should preserve available identity metadata while later replayability and reachability policies decide whether that metadata is usable.
+Persistable results that do not yet have an identifier will retain the existing transaction-flush behavior before bookmark lookup.
+
+Restricting capture to entities was rejected because it loses maintenance-compatible view-model results and prevents later mapping from observing them.
+Introducing a new replay-stability marker at this layer was rejected because no such maintenance contract exists and it would couple neutral recording to downstream replay policy.
+
+### Normalize singleton containers during result capture
+
+A direct result is a candidate as-is.
+A framework-packed result container is a candidate only when it contains exactly one non-empty element.
+Empty containers, multi-element containers, and singleton elements without bookmarks produce no command result.
+The command stores only the contained bookmark and does not preserve the original container shape.
+
+General collection serialization was rejected because the downstream command contract represents one result bookmark and treating one member of a multi-result collection as authoritative would be ambiguous.
+
+### Preserve the first command result
+
+Result capture will not overwrite a result already held by the command.
+This preserves existing nested invocation and mixin behavior and matches the maintenance guard against trampling an earlier result.
+
+### Keep transfer values with `CommandDtoUtils`
+
+`CommandExportDto`, `ImportedCommandDto`, and `BookmarkDto` will be public static value types associated with `CommandDtoUtils`, matching the maintenance API location and avoiding a new package-level abstraction before downstream consumers exist.
+The transfer types will remain mutable JavaBean-style values so Causeway 4's Jackson/JAXB integration can construct them without custom creators.
+`CommandExportDto` will contain an embedded `CommandDto` and nullable `BookmarkDto result`.
+`ImportedCommandDto` will contain an embedded `CommandDto` and nullable applib `Bookmark result` for later import consumers.
+`BookmarkDto` will convert between bookmark identity and YAML fields `type` and `id` without resolving a domain object.
+Unknown YAML properties will be ignored on the export envelope, but the legacy `returnedObject` name will not be introduced as an alias.
+
+Creating a new schema version was rejected because the metadata is an export envelope around the existing command schema rather than a change to `CommandDto` itself.
+
+### Add export YAML without replacing legacy YAML APIs
+
+A dedicated result-bearing export method will emit one `CommandExportDto` per YAML document and omit null fields.
+It will reuse Causeway 4's existing `CommandDtoJacksonSupport` and YAML utilities rather than porting maintenance's older Jackson customization internals.
+Existing `toYaml`, `toMultiDocYaml`, and `fromYaml` semantics for plain `CommandDto` values will remain unchanged.
+Actual replay-import fallback, list rejection, persistence, and manager integration remain E1 work.
+
+### Deep-copy through the established command schema mapping
+
+`CommandDtoUtils.copy(CommandDto)` will create a structurally independent command DTO through the established JAXB command schema mapping and return null for null input.
+Manual field copying was rejected because it is brittle as the generated schema evolves.
+JSON or YAML copying was rejected because XML/JAXB is already the authoritative full-fidelity mapping for `CommandDto`.
+
+## Risks / Trade-offs
+
+- [Capturing any bookmarkable view model may retain identities that later prove unsuitable for replay] → Keep capture neutral and require later P1/R2 policies to decide replayability and reachability.
+- [Container handling depends on Causeway 4's packed managed-object representation] → Test empty, singleton, multiple, and non-bookmarkable cases at the runtime-service seam.
+- [New public nested DTOs become compatibility commitments] → Match maintenance names and field shapes and document the additions.
+- [Adding YAML support could accidentally alter legacy command YAML] → Add regression tests for the existing methods and introduce a separate export method.
+- [JAXB deep copying has per-call allocation cost] → Use it only where callers explicitly request an independent replay/execution DTO, not on every command invocation.
+
+## Migration Plan
+
+The change is additive except for broader result capture on commands that are already being published.
+Applications will begin retaining result bookmarks for bookmarkable view models and singleton result containers, while publishing eligibility remains unchanged.
+Rollback consists of removing the additive transfer/copy APIs and restoring entity-only result capture; no persistent schema migration is required.
+
+## Open Questions
+
+None.
+The programme question about view-model scope is resolved in favor of every bookmarkable single result, with replay stability deferred to downstream policy.
diff --git a/openspec/changes/reconcile-command-result-metadata/proposal.md b/openspec/changes/reconcile-command-result-metadata/proposal.md
new file mode 100644
index 00000000000..ca02771e8e9
--- /dev/null
+++ b/openspec/changes/reconcile-command-result-metadata/proposal.md
@@ -0,0 +1,32 @@
+## Why
+
+Causeway 4 records command results only for persistable scalar entities and its command YAML utilities expose only the legacy `CommandDto` shape.
+The maintenance line adds portable result bookmarks and broader result capture that later replay mapping, projections, validation, and export/import workflows require.
+
+## What Changes
+
+- Expand command result capture to retain the bookmark of a bookmarkable scalar entity or view model.
+- Treat a framework-supported result container containing exactly one bookmarkable object as the same single command result, while leaving empty, multi-object, and non-bookmarkable results unset.
+- Preserve an existing command result when later nested or mixin execution also returns a value.
+- Add result-bearing command transfer types for an embedded `CommandDto` and optional bookmark metadata represented as `type` and `id`.
+- Add deep-copy support for command DTOs so later replay processing can adapt execution input without mutating recorded command data.
+- Add multi-document YAML serialization support for result-bearing command transfer values while retaining the existing `CommandDto` YAML APIs.
+- Defer recording-aware publishing, synthetic navigation, replay mapping, replayable-command presentation, reachability validation, and validated export/import workflows to their dependency-ordered changes.
+
+## Capabilities
+
+### New Capabilities
+
+- `command-result-metadata`: Captures one bookmarkable command result and provides portable result-bearing command DTO, copy, and YAML foundations.
+
+### Modified Capabilities
+
+None.
+
+## Impact
+
+- `core/runtimeservices`: command result extraction during action execution, including singleton result containers and bookmarkable view models.
+- `api/applib`: public command DTO utility types, bookmark metadata conversion, deep-copy support, and YAML serialization foundations.
+- `extensions/core/commandlog/applib`: focused integration with command log result data where needed to verify the transfer contract without introducing later manager workflows.
+- Public API additions require API documentation and compatibility tests.
+- Existing command publishing eligibility and legacy `CommandDto` YAML behaviour remain unchanged.
diff --git a/openspec/changes/reconcile-command-result-metadata/specs/command-result-metadata/spec.md b/openspec/changes/reconcile-command-result-metadata/specs/command-result-metadata/spec.md
new file mode 100644
index 00000000000..6c987549160
--- /dev/null
+++ b/openspec/changes/reconcile-command-result-metadata/specs/command-result-metadata/spec.md
@@ -0,0 +1,128 @@
+## ADDED Requirements
+
+### Requirement: Commands capture a bookmarkable direct action result
+The system SHALL store the bookmark of a direct action result on the current command when the result is non-empty and bookmarkable.
+The system SHALL capture bookmarkable entity and view-model results without requiring the result type to be persistable.
+When a persistable result requires transaction synchronization before it has a bookmark, the system SHALL synchronize the transaction before re-evaluating its bookmark.
+The system MUST leave the command result unset when the direct result has no bookmark.
+The system MUST NOT replace a command result that was already captured.
+
+#### Scenario: Action returns a bookmarkable entity
+- **WHEN** a published action returns an entity with bookmark `demoCustomer:1`
+- **THEN** the current command result is bookmark `demoCustomer:1`
+
+#### Scenario: Action returns a bookmarkable view model
+- **WHEN** a published action returns a view model with bookmark `demoReport:weekly`
+- **THEN** the current command result is bookmark `demoReport:weekly`
+
+#### Scenario: Persistable result obtains an identifier during synchronization
+- **WHEN** a published action returns a persistable result that has no identifier before transaction synchronization
+- **AND** transaction synchronization assigns bookmark `demoCustomer:1`
+- **THEN** the current command result is bookmark `demoCustomer:1`
+
+#### Scenario: Direct result is not bookmarkable
+- **WHEN** a published action returns a non-empty result that has no bookmark
+- **THEN** the current command has no result bookmark from that return value
+
+#### Scenario: Earlier command result is preserved
+- **GIVEN** the current command already has result bookmark `demoCustomer:1`
+- **WHEN** a later nested or mixin invocation returns bookmark `demoCustomer:2`
+- **THEN** the current command result remains bookmark `demoCustomer:1`
+
+### Requirement: Commands normalize singleton result containers
+The system SHALL inspect framework-supported packed result containers when capturing a command result.
+When such a container contains exactly one non-empty bookmarkable object, the system SHALL store that object's bookmark as the command result.
+The system MUST leave the command result unset for an empty container, a container with more than one element, or a singleton element without a bookmark.
+The system MUST store only the contained object's bookmark and MUST NOT encode the original container shape in the command result.
+
+#### Scenario: Singleton list contains one bookmarkable object
+- **WHEN** a published action returns a list containing only bookmark `demoCustomer:1`
+- **THEN** the current command result is bookmark `demoCustomer:1`
+- **AND** the command result does not record that the action returned a list
+
+#### Scenario: Result container is empty
+- **WHEN** a published action returns an empty result container
+- **THEN** the current command has no result bookmark from that container
+
+#### Scenario: Result container has multiple elements
+- **WHEN** a published action returns a result container containing bookmarks `demoCustomer:1` and `demoCustomer:2`
+- **THEN** the current command has no result bookmark from that container
+
+#### Scenario: Singleton element is not bookmarkable
+- **WHEN** a published action returns a result container containing one non-bookmarkable object
+- **THEN** the current command has no result bookmark from that container
+
+### Requirement: Portable command exports carry optional result metadata
+The system SHALL provide a public `CommandExportDto` transfer value containing an embedded `CommandDto` and optional result metadata.
+The result metadata SHALL be represented by a public `BookmarkDto` with fields `type` and `id` derived from the bookmark logical type name and identifier.
+The system SHALL convert `BookmarkDto` back to a bookmark without requiring the bookmark to resolve to a live domain object.
+The system SHALL provide an `ImportedCommandDto` transfer value containing an embedded `CommandDto` and optional applib bookmark for later replay-import consumers.
+The transfer factories SHALL preserve a null result as absent result metadata.
+The `CommandExportDto` reader SHALL ignore unknown fields but MUST NOT treat the legacy `returnedObject` field as an alias for `result`.
+
+#### Scenario: Create command export with result
+- **WHEN** a command DTO is wrapped with result bookmark `demoCustomer:1`
+- **THEN** the command export contains the original command DTO
+- **AND** its result has type `demoCustomer` and id `1`
+
+#### Scenario: Create command export without result
+- **WHEN** a command DTO is wrapped without a result bookmark
+- **THEN** the command export contains the original command DTO
+- **AND** its result metadata is absent
+
+#### Scenario: Convert unresolved bookmark metadata
+- **WHEN** result metadata has type `remoteInvoice` and id `42`
+- **AND** no corresponding local domain object exists
+- **THEN** the metadata converts to bookmark `remoteInvoice:42` without resolving that object
+
+#### Scenario: Unknown legacy result field is not an alias
+- **WHEN** a command export document contains `returnedObject` but does not contain `result`
+- **THEN** the command export has no result metadata
+
+### Requirement: Result-bearing command exports use multi-document YAML
+The system SHALL serialize command export values as a multi-document YAML stream.
+Each YAML document SHALL contain the embedded command under field `command`.
+When a command export has result metadata, the YAML document SHALL contain it under field `result` with fields `type` and `id`.
+When a command export has no result metadata, the YAML document SHALL omit field `result`.
+The YAML output MUST NOT contain the legacy field `returnedObject`.
+Adding result-bearing YAML support MUST NOT change the existing plain `CommandDto` YAML APIs or their accepted formats.
+
+#### Scenario: Serialize command export with result
+- **WHEN** a command export has result bookmark `demoCustomer:1`
+- **THEN** its YAML document contains an embedded `command`
+- **AND** it contains `result` with type `demoCustomer` and id `1`
+- **AND** it does not contain `returnedObject`
+
+#### Scenario: Serialize command export without result
+- **WHEN** a command export has no result bookmark
+- **THEN** its YAML document contains an embedded `command`
+- **AND** it omits `result`
+- **AND** it does not contain `returnedObject`
+
+#### Scenario: Serialize multiple command exports
+- **WHEN** two command export values are serialized
+- **THEN** the output contains two YAML documents in input order
+
+#### Scenario: Legacy command YAML remains compatible
+- **WHEN** callers use the existing plain `CommandDto` YAML methods
+- **THEN** their list and multi-document behavior remains unchanged
+
+### Requirement: Command DTOs can be copied independently
+The system SHALL provide a deep-copy operation for `CommandDto`.
+The copied command DTO SHALL preserve the schema content of the original command DTO.
+The copied command DTO and its nested mutable values MUST be structurally independent from the original.
+The copy operation SHALL return null when supplied null.
+
+#### Scenario: Copy populated command DTO
+- **WHEN** a populated command DTO is copied
+- **THEN** the copy has the same serialized command schema content as the original
+- **AND** the copy is a different object from the original
+
+#### Scenario: Mutate copied command DTO
+- **GIVEN** a command DTO has been copied
+- **WHEN** a nested value in the copy is changed
+- **THEN** the corresponding value in the original command DTO is unchanged
+
+#### Scenario: Copy null command DTO
+- **WHEN** the copy operation is supplied null
+- **THEN** it returns null
diff --git a/openspec/changes/reconcile-command-result-metadata/tasks.md b/openspec/changes/reconcile-command-result-metadata/tasks.md
new file mode 100644
index 00000000000..07ffdbedfc9
--- /dev/null
+++ b/openspec/changes/reconcile-command-result-metadata/tasks.md
@@ -0,0 +1,21 @@
+## 1. Portable command result transfer contract
+
+- [ ] 1.1 Add public `CommandExportDto`, `ImportedCommandDto`, and `BookmarkDto` value types to `CommandDtoUtils`, including null-safe factories and bookmark conversion without object resolution.
+- [ ] 1.2 Add `CommandDtoUtils.copy(CommandDto)` using full-fidelity command-schema mapping and ensure copied nested values are independent.
+- [ ] 1.3 Add multi-document YAML serialization for `CommandExportDto` values using Causeway 4 `CommandDtoJacksonSupport`, omitting null results and never emitting `returnedObject`.
+- [ ] 1.4 Preserve the existing plain `CommandDto` list, multi-document, and read APIs without changing their accepted formats.
+- [ ] 1.5 Add focused applib/mmtest coverage for transfer factories, unresolved bookmarks, unknown legacy fields, YAML output and ordering, null omission, deep copying, and legacy YAML regressions.
+
+## 2. Runtime command result capture
+
+- [ ] 2.1 Refactor `MemberExecutorServiceDefault` result candidate selection to accept direct results and exactly-one-element packed result containers.
+- [ ] 2.2 Capture bookmarks for bookmarkable entities and view models while retaining transaction synchronization for persistable results that need identifiers.
+- [ ] 2.3 Preserve an existing command result and leave empty, multi-element, null, unspecified, and non-bookmarkable results unset.
+- [ ] 2.4 Add focused runtime-service tests for direct entities, view models, transient persistable results, singleton containers, empty and multiple containers, non-bookmarkable values, and existing-result preservation.
+
+## 3. Integration, documentation, and verification
+
+- [ ] 3.1 Verify commandlog consumers can construct result-bearing transfer values from recorded `CommandDto` and result bookmarks without introducing manager export/import or replay behavior.
+- [ ] 3.2 Document the new public command result transfer and copy APIs, including the `result.type` and `result.id` YAML shape and the exclusion of `returnedObject`.
+- [ ] 3.3 Run focused tests for applib/mmtest, runtime services, and commandlog, then run the affected aggregate Maven build with JDK 21.
+- [ ] 3.4 Run IDE compilation/inspection and repository checks, and confirm the implementation does not include C2, C3/C4b, M1-M3, P1/P2, R1/R2, E1, W1, or B1/B2 behavior.