(groovy-website) 02/05: GEP-27: update with prototype findings; remove outdated framing

[pa...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

paulk-asert pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/groovy-website.git

commit 1ca316812363c3fb630b636387c3c30b0e66273e
Author: Paul King <pa...@asert.com.au>
AuthorDate: Fri Jul 10 12:52:38 2026 +1000

    GEP-27: update with prototype findings; remove outdated framing
    
    Fold in the validated spike results and reconcile stale sections:
    - Reference implementation: PackedClosure runtime guard and compile-time escape
      decline are implemented+validated (every delegate-redirect funnels through the
      two setters); add measured impact (FormTagLib 8->1 classes, ~90% of closure
      bytes are scaffolding; RemarkCriteria domain 24 classes/80KB); document the
      Grails artefact-DSL safety and the unsound naive nested-validator hoist (needs
      owner-correct retargeting = Groovy 7).
    - Piece 2 (read-only-capturing SAM lambdas) is implemented, not a stretch; phase
      table and abstract updated accordingly.
    - Guarding section: resolveStrategy guard covers all non-owner modes (incl.
      TO_SELF); escape gate described as implemented syntactic form.
    - Bump last-modified date.
---
 site/src/site/wiki/GEP-27.adoc | 162 ++++++++++++++++++++++++++++++++++-------
 1 file changed, 134 insertions(+), 28 deletions(-)

diff --git a/site/src/site/wiki/GEP-27.adoc b/site/src/site/wiki/GEP-27.adoc
index b7cca8f..7e38eac 100644
--- a/site/src/site/wiki/GEP-27.adoc
+++ b/site/src/site/wiki/GEP-27.adoc
@@ -14,7 +14,7 @@
 *Comment*:: Reduces the per-closure/per-lambda generated-class explosion by hoisting bodies onto the enclosing class. A small, sound-by-construction slice potentially targets Groovy 6; the broader closure work targets Groovy 7. Strictly additive; gated by a flag for back-out.
 *Leader*:: Paul King
 *Created*:: 2026-07-09
-*Last modification*:: 2026-07-09
+*Last modification*:: 2026-07-10
 ****
 
 [width="80%",align="center"]
@@ -52,8 +52,8 @@ actually needs:
 * where a value neither escapes nor needs closure identity, the object can be elided entirely.
 
 The proposal is deliberately phased. A **small, sound-by-construction slice — eliminating the
-generated class for non-capturing SAM lambdas — targets Groovy 6.0**. The **broader closure work,
-which needs a soundness analysis and a performance path, targets Groovy 7.0**. The change is
+generated class for non-capturing and read-only-capturing SAM lambdas — targets Groovy 6.0**. The
+**broader closure work, which needs a soundness analysis and a performance path, targets Groovy 7.0**. The change is
 strictly additive, defaults can be flipped via a flag for back-out, and it is gated so that any
 tooling that reflects on generated-class names has an escape hatch.
 
@@ -293,10 +293,59 @@ resolution — a `Function` has no `delegate`. And a SAM-typed lambda is, by its
 that SAM. So the entire soundness cliff is defined away for the SAM case, which is precisely why
 it can land first.
 
+=== Guarding the dynamic trust assertion
+
+Under `@CompileStatic` the escape analysis _proves_ delegate-independence and declines the rest,
+so the boundary is enforced by construction. Under dynamic compilation `@Packed` is a *trust
+assertion* the compiler cannot verify — so the design must protect a user who annotates in good
+faith and gets it wrong. Two mechanisms do this, and they deliberately cover *different* failure
+modes:
+
+* *Runtime guard (the safety net).* A `PackedClosure` throws — with an actionable message naming
+  the hoisted method — when asked for a configuration it cannot honour: `setDelegate(x)` to anything
+  other than the owner, or `setResolveStrategy(...)` to any non-owner mode (`DELEGATE_FIRST`,
+  `DELEGATE_ONLY`, or `TO_SELF` — which would resolve against the memberless shared adapter). This
+  is *fail-fast at the exact point the wrong assumption is exercised*,
+  rather than the silent alternative (free names quietly resolve against the owner, surfacing as a
+  confusing `MissingMethodException` frames away). Almost all delegate-dependent usage funnels
+  through these setters — explicit assignment, `with`/`tap` (which set `delegate` +
+  `DELEGATE_FIRST` internally), and builder DSLs — so the guard catches the realistic paths.
+  Setting `delegate` to the owner stays a no-op, so benign/defensive framework code is unaffected.
+  There is no runtime de-opt option: the hoisted body is already compiled to owner-bound bytecode,
+  so throwing is the only honest response. In the `@CompileStatic` case the guard should never fire
+  (those closures are proven and declined), but keeping it is cheap defense-in-depth and documents
+  the contract uniformly.
+* *`@Packed(strict = true)` (compile-time visibility, *not* a safety net).* Errors at compile time
+  if any closure in scope cannot be packed, so a decline is reported rather than silently yielding
+  a class. It surfaces the *visible* declines the compiler already knows about (local
+  `delegate`/`resolveStrategy` use, serialization, syntactic escape). It deliberately does *not*
+  protect against the wrong-assumption case above — an externally-set delegate is invisible at the
+  closure's definition site (that is the very reason dynamic packing needs a trust assertion), so
+  `strict` will report "packed" on exactly the closure the runtime guard later has to catch.
+
+[cols="1,1,3",options="header"]
+|===
+|Mechanism |When |Catches
+|`@Packed(strict = true)` |compile time |_visible_ declines — "did I pack what I intended?"
+|`PackedClosure` throws on `setDelegate`/`setResolveStrategy` |runtime |the _invisible_ wrong assumption — "did anyone misuse what I packed?"
+|===
+
+A conservative *escape gate* shrinks the runtime surface further and is implemented in the spike: a
+closure literal that visibly escapes its method — stored into a field/property/index (the Grails
+`static constraints = { … }` shape), returned, appended with `<<`, or placed in a collection literal
+— is declined at compile time and kept as a class. This moves the clearest delegate-risk cases from
+a runtime throw to a compile-time non-event; those declines are also what `strict` would report.
+Because syntactic escape analysis cannot prove that, say, a custom `each` does not set a delegate,
+the runtime guard remains the essential backstop under dynamic compilation. (A closure that is
+fully self-contained — every free name binds to the owner or its own parameters — is
+delegate-independent and safe to pack even where it escapes; exploiting that for nested closures
+needs owner-correct retargeting, see _Reference implementation_.)
+
 == Groovy 6.0 deliverables (pending discussion and findings)
 
-The Groovy 6 target is the sound-by-construction slice. One piece is proposed as the primary
-deliverable, with a second offered as an option.
+The Groovy 6 target is the sound-by-construction slice: eliminating the generated class for SAM
+lambdas. Both pieces below are implemented on the first-step branch behind a single flag; the
+mutated-capture and serializable cases correctly stay on the class-based path.
 
 === Piece 1 (primary): eliminate the class for non-capturing SAM lambdas
 
@@ -329,14 +378,16 @@ method onto the enclosing class). It requires no change to the `invokedynamic`/m
 bootstrap other than the implementation method's owner class, and does not require the
 class-visitor changes the closure work needs.
 
-=== Piece 2 (option): capturing SAM lambdas
+=== Piece 2 (implemented): read-only-capturing SAM lambdas
 
 A capturing SAM lambda today instantiates its lambda class to hold the captured values (Groovy
-even `Reference`-wraps them). The Java-style form uses a `static` implementation method taking
-the captured values as leading arguments, with the metafactory capturing them directly — which
-_also_ removes the `Closure` allocation and the `Reference` wrapping. This is a larger change
-than Piece 1 (the impl signature and bootstrap arguments change), so it is offered as a Groovy 6
-*stretch* or an early Groovy 7 item rather than a commitment.
+even `Reference`-wraps them). The Java-style form uses a `static` implementation method taking the
+captured values as leading arguments, with the metafactory capturing them directly — which _also_
+removes the `Closure` allocation and the `Reference` wrapping. This is implemented on the same
+first-step branch for the *read-only* capture case (the common one): captured values are passed as
+typed leading parameters. A lambda that *mutates* a capture still needs the shared `Reference`, so
+it correctly declines and keeps its class. This lands the bulk of the capturing benefit in Groovy 6
+behind the same flag; only mutated-capture lambdas remain on the class-based path.
 
 == Groovy 7.0 deliverables
 
@@ -349,9 +400,14 @@ closures rather than lambdas. A proof-of-concept in the code generator already d
 mechanism across the hard cases: no-capture and captured closures (by value; written captures via
 shared `Reference`, including `+=`/`++`), *arbitrary-depth nested closures*, implicit `it`, under
 both `@CompileStatic` and dynamic compilation, correctly declining closures that use
-`delegate`/`owner`/`super`. What remains for a real feature is the *capability/escape analysis*
-described above, so packing is chosen only when it is provably sound (automatic under
-`@CompileStatic`; opt-in via `@Packed` under dynamic), falling back to S0 otherwise.
+`delegate`/`owner`/`super`. The dynamic opt-in's two runtime/compile safety mechanisms from
+_Guarding the dynamic trust assertion_ are already implemented and validated in the spike: the
+mandatory `PackedClosure` runtime guard and the compile-time escape decline (the optional
+`@Packed(strict = true)` enforcement is designed but not yet wired). What remains for a real feature
+is the *capability/escape analysis* that makes packing provably sound and therefore automatic under
+`@CompileStatic` (no annotation), falling back to S0 otherwise; and the owner-correct retargeting
+that lets self-contained nested closures (e.g. Grails `validator` blocks) be hoisted — see
+_Reference implementation_.
 
 === Typed static dispatch (making it fast, not just small)
 
@@ -388,6 +444,11 @@ means byte-for-byte today's behaviour.
 |Closure packing under dynamic compilation |`@Packed` opt-in only |off unless annotated
 |===
 
+Under dynamic compilation `@Packed` carries an optional `strict` element
+(`@Packed(strict = true)`) that turns any *decline* in scope into a compile error, and every
+packed closure is backed by the `PackedClosure` runtime guard; see _Guarding the dynamic trust
+assertion_ for what each does and does not catch.
+
 The default *direction* is a policy decision independent of the mechanism: a feature can ship
 *on* with an opt-out flag (deliver the win immediately, keep a back-out valve), or *off* with an
 opt-in flag and be flipped a release later (conservative). The same one-line gate serves either
@@ -483,19 +544,64 @@ development:
   first step sits behind the `groovy.target.lambda.hoist` flag. The existing lambda tests that
   assert the old bytecode shape need updating to the new structure (behaviour is unchanged); that
   test migration is the bulk of the remaining first-step work.
-* *Closure packing (Groovy 7 direction).* A `ClosureWriter`-based spike (with a `PackedClosure`
-  runtime adapter) demonstrates hoisting closures — captures, arbitrary-depth nesting, written
-  captures via `Reference`, implicit `it` — under both compilation modes, with correct declines
-  for delegate/owner/super. It confirms the writer is the right home (no `@CompileStatic`
-  boundary, unlike an AST-transform approach) and empirically establishes the soundness cliff and
-  what is preserved. It does *not* yet include the capability/escape analysis or typed dispatch;
-  those are the Groovy 7 work.
-
-One supporting change surfaced by the closure spike is worth noting: hoisting nested closures
+* *Closure packing (Groovy 7 direction).* A `ClosureWriter`-based spike (with a shared
+  `PackedClosure` runtime adapter), opt-in via `@Packed`, demonstrates hoisting closures — captures,
+  arbitrary-depth nesting, written captures via `Reference`, implicit `it` — under both compilation
+  modes. It confirms the writer is the right home (no `@CompileStatic` boundary, unlike an
+  AST-transform approach), establishes the soundness cliff empirically, and confirms what is
+  preserved (`curry`/`memoize`/`trampoline` all keep working on the adapter). Two of the three
+  safety layers from _Guarding the dynamic trust assertion_ are now implemented and validated:
+** the *runtime guard* — `PackedClosure` rejects `setDelegate`/`setResolveStrategy` set to any
+   non-owner configuration. Probing confirmed every delegate-redirect path (`with`/`tap`,
+   `rehydrate`, builder DSLs, DGM) funnels through those two setters, so the guard turns what was a
+   silent wrong answer (owner-resolved where a delegate was expected) into a fail-fast error at the
+   point of misuse; owner-based `each`/`collect` closures stay byte-identical.
+** the *compile-time escape decline* — a closure literal that visibly escapes its method (stored
+   into a field/property/index, returned, appended with `<<`, or placed in a collection literal) is
+   kept as a class, moving the clearest delegate-risk cases from a runtime throw to a compile-time
+   non-event.
+
+The remaining Groovy 7 work is the *capability/escape analysis* for the `@CompileStatic` automatic
+path (proving packing sound so no annotation is needed) and typed static dispatch.
+
+=== Measured impact
+
+Closure classes are dominated by scaffolding, not logic. On a class reproducing seven real
+`FormTagLib` closure patterns, packing eliminated all seven closure classes (8 class files → 1;
+23.8 KB → 7.3 KB, ~69% fewer bytes); the hoisted method bodies added only 1.8 KB, so roughly *90%
+of a closure class's bytes were `Closure` scaffolding* — constructor, metaclass init,
+`serialVersionUID`, `Reference` fields — rather than the closure's own code. Grails domain classes
+are the extreme case: a representative `RemarkCriteria` domain generated *24 closure classes
+totalling 80 KB* — more bytecode than the entire rest of the class (1.19×).
+
+=== Grails artefact DSLs and nested closures
+
+The delegate DSLs Grails relies on — `static constraints = { … }`, `static mapping`,
+`namedQueries`, old-style `def action = { … }` — are field-assigned, which is exactly the escape
+gate's decline signal, so they are kept as real closure classes and their delegate resolution is
+unchanged (verified end-to-end against a builder delegate). Because Grails *discovers* these blocks
+by their being fields, the whole family lands on the safe side by construction rather than by
+special-casing; and the Groovy 6 SAM-lambda work never touches them at all (they are `Closure`s, not
+lambdas).
+
+The value left on the table in a domain class is the many *self-contained* nested closures — the
+`validator: { val, obj -> … }` blocks inside `constraints`, which reference only their own
+parameters and owner statics, not the constraints delegate. A spike that relaxed the nested-closure
+decline for these confirmed the class-count prize (the shown `RemarkCriteria` subset dropped
+6 → 2 closure classes) but proved the naive form *unsound*: the hoisted body was added to the
+enclosing closure class as an instance method, while a `static constraints` block's owner resolves
+to the domain *class*, so dispatch attempted a non-existent static method (`MissingMethodException`),
+and where it did dispatch, owner-static references resolved against the wrong owner. Hoisting a
+nested closure therefore requires *retargeting* the method onto the correct outermost owner,
+rebinding its free names to that owner, and emitting static-vs-instance dispatch to match — the
+owner-correct capability work already scoped to Groovy 7. This makes domain classes the single
+highest-value target for that phase, and the reason it is Groovy 7 and not a cheap gate change.
+
+A supporting change surfaced by the closure spike is worth noting: hoisting nested closures also
 requires the class visitor to visit methods added _during_ code generation to a fixpoint (the
-current two-round `visitMethods` misses methods added while compiling a method that was itself
-added during compilation). That change is general and low-risk but must be validated against the
-full suite before it lands.
+current two-round `visitMethods` misses methods added while compiling a method that was itself added
+during compilation). That change is general and low-risk but must be validated against the full
+suite before it lands.
 
 === Phased delivery
 
@@ -504,8 +610,8 @@ full suite before it lands.
 |Phase |What ships |What works |Target
 
 |1 |Non-capturing SAM lambda class elimination + flag + test migration + IDE/tooling check |Java-parity lambda codegen for the common SAM case; back-out flag |Groovy 6.0
-|2 |Capturing SAM lambdas (metafactory-captured static impl) |Removes the class, `Closure` alloc, and `Reference` wrapping for capturing SAM lambdas |Groovy 6.0 stretch / 7.0
-|3 |Closure packing (S1) + capability/escape analysis + `@Packed` |Sound closure packing: automatic under `@CompileStatic`, opt-in under dynamic |Groovy 7.0
+|2 |Read-only-capturing SAM lambdas (metafactory-captured static impl) |Removes the class, `Closure` alloc, and `Reference` wrapping for read-only capturing SAM lambdas (mutated captures decline) |Groovy 6.0
+|3 |Closure packing (S1) + capability/escape analysis + `@Packed` |Sound closure packing: automatic under `@CompileStatic`, opt-in (with runtime guard + escape decline) under dynamic |Groovy 7.0
 |4 |Typed static dispatch of hoisted bodies |Packed `@CompileStatic` closures are faster, not just smaller |Groovy 7.0
 |5 |Object elision (S3) + unify the writers |No object for non-escaping call-once bodies; one backend for closures and lambdas |Groovy 7.0
 |===