You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by wu...@apache.org on 2026/03/03 12:55:46 UTC
(skywalking) 01/27: Add Groovy replacement plan for MAL, LAL, and Hierarchy scripts
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch groovy-replace
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit 1710e48c508e5e7d0432fb7b431a2538b1444d66
Author: Wu Sheng <wu...@foxmail.com>
AuthorDate: Sat Feb 28 14:10:50 2026 +0800
Add Groovy replacement plan for MAL, LAL, and Hierarchy scripts
Document the detailed implementation plan for eliminating Groovy from
OAP runtime via build-time transpilers (MAL/LAL) and v1/v2 module
split (hierarchy), based on Discussion #13716 and skywalking-graalvm-distro.
Co-Authored-By: Claude Opus 4.6 <no...@anthropic.com>
---
docs/en/academy/groovy-replacement-plan.md | 912 +++++++++++++++++++++++++++++
1 file changed, 912 insertions(+)
diff --git a/docs/en/academy/groovy-replacement-plan.md b/docs/en/academy/groovy-replacement-plan.md
new file mode 100644
index 0000000000..1018615038
--- /dev/null
+++ b/docs/en/academy/groovy-replacement-plan.md
@@ -0,0 +1,912 @@
+# Groovy Replacement Plan: Build-Time Transpiler for MAL, LAL, and Hierarchy Scripts
+
+Reference: [Discussion #13716](https://github.com/apache/skywalking/discussions/13716)
+Reference Implementation: [skywalking-graalvm-distro](https://github.com/apache/skywalking-graalvm-distro)
+
+## 1. Background and Motivation
+
+SkyWalking OAP server currently uses Groovy as the runtime scripting engine for three subsystems:
+
+| Subsystem | YAML Files | Expressions | Groovy Pattern |
+|-----------|-----------|-------------|----------------|
+| MAL (Meter Analysis Language) | 71 (11 meter-analyzer-config, 55 otel-rules, 2 log-mal-rules, 2 envoy-metrics-rules, 1 telegraf-rules) | 1,254 metric + 29 filter | Dynamic Groovy: `propertyMissing()`, `ExpandoMetaClass` on `Number`, closures |
+| LAL (Log Analysis Language) | 8 | 10 rules | `@CompileStatic` Groovy: delegation-based closure DSL, safe navigation (`?.`), `as` casts |
+| Hierarchy Matching | 1 (hierarchy-definition.yml) | 4 rules | `GroovyShell.evaluate()` for `Closure<Boolean>` |
+
+### Problems with Groovy Runtime
+
+1. **Startup Cost**: 1,250+ `GroovyShell.parse()` calls at OAP boot, each spinning up the full Groovy compiler pipeline.
+2. **Runtime Errors Instead of Compile-Time Errors**: MAL uses dynamic Groovy -- typos in metric names or invalid method chains are only discovered when that specific expression runs with real data.
+3. **Debugging Complexity**: Stack traces include Groovy MOP internals (`CallSite`, `MetaClassImpl`, `ExpandoMetaClass`), obscuring the actual expression logic.
+4. **Runtime Execution Performance (Most Critical)**: MAL expressions execute on every metrics ingestion cycle. Per-expression overhead from dynamic Groovy compounds at scale:
+ - Property resolution: `CallSite` -> `MetaClassImpl.invokePropertyOrMissing()` -> `ExpressionDelegate.propertyMissing()` -> `ThreadLocal<Map>` lookup (4+ layers of indirection per metric name lookup)
+ - Method calls: Groovy `CallSite` dispatch with MetaClass lookup and MOP interception checks
+ - Arithmetic (`metric * 1000`): `ExpandoMetaClass` closure allocation + metaclass lookup + dynamic dispatch for what Java does as a single `imul`
+ - Per ingestion cycle: ~1,250 `propertyMissing()` calls, ~3,750 MOP method dispatches, ~29 metaclass arithmetic ops, ~200 closure allocations
+ - JIT cannot optimize Groovy's megamorphic call sites, defeating inlining and branch prediction
+5. **GraalVM Incompatibility**: `invokedynamic` bootstrapping and `ExpandoMetaClass` are fundamentally incompatible with AOT compilation.
+
+### Goal
+
+Eliminate Groovy from the OAP runtime entirely. Groovy becomes a **build-time-only** dependency used solely for AST parsing by the transpiler. Zero `GroovyShell`, zero `ExpandoMetaClass`, zero MOP at runtime.
+
+---
+
+## 2. Solution Architecture
+
+### Build-Time Transpiler: Groovy DSL -> Pure Java Source Code
+
+```
+BUILD TIME (Maven compile phase):
+ MAL YAML files (71 files, 1,250+ expressions)
+ LAL YAML files (8 files, 10 scripts)
+ |
+ v
+ MalToJavaTranspiler / LalToJavaTranspiler
+ (Groovy CompilationUnit at CONVERSION phase -- AST parsing only, no execution)
+ |
+ v
+ ~1,254 MalExpr_*.java + ~6 LalExpr_*.java + MalFilter_*.java
+ |
+ v
+ javax.tools.JavaCompiler -> .class files on classpath
+ META-INF/mal-expressions.txt (manifest)
+ META-INF/mal-filter-expressions.properties (manifest)
+ META-INF/lal-expressions.txt (manifest)
+
+RUNTIME (OAP Server):
+ Class.forName(className) -> MalExpression / LalExpression instance
+ Zero Groovy. Zero GroovyShell. Zero ExpandoMetaClass.
+```
+
+The transpiler approach is already fully implemented and validated in the [skywalking-graalvm-distro](https://github.com/apache/skywalking-graalvm-distro) repository.
+
+---
+
+## 3. Detailed Design
+
+### 3.1 New Functional Interfaces
+
+Three core interfaces replace Groovy's `DelegatingScript` and `Closure`:
+
+```java
+// MAL: replaces DelegatingScript + ExpandoMetaClass + ExpressionDelegate.propertyMissing()
+@FunctionalInterface
+public interface MalExpression {
+ SampleFamily run(Map<String, SampleFamily> samples);
+}
+
+// MAL: replaces Closure<Boolean> from GroovyShell.evaluate() for filter expressions
+@FunctionalInterface
+public interface MalFilter {
+ boolean test(Map<String, String> tags);
+}
+
+// LAL: replaces LALDelegatingScript + @CompileStatic closure DSL
+@FunctionalInterface
+public interface LalExpression {
+ void execute(FilterSpec filterSpec, Binding binding);
+}
+```
+
+### 3.2 SampleFamily: Closure -> Functional Interface
+
+Five `SampleFamily` methods currently accept `groovy.lang.Closure`. Each gets a new overload with a Java functional interface. During transition both overloads coexist; eventually the Closure overloads are removed.
+
+| Method | Current (Groovy) | New (Java) | Functional Interface |
+|--------|-----------------|------------|---------------------|
+| `tag()` | `Closure<?>` with `tags.key = val` | `TagFunction` | `Function<Map<String,String>, Map<String,String>>` |
+| `filter()` | `Closure<Boolean>` with `tags.x == 'y'` | `SampleFilter` | `Predicate<Map<String,String>>` |
+| `forEach()` | `Closure<Void>` with `(prefix, tags) -> ...` | `ForEachFunction` | `BiConsumer<String, Map<String,String>>` |
+| `decorate()` | `Closure<Void>` with `entity -> ...` | `DecorateFunction` | `Consumer<MeterEntity>` |
+| `instance(..., closure)` | `Closure<?>` with `tags -> Map.of(...)` | `PropertiesExtractor` | `Function<Map<String,String>, Map<String,String>>` |
+
+Source location in upstream: `oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/SampleFamily.java`
+
+### 3.3 MAL Transpiler: AST Mapping Rules
+
+The `MalToJavaTranspiler` (~1,230 lines) parses each MAL expression string into a Groovy AST at `Phases.CONVERSION` (no code execution), then walks the AST to emit equivalent Java code.
+
+#### Expression Mappings
+
+| Groovy Construct | Java Output | Notes |
+|-----------------|-------------|-------|
+| `metric_name` (bare property) | `samples.getOrDefault("metric_name", SampleFamily.EMPTY)` | Replaces `propertyMissing()` dispatch |
+| `.sum(['a','b'])` | `.sum(List.of("a", "b"))` | Direct method call |
+| `.tagEqual('resource', 'cpu')` | `.tagEqual("resource", "cpu")` | Direct method call |
+| `100 * metric` | `metric.multiply(100)` | Commutative: operands swapped |
+| `100 - metric` | `metric.minus(100).negative()` | Non-commutative: negate |
+| `100 / metric` | `metric.newValue(v -> 100 / v)` | Non-commutative: newValue |
+| `metricA / metricB` | `metricA.div(metricB)` | SampleFamily-SampleFamily op |
+| `.tag({tags -> tags.cluster = ...})` | `.tag(tags -> { tags.put("cluster", ...); return tags; })` | Closure -> lambda |
+| `.filter({tags -> tags.job_name in [...]})` | `.filter(tags -> "...".equals(tags.get("job_name")))` | Closure -> predicate |
+| `.forEach(['a','b'], {p, tags -> ...})` | `.forEach(List.of("a","b"), (p, tags) -> { ... })` | Closure -> BiConsumer |
+| `.decorate({entity -> ...})` | `.decorate(entity -> { ... })` | Closure -> Consumer |
+| `.instance(..., {tags -> Map.of(...)})` | `.instance(..., tags -> Map.of(...))` | Closure -> Function |
+| `Layer.K8S` | `Layer.K8S` | Enum constant, passed through |
+| `time()` | `Instant.now().getEpochSecond()` | Direct Java API |
+| `AVG`, `SUM`, etc. | `DownsamplingType.AVG`, etc. | Enum constant reference |
+
+#### Closure Body Translation
+
+Inside closures, Groovy property-style access is mapped to explicit `Map` operations:
+
+| Groovy Closure Pattern | Java Lambda Output |
+|----------------------|-------------------|
+| `tags.key = "value"` | `tags.put("key", "value")` |
+| `tags.key` (read) | `tags.get("key")` |
+| `tags.remove("key")` | `tags.remove("key")` |
+| `tags.key == "value"` | `"value".equals(tags.get("key"))` |
+| `tags.key != "value"` | `!"value".equals(tags.get("key"))` |
+| `tags.key in ["a","b"]` | `List.of("a","b").contains(tags.get("key"))` |
+| `if/else` in closure | `if/else` in lambda |
+| `entity.serviceId = val` | `entity.setServiceId(val)` |
+
+#### Filter Expression Mappings
+
+Filter expressions (`filter: "{ tags -> ... }"`) generate `MalFilter` implementations:
+
+| Groovy Filter | Java Output |
+|--------------|-------------|
+| `tags.job_name == 'mysql'` | `"mysql".equals(tags.get("job_name"))` |
+| `tags.job_name != 'test'` | `!"test".equals(tags.get("job_name"))` |
+| `tags.job_name in ['a','b']` | `List.of("a","b").contains(tags.get("job_name"))` |
+| `cond1 && cond2` | `cond1 && cond2` |
+| `cond1 \|\| cond2` | `cond1 \|\| cond2` |
+| `!cond` | `!cond` |
+| `tags.job_name` (truthiness) | `tags.get("job_name") != null` |
+
+### 3.4 LAL Transpiler: AST Mapping Rules
+
+The `LalToJavaTranspiler` (~950 lines) handles LAL's `@CompileStatic` delegation-based DSL. LAL scripts have a fundamentally different structure from MAL -- they are statement-based builder patterns rather than expression-based computations.
+
+#### Statement Mappings
+
+| Groovy Construct | Java Output |
+|-----------------|-------------|
+| `filter { ... }` | Body unwrapped, emitted directly on `filterSpec` |
+| `json {}` | `filterSpec.json()` |
+| `json { abortOnFailure false }` | `filterSpec.json(jp -> { jp.abortOnFailure(false); })` |
+| `text { regexp /pattern/ }` | `filterSpec.text(tp -> { tp.regexp("pattern"); })` |
+| `yaml {}` | `filterSpec.yaml()` |
+| `extractor { ... }` | `filterSpec.extractor(ext -> { ... })` |
+| `sink { ... }` | `filterSpec.sink(s -> { ... })` |
+| `abort {}` | `filterSpec.abort()` |
+| `service parsed.service as String` | `ext.service(String.valueOf(getAt(binding.parsed(), "service")))` |
+| `layer parsed.layer as String` | `ext.layer(String.valueOf(getAt(binding.parsed(), "layer")))` |
+| `tag(key: val)` | `ext.tag(Map.of("key", val))` |
+| `timestamp parsed.time as String` | `ext.timestamp(String.valueOf(getAt(binding.parsed(), "time")))` |
+
+#### Property Access and Safe Navigation
+
+| Groovy Pattern | Java Output |
+|---------------|-------------|
+| `parsed.field` | `getAt(binding.parsed(), "field")` |
+| `parsed.field.nested` | `getAt(getAt(binding.parsed(), "field"), "nested")` |
+| `parsed?.field?.nested` | `((__v0 = binding.parsed()) == null ? null : ((__v1 = getAt(__v0, "field")) == null ? null : getAt(__v1, "nested")))` |
+| `log.tags` | `binding.log().getTags()` |
+
+#### Cast and Type Handling
+
+| Groovy Pattern | Java Output |
+|---------------|-------------|
+| `expr as String` | `String.valueOf(expr)` |
+| `expr as Long` | `toLong(expr)` |
+| `expr as Integer` | `toInt(expr)` |
+| `expr as Boolean` | `toBoolean(expr)` |
+| `"${expr}"` (GString) | `"" + expr` |
+
+#### LAL Spec Consumer Overloads
+
+LAL spec classes (`FilterSpec`, `ExtractorSpec`, `SinkSpec`) get additional method overloads accepting `java.util.function.Consumer` alongside existing Groovy `Closure` parameters:
+
+```java
+// FilterSpec - existing
+public void extractor(Closure<?> cl) { ... }
+// FilterSpec - new overload
+public void extractor(Consumer<ExtractorSpec> consumer) { ... }
+
+// SinkSpec - existing
+public void sampler(Closure<?> cl) { ... }
+// SinkSpec - new overload
+public void sampler(Consumer<SamplerSpec> consumer) { ... }
+```
+
+Methods requiring Consumer overloads: `text()`, `json()`, `yaml()`, `extractor()`, `sink()`, `slowSql()`, `sampledTrace()`, `metrics()`, `sampler()`, `enforcer()`, `dropper()`.
+
+#### SHA-256 Deduplication
+
+LAL manifest is keyed by SHA-256 hash of the DSL content. Identical scripts across different YAML files share one compiled class. In practice, 10 LAL rules map to 6 unique classes.
+
+### 3.5 Hierarchy Script: v1/v2 Module Split
+
+The hierarchy matching rules in `hierarchy-definition.yml` use `GroovyShell.evaluate()` to compile 4 Groovy closures at runtime. Unlike MAL/LAL, hierarchy does not need a transpiler (only 4 rules, finite set), but it follows the same v1/v2/checker module pattern for consistency and to remove Groovy from `server-core`.
+
+#### Current State (server-core, Groovy-coupled)
+
+`HierarchyDefinitionService.java` lives in `server-core` and is registered as a `Service` in `CoreModule`. Its inner class `MatchingRule` holds a Groovy `Closure<Boolean>`:
+
+```java
+// server-core/...config/HierarchyDefinitionService.java (current)
+public static class MatchingRule {
+ private final String name;
+ private final String expression;
+ private final Closure<Boolean> closure; // groovy.lang.Closure
+
+ public MatchingRule(final String name, final String expression) {
+ GroovyShell sh = new GroovyShell();
+ closure = (Closure<Boolean>) sh.evaluate(expression); // Groovy at runtime
+ }
+}
+```
+
+This `MatchingRule` is referenced by three classes in `server-core`:
+- `HierarchyDefinitionService` -- builds the rule map from YAML
+- `HierarchyService` -- calls `matchingRule.getClosure().call(service, comparedService)` for auto-matching
+- `HierarchyQueryService` -- reads the hierarchy definition map
+
+#### Step 1: Make server-core Groovy-Free
+
+Refactor `MatchingRule` in `server-core` to use a Java functional interface instead of `Closure<Boolean>`:
+
+```java
+// server-core/...config/HierarchyDefinitionService.java (refactored)
+public static class MatchingRule {
+ private final String name;
+ private final String expression;
+ private final BiFunction<Service, Service, Boolean> matcher; // pure Java
+
+ public MatchingRule(final String name, final String expression,
+ final BiFunction<Service, Service, Boolean> matcher) {
+ this.name = name;
+ this.expression = expression;
+ this.matcher = matcher;
+ }
+
+ public boolean match(Service upper, Service lower) {
+ return matcher.apply(upper, lower);
+ }
+}
+```
+
+`HierarchyDefinitionService.init()` no longer compiles Groovy expressions itself. Instead, it receives a `Map<String, BiFunction<Service, Service, Boolean>>` (the rule registry) from outside -- injected by whichever implementation module (v1 or v2) is active.
+
+`HierarchyService` changes from `matchingRule.getClosure().call(u, l)` to `matchingRule.match(u, l)`.
+
+Remove all `groovy.lang.*` imports from `server-core`.
+
+#### Step 2: hierarchy-v1 (Groovy-based, for checker only)
+
+```java
+// analyzer/hierarchy-v1/.../GroovyHierarchyRuleProvider.java
+public class GroovyHierarchyRuleProvider {
+ public static Map<String, BiFunction<Service, Service, Boolean>> buildRules(
+ Map<String, String> ruleExpressions) {
+ Map<String, BiFunction<Service, Service, Boolean>> rules = new HashMap<>();
+ GroovyShell sh = new GroovyShell();
+ ruleExpressions.forEach((name, expression) -> {
+ Closure<Boolean> closure = (Closure<Boolean>) sh.evaluate(expression);
+ rules.put(name, (u, l) -> closure.call(u, l));
+ });
+ return rules;
+ }
+}
+```
+
+This module depends on Groovy and wraps the original `GroovyShell.evaluate()` logic. It is NOT included in the runtime classpath -- only used by the checker.
+
+#### Step 3: hierarchy-v2 (Pure Java, for runtime)
+
+```java
+// analyzer/hierarchy-v2/.../JavaHierarchyRuleProvider.java
+public class JavaHierarchyRuleProvider {
+ private static final Map<String, BiFunction<Service, Service, Boolean>> RULE_REGISTRY;
+ static {
+ RULE_REGISTRY = new HashMap<>();
+ RULE_REGISTRY.put("name",
+ (u, l) -> Objects.equals(u.getName(), l.getName()));
+ RULE_REGISTRY.put("short-name",
+ (u, l) -> Objects.equals(u.getShortName(), l.getShortName()));
+ RULE_REGISTRY.put("lower-short-name-remove-ns", (u, l) -> {
+ String sn = l.getShortName();
+ int dot = sn.lastIndexOf('.');
+ return dot > 0 && Objects.equals(u.getShortName(), sn.substring(0, dot));
+ });
+ RULE_REGISTRY.put("lower-short-name-with-fqdn", (u, l) -> {
+ String sn = u.getShortName();
+ int colon = sn.lastIndexOf(':');
+ return colon > 0 && Objects.equals(
+ sn.substring(0, colon),
+ l.getShortName() + ".svc.cluster.local");
+ });
+ }
+
+ public static Map<String, BiFunction<Service, Service, Boolean>> buildRules(
+ Map<String, String> ruleExpressions) {
+ Map<String, BiFunction<Service, Service, Boolean>> rules = new HashMap<>();
+ ruleExpressions.forEach((name, expression) -> {
+ BiFunction<Service, Service, Boolean> fn = RULE_REGISTRY.get(name);
+ if (fn == null) {
+ throw new IllegalArgumentException(
+ "Unknown hierarchy matching rule: " + name
+ + ". Known rules: " + RULE_REGISTRY.keySet());
+ }
+ rules.put(name, fn);
+ });
+ return rules;
+ }
+}
+```
+
+Unknown rule names fail fast at startup with `IllegalArgumentException`. The YAML file (`hierarchy-definition.yml`) continues to reference rule names (`name`, `short-name`, etc.) -- the Groovy expression strings in `auto-matching-rules` become documentation-only at runtime.
+
+#### Step 4: hierarchy-v1-v2-checker
+
+```java
+// analyzer/hierarchy-v1-v2-checker/.../HierarchyRuleComparisonTest.java
+class HierarchyRuleComparisonTest {
+ // Load rule expressions from hierarchy-definition.yml
+ // For each rule:
+ // Path A: GroovyHierarchyRuleProvider.buildRules() (v1)
+ // Path B: JavaHierarchyRuleProvider.buildRules() (v2)
+ // Construct test Service pairs (matching and non-matching cases)
+ // Assert v1.match(u, l) == v2.match(u, l) for all test pairs
+}
+```
+
+Test cases cover all 4 rules with realistic service name patterns:
+- `name`: exact match and mismatch
+- `short-name`: exact shortName match and mismatch
+- `lower-short-name-remove-ns`: `"svc" == "svc.namespace"` and edge cases (no dot, empty)
+- `lower-short-name-with-fqdn`: `"db:3306"` vs `"db.svc.cluster.local"` and edge cases (no colon, wrong suffix)
+
+---
+
+## 4. Module Structure
+
+### 4.1 Upstream Module Layout
+
+```
+oap-server/
+ server-core/ # MODIFIED: MatchingRule uses BiFunction (no Groovy imports)
+
+ analyzer/
+ meter-analyzer/ # Modified: add MalExpression, functional interfaces
+ log-analyzer/ # Modified: add LalExpression, Consumer overloads
+
+ mal-lal-v1/ # NEW: Move existing Groovy-based code here
+ meter-analyzer-v1/ # Original MAL (GroovyShell + ExpandoMetaClass)
+ log-analyzer-v1/ # Original LAL (GroovyShell + @CompileStatic)
+
+ mal-lal-v2/ # NEW: Pure Java transpiler-based implementations
+ meter-analyzer-v2/ # MalExpression loader + functional interface dispatch
+ log-analyzer-v2/ # LalExpression loader + Consumer dispatch
+ mal-transpiler/ # Build-time: Groovy AST -> Java source (MAL)
+ lal-transpiler/ # Build-time: Groovy AST -> Java source (LAL)
+
+ mal-lal-v1-v2-checker/ # NEW: Dual-path comparison tests (MAL + LAL)
+ 73 MAL test classes (1,281 assertions)
+ 5 LAL test classes (19 assertions)
+
+ hierarchy-v1/ # NEW: Groovy-based hierarchy rule provider (checker only)
+ hierarchy-v2/ # NEW: Pure Java hierarchy rule provider (runtime)
+ hierarchy-v1-v2-checker/ # NEW: Dual-path comparison tests (hierarchy)
+```
+
+### 4.2 Dependency Graph
+
+```
+mal-transpiler ──────────────> groovy (build-time only, for AST parsing)
+lal-transpiler ──────────────> groovy (build-time only, for AST parsing)
+hierarchy-v1 ────────────────> groovy (checker only, not runtime)
+
+meter-analyzer-v2 ──────────> meter-analyzer (interfaces + SampleFamily)
+log-analyzer-v2 ────────────> log-analyzer (interfaces + spec classes)
+hierarchy-v2 ───────────────> server-core (MatchingRule with BiFunction)
+
+mal-lal-v1-v2-checker ──────> mal-lal-v1 (Groovy path)
+mal-lal-v1-v2-checker ──────> mal-lal-v2 (Java path)
+hierarchy-v1-v2-checker ────> hierarchy-v1 (Groovy path)
+hierarchy-v1-v2-checker ────> hierarchy-v2 (Java path)
+
+server-starter ─────────────> meter-analyzer-v2 (runtime, no Groovy)
+server-starter ─────────────> log-analyzer-v2 (runtime, no Groovy)
+server-starter ─────────────> hierarchy-v2 (runtime, no Groovy)
+server-starter ────────────X─> mal-lal-v1 (NOT in runtime)
+server-starter ────────────X─> hierarchy-v1 (NOT in runtime)
+```
+
+### 4.3 Key Design Principle: No Coexistence
+
+v1 (Groovy) and v2 (Java) never coexist in the OAP runtime classpath. The `mal-lal-v1` and `hierarchy-v1` modules are only dependencies of their respective checker modules for CI validation. The runtime (`server-starter`) depends only on v2 modules.
+
+---
+
+## 5. Implementation Steps
+
+### Phase 1: Interfaces and SampleFamily Modifications
+
+**Files to modify:**
+
+1. **Create `MalExpression.java`** in `meter-analyzer`
+ - Path: `oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/MalExpression.java`
+
+2. **Create `MalFilter.java`** in `meter-analyzer`
+ - Path: `oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/MalFilter.java`
+
+3. **Create functional interfaces** in `meter-analyzer`
+ - `TagFunction extends Function<Map<String,String>, Map<String,String>>`
+ - `SampleFilter extends Predicate<Map<String,String>>`
+ - `ForEachFunction extends BiConsumer<String, Map<String,String>>`
+ - `DecorateFunction extends Consumer<MeterEntity>`
+ - `PropertiesExtractor extends Function<Map<String,String>, Map<String,String>>`
+
+4. **Add overloads to `SampleFamily.java`**
+ - Add `tag(TagFunction)` alongside existing `tag(Closure<?>)`
+ - Add `filter(SampleFilter)` alongside existing `filter(Closure<Boolean>)`
+ - Add `forEach(List, ForEachFunction)` alongside existing `forEach(List, Closure)`
+ - Add `decorate(DecorateFunction)` alongside existing `decorate(Closure)`
+ - Add `instance(..., PropertiesExtractor)` alongside existing `instance(..., Closure)`
+
+5. **Create `LalExpression.java`** in `log-analyzer`
+ - Path: `oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/dsl/LalExpression.java`
+
+6. **Add Consumer overloads to LAL spec classes**
+ - `FilterSpec`: `text(Consumer)`, `json(Consumer)`, `yaml(Consumer)`, `extractor(Consumer)`, `sink(Consumer)`, `filter(Consumer)`
+ - `ExtractorSpec`: `slowSql(Consumer)`, `sampledTrace(Consumer)`, `metrics(Consumer)`
+ - `SinkSpec`: `sampler(Consumer)`, `enforcer(Consumer)`, `dropper(Consumer)`
+
+### Phase 2: MAL Transpiler
+
+**New module: `oap-server/analyzer/mal-lal-v2/mal-transpiler/`**
+
+1. **`MalToJavaTranspiler.java`** (~1,230 lines)
+ - Uses `org.codehaus.groovy.control.CompilationUnit` at `Phases.CONVERSION`
+ - Walks AST via recursive visitor pattern
+ - Core methods:
+ - `transpileExpression(String className, String expression)` -> generates Java source for `MalExpression`
+ - `transpileFilter(String className, String filterLiteral)` -> generates Java source for `MalFilter`
+ - `collectSampleNames(Expression expr)` -> extracts all metric name references
+ - `visitExpression(Expression node)` -> recursive Java code emitter
+ - `visitClosureExpression(ClosureExpression node, String contextType)` -> closure-to-lambda
+ - `compileAll()` -> batch `javac` compile + manifest generation
+
+2. **Maven integration**: `exec-maven-plugin` during `generate-sources` phase
+ - Reads all MAL YAML files from resources
+ - Generates Java source to `target/generated-sources/mal/`
+ - Compiles to `target/classes/`
+ - Writes `META-INF/mal-expressions.txt` and `META-INF/mal-filter-expressions.properties`
+
+### Phase 3: LAL Transpiler
+
+**New module: `oap-server/analyzer/mal-lal-v2/lal-transpiler/`**
+
+1. **`LalToJavaTranspiler.java`** (~950 lines)
+ - Same AST approach as MAL but statement-based emission
+ - Core methods:
+ - `transpile(String className, String dslText)` -> generates Java source for `LalExpression`
+ - `emitStatement(Statement node, String receiver, BindingContext ctx)` -> statement emitter
+ - `visitConditionExpr(Expression node)` -> boolean expression emitter
+ - `emitPropertyAccess(PropertyExpression node)` -> `getAt()` with null safety
+ - SHA-256 deduplication: identical DSL content shares one class
+ - Helper methods in generated class: `getAt()`, `toLong()`, `toInt()`, `toBoolean()`, `isTruthy()`, `isNonEmptyString()`
+
+2. **Maven integration**: same `exec-maven-plugin` approach
+ - Writes `META-INF/lal-expressions.txt` (SHA-256 hash -> FQCN)
+
+### Phase 4: Runtime Loading (v2 Modules)
+
+**New module: `oap-server/analyzer/mal-lal-v2/meter-analyzer-v2/`**
+
+1. **Modified `DSL.java`** (MAL runtime):
+ ```java
+ public static Expression parse(String metricName, String expression) {
+ Map<String, String> manifest = loadManifest("META-INF/mal-expressions.txt");
+ String className = manifest.get(metricName);
+ MalExpression malExpr = (MalExpression) Class.forName(className)
+ .getDeclaredConstructor().newInstance();
+ return new Expression(metricName, expression, malExpr);
+ }
+ ```
+
+2. **Modified `Expression.java`** (MAL runtime):
+ - Wraps `MalExpression` instead of `DelegatingScript`
+ - `run()` calls `malExpression.run(sampleFamilies)` directly
+ - No `ExpandoMetaClass`, no `ExpressionDelegate`, no `ThreadLocal<Map>`
+
+3. **Modified `FilterExpression.java`** (MAL runtime):
+ - Loads `MalFilter` from `META-INF/mal-filter-expressions.properties`
+ - `filter()` calls `malFilter::test` via `SampleFamily.filter(SampleFilter)`
+
+**New module: `oap-server/analyzer/mal-lal-v2/log-analyzer-v2/`**
+
+4. **Modified `DSL.java`** (LAL runtime):
+ - Computes SHA-256 of DSL text, loads class from `META-INF/lal-expressions.txt`
+ - `evaluate()` calls `lalExpression.execute(filterSpec, binding)` directly
+ - No `GroovyShell`, no `LALDelegatingScript`
+
+### Phase 5: Hierarchy v1/v2 Module Split
+
+**Step 5a: Refactor `server-core` to remove Groovy**
+
+**File to modify:** `oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/config/HierarchyDefinitionService.java`
+
+1. Change `MatchingRule.closure` from `Closure<Boolean>` to `BiFunction<Service, Service, Boolean> matcher`
+2. Add constructor that accepts the `BiFunction` matcher directly
+3. Replace `getClosure()` with `match(Service upper, Service lower)` method
+4. Change `init()` to accept a rule registry (`Map<String, BiFunction<Service, Service, Boolean>>`) from outside instead of calling `GroovyShell.evaluate()` internally
+5. Remove all `groovy.lang.*` imports
+
+**File to modify:** `oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/hierarchy/HierarchyService.java`
+
+6. Change `matchingRule.getClosure().call(service, comparedService)` (lines 201-203, 220-222) to `matchingRule.match(service, comparedService)`
+
+**Step 5b: Create hierarchy-v1 module**
+
+**New module:** `oap-server/analyzer/hierarchy-v1/`
+
+1. `GroovyHierarchyRuleProvider.java`: wraps original `GroovyShell.evaluate()` logic
+2. Takes `Map<String, String>` (rule name -> Groovy expression) from YAML
+3. Returns `Map<String, BiFunction<Service, Service, Boolean>>` by evaluating closures
+4. Depends on Groovy -- NOT included in runtime, only used by checker
+
+**Step 5c: Create hierarchy-v2 module**
+
+**New module:** `oap-server/analyzer/hierarchy-v2/`
+
+1. `JavaHierarchyRuleProvider.java`: static `RULE_REGISTRY` with 4 Java lambdas
+2. Takes `Map<String, String>` (rule name -> Groovy expression) from YAML (expression ignored, only name used for registry lookup)
+3. Returns `Map<String, BiFunction<Service, Service, Boolean>>` from the registry
+4. Fails fast with `IllegalArgumentException` for unknown rule names
+5. Zero Groovy dependency
+
+### Phase 6: Comparison Test Suites
+
+**New module: `oap-server/analyzer/mal-lal-v1-v2-checker/`**
+
+1. **MAL comparison tests** (73 test classes):
+ - Base class `MALScriptComparisonBase` runs dual-path comparison:
+ - Path A: Fresh Groovy compilation with upstream `CompilerConfiguration`
+ - Path B: Load transpiled `MalExpression` from manifest
+ - Both receive identical `Map<String, SampleFamily>` input
+ - Compare: `ExpressionParsingContext` (scope, function, datatype, downsampling), `SampleFamily` result (values, labels, entity descriptions)
+ - JUnit 5 `@TestFactory` generates `DynamicTest` per metric rule
+ - Test data must be non-trivial to prevent vacuous agreement (both returning empty/null)
+
+2. **LAL comparison tests** (5 test classes):
+ - Base class `LALScriptComparisonBase` runs dual-path comparison:
+ - Path A: Groovy with `@CompileStatic` + `LALPrecompiledExtension`
+ - Path B: Load `LalExpression` from manifest via SHA-256
+ - Compare: `shouldAbort()`, `shouldSave()`, LogData.Builder state, metrics container, `databaseSlowStatement`, `sampledTraceBuilder`
+
+3. **Test statistics**: 1,281 MAL assertions + 19 LAL assertions = 1,300 total
+
+**New module: `oap-server/analyzer/hierarchy-v1-v2-checker/`**
+
+4. **Hierarchy comparison tests**:
+ - Load rule expressions from `hierarchy-definition.yml`
+ - For each of the 4 rules:
+ - Path A: `GroovyHierarchyRuleProvider.buildRules()` (v1, Groovy closures)
+ - Path B: `JavaHierarchyRuleProvider.buildRules()` (v2, Java lambdas)
+ - Construct test `Service` pairs covering matching and non-matching cases:
+ - `name`: exact match `("svc", "svc")` -> true, `("svc", "other")` -> false
+ - `short-name`: shortName match/mismatch
+ - `lower-short-name-remove-ns`: `"svc"` vs `"svc.namespace"` -> true, no dot -> false, empty -> false
+ - `lower-short-name-with-fqdn`: `"db:3306"` vs `"db.svc.cluster.local"` -> true, no colon -> false, wrong suffix -> false
+ - Assert `v1.match(u, l) == v2.match(u, l)` for all test pairs
+
+### Phase 7: Cleanup and Dependency Removal
+
+1. **Move v1 code to `mal-lal-v1/` and `hierarchy-v1/`** (or mark as `<scope>test</scope>`)
+2. **Remove Groovy from runtime classpath**: `groovy-5.0.3.jar` (~7 MB) becomes test-only
+3. **Remove from `server-starter` dependencies**: replace v1 with v2 module references for MAL, LAL, and hierarchy
+4. **Remove `NumberClosure.java`**: no longer needed without `ExpandoMetaClass`
+5. **Remove `ExpressionDelegate.propertyMissing()`**: replaced by `samples.getOrDefault()`
+6. **Remove Groovy closure overloads from `SampleFamily`** (after v1 is fully deprecated)
+7. **Remove `LALDelegatingScript.java`**: replaced by `LalExpression` interface
+8. **Verify `server-core` has zero Groovy imports**: `HierarchyDefinitionService` and `HierarchyService` now use `BiFunction` only
+
+---
+
+## 6. What Gets Removed from Runtime
+
+| Component | Current | After |
+|-----------|---------|-------|
+| `GroovyShell.parse()` in MAL `DSL.java` | 1,250+ calls at boot | `Class.forName()` from manifest |
+| `GroovyShell.evaluate()` in MAL `FilterExpression.java` | 29 filter compilations | `Class.forName()` from manifest |
+| `GroovyShell.parse()` in LAL `DSL.java` | 10 script compilations | `Class.forName()` from manifest |
+| `GroovyShell.evaluate()` in `HierarchyDefinitionService` | 4 rule compilations | `hierarchy-v2` Java lambda registry |
+| `Closure<Boolean>` in `MatchingRule` | Groovy closure in `server-core` | `BiFunction<Service, Service, Boolean>` (Groovy-free `server-core`) |
+| `ExpandoMetaClass` registration in `Expression.empower()` | Runtime metaclass on `Number` | Direct `multiply()`/`div()` method calls |
+| `ExpressionDelegate.propertyMissing()` | Dynamic property dispatch | `samples.getOrDefault()` |
+| `groovy.lang.Closure` in `SampleFamily` | 5 method signatures | Java functional interfaces |
+| `groovy-5.0.3.jar` runtime dependency | ~7 MB on classpath | Removed (build-time only) |
+
+---
+
+## 7. Transpiler Technical Details
+
+### 7.1 AST Parsing Strategy
+
+Both transpilers use Groovy's `CompilationUnit` at `Phases.CONVERSION`:
+
+```java
+CompilationUnit cu = new CompilationUnit();
+cu.addSource("expression", new StringReaderSource(
+ new StringReader(groovyCode), cu.getConfiguration()));
+cu.compile(Phases.CONVERSION); // Parse + AST transform, no codegen
+ModuleNode ast = cu.getAST();
+```
+
+This extracts the complete syntax tree without:
+- Generating Groovy bytecode
+- Resolving classes on classpath
+- Activating MOP or MetaClass
+
+The Groovy dependency is therefore **build-time only**.
+
+### 7.2 MAL Arithmetic Operand Swap
+
+The transpiler must replicate the exact behavior of upstream's `ExpandoMetaClass` on `Number`. When a `Number` appears on the left side of an operator with a `SampleFamily` on the right, the operands must be handled carefully:
+
+```
+N + SF -> SF.plus(N) // commutative, swap operands
+N - SF -> SF.minus(N).negative() // non-commutative: (N - SF) = -(SF - N)
+N * SF -> SF.multiply(N) // commutative, swap operands
+N / SF -> SF.newValue(v -> N / v) // non-commutative: per-sample (N / sample_value)
+```
+
+The transpiler detects `Number` vs `SampleFamily` operand types by tracking whether a sub-expression references sample names (metric properties) or is a numeric literal/constant.
+
+### 7.3 Batch Compilation
+
+Generated Java sources are compiled in a single `javac` invocation via `javax.tools.JavaCompiler`:
+
+```java
+JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
+List<JavaFileObject> sources = /* all generated .java files */;
+compiler.getTask(null, fileManager, diagnostics, options, null, sources).call();
+```
+
+This avoids 1,250+ individual `javac` invocations and provides full cross-file type checking.
+
+### 7.4 Manifest Format
+
+**`META-INF/mal-expressions.txt`**:
+```
+metric_name_1=org.apache.skywalking.oap.meter.analyzer.dsl.generated.MalExpr_metric_name_1
+metric_name_2=org.apache.skywalking.oap.meter.analyzer.dsl.generated.MalExpr_metric_name_2
+...
+```
+
+**`META-INF/mal-filter-expressions.properties`**:
+```
+{ tags -> tags.job_name == 'mysql-monitoring' }=org.apache.skywalking.oap.meter.analyzer.dsl.generated.MalFilter_0a1b2c3d
+...
+```
+
+**`META-INF/lal-expressions.txt`**:
+```
+sha256_hash_1=org.apache.skywalking.oap.log.analyzer.dsl.generated.LalExpr_sha256_hash_1
+sha256_hash_2=org.apache.skywalking.oap.log.analyzer.dsl.generated.LalExpr_sha256_hash_2
+...
+```
+
+---
+
+## 8. Worked Examples
+
+### 8.1 MAL Expression: K8s Node CPU Capacity
+
+**Groovy (upstream):**
+```groovy
+kube_node_status_capacity.tagEqual('resource', 'cpu')
+ .sum(['node'])
+ .tag({tags -> tags.node_name = tags.node; tags.remove("node")})
+ .service(['node_name'], Layer.K8S)
+```
+
+**Transpiled Java:**
+```java
+public class MalExpr_k8s_node_cpu implements MalExpression {
+ @Override
+ public SampleFamily run(Map<String, SampleFamily> samples) {
+ return samples.getOrDefault("kube_node_status_capacity", SampleFamily.EMPTY)
+ .tagEqual("resource", "cpu")
+ .sum(List.of("node"))
+ .tag(tags -> {
+ tags.put("node_name", tags.get("node"));
+ tags.remove("node");
+ return tags;
+ })
+ .service(List.of("node_name"), Layer.K8S);
+ }
+}
+```
+
+### 8.2 MAL Expression: Arithmetic with Number on Left
+
+**Groovy (upstream):**
+```groovy
+100 - container_cpu_usage / container_resource_limit_cpu * 100
+```
+
+**Transpiled Java:**
+```java
+public class MalExpr_cpu_percent implements MalExpression {
+ @Override
+ public SampleFamily run(Map<String, SampleFamily> samples) {
+ return samples.getOrDefault("container_cpu_usage", SampleFamily.EMPTY)
+ .div(samples.getOrDefault("container_resource_limit_cpu", SampleFamily.EMPTY))
+ .multiply(100)
+ .minus(100)
+ .negative();
+ }
+}
+```
+
+### 8.3 MAL Filter Expression
+
+**Groovy (upstream):**
+```groovy
+{ tags -> tags.job_name == 'mysql-monitoring' }
+```
+
+**Transpiled Java:**
+```java
+public class MalFilter_mysql implements MalFilter {
+ @Override
+ public boolean test(Map<String, String> tags) {
+ return "mysql-monitoring".equals(tags.get("job_name"));
+ }
+}
+```
+
+### 8.4 LAL Expression: MySQL Slow SQL
+
+**Groovy (upstream):**
+```groovy
+filter {
+ json {}
+ extractor {
+ layer parsed.layer as String
+ service parsed.service as String
+ timestamp parsed.time as String
+ if (tag("LOG_KIND") == "SLOW_SQL") {
+ slowSql {
+ id parsed.id as String
+ statement parsed.statement as String
+ latency parsed.query_time as Long
+ }
+ }
+ }
+ sink {}
+}
+```
+
+**Transpiled Java:**
+```java
+public class LalExpr_mysql_slowsql implements LalExpression {
+
+ @Override
+ public void execute(FilterSpec filterSpec, Binding binding) {
+ filterSpec.json();
+ filterSpec.extractor(ext -> {
+ ext.layer(String.valueOf(getAt(binding.parsed(), "layer")));
+ ext.service(String.valueOf(getAt(binding.parsed(), "service")));
+ ext.timestamp(String.valueOf(getAt(binding.parsed(), "time")));
+ if ("SLOW_SQL".equals(ext.tag("LOG_KIND"))) {
+ ext.slowSql(ss -> {
+ ss.id(String.valueOf(getAt(binding.parsed(), "id")));
+ ss.statement(String.valueOf(getAt(binding.parsed(), "statement")));
+ ss.latency(toLong(getAt(binding.parsed(), "query_time")));
+ });
+ }
+ });
+ filterSpec.sink(s -> {});
+ }
+
+ private static Object getAt(Object obj, String key) {
+ if (obj instanceof Binding.Parsed) return ((Binding.Parsed) obj).getAt(key);
+ if (obj instanceof Map) return ((Map<?, ?>) obj).get(key);
+ return null;
+ }
+
+ private static long toLong(Object val) {
+ if (val instanceof Number) return ((Number) val).longValue();
+ if (val instanceof String) return Long.parseLong((String) val);
+ return 0L;
+ }
+}
+```
+
+### 8.5 Hierarchy Rule: lower-short-name-remove-ns
+
+**Groovy (upstream, in hierarchy-definition.yml):**
+```groovy
+{ (u, l) -> {
+ if(l.shortName.lastIndexOf('.') > 0)
+ return u.shortName == l.shortName.substring(0, l.shortName.lastIndexOf('.'));
+ return false;
+} }
+```
+
+**Java replacement (in HierarchyDefinitionService.java):**
+```java
+RULE_REGISTRY.put("lower-short-name-remove-ns", (u, l) -> {
+ String sn = l.getShortName();
+ int dot = sn.lastIndexOf('.');
+ return dot > 0 && Objects.equals(u.getShortName(), sn.substring(0, dot));
+});
+```
+
+---
+
+## 9. Verification Strategy
+
+### 9.1 Dual-Path Comparison Testing
+
+Every generated Java class is validated against the original Groovy behavior in CI:
+
+```
+For each MAL YAML file:
+ For each metric rule:
+ 1. Compile expression with Groovy (v1 path)
+ 2. Load transpiled MalExpression (v2 path)
+ 3. Construct realistic sample data (non-trivial to prevent vacuous agreement)
+ 4. Run both paths with identical input
+ 5. Assert identical output:
+ - ExpressionParsingContext (scope, function, datatype, samples, downsampling)
+ - SampleFamily result (values, labels, entity descriptions)
+```
+
+### 9.2 Staleness Detection
+
+Properties files record SHA-256 hashes of upstream classes that have same-FQCN replacements. If upstream changes a class, the staleness test fails, forcing review of the replacement.
+
+### 9.3 Automatic Coverage
+
+New MAL/LAL YAML rules added to `server-starter/src/main/resources/` are automatically covered by the transpiler and comparison tests -- if the transpiler produces different results from Groovy, the build fails.
+
+---
+
+## 10. Statistics
+
+| Metric | Count |
+|--------|-------|
+| MAL YAML files processed | 71 |
+| MAL metric expressions transpiled | 1,254 |
+| MAL filter expressions transpiled | 29 |
+| LAL YAML files processed | 8 |
+| LAL rules transpiled | 10 (6 unique after SHA-256 dedup) |
+| Hierarchy rules replaced | 4 |
+| Hierarchy rules replaced | 4 |
+| Total generated Java classes | ~1,289 |
+| Comparison test assertions | 1,300+ (MAL: 1,281, LAL: 19, hierarchy: 4 rules x multiple service pairs) |
+| Lines of transpiler code (MAL) | ~1,230 |
+| Lines of transpiler code (LAL) | ~950 |
+| Runtime JAR removed | groovy-5.0.3.jar (~7 MB) |
+
+---
+
+## 11. Risk Assessment
+
+| Risk | Mitigation |
+|------|-----------|
+| Transpiler misses an AST pattern | 1,300 dual-path comparison tests catch any divergence |
+| New MAL/LAL expression uses unsupported Groovy syntax | Transpiler throws clear error at build time; new pattern must be added |
+| Upstream SampleFamily/Spec changes break replacement | Staleness tests detect SHA-256 changes |
+| Performance regression | Eliminated dynamic dispatch should only improve performance; benchmark with `MetricConvert` pipeline |
+| Custom user MAL/LAL scripts | Users who extend default rules with custom Groovy scripts must follow the same syntax subset supported by the transpiler |
+
+---
+
+## 12. Migration Timeline
+
+1. **Phase 1**: Add interfaces and functional interface overloads to existing `meter-analyzer` and `log-analyzer` (non-breaking, additive changes)
+2. **Phase 2-3**: Implement MAL and LAL transpilers in new `mal-lal-v2/` modules
+3. **Phase 4**: Implement v2 runtime loaders (modified `DSL.java`, `Expression.java`, `FilterExpression.java`)
+4. **Phase 5**: Hierarchy v1/v2 module split -- refactor `server-core` to remove Groovy, create `hierarchy-v1/` (Groovy, checker-only) and `hierarchy-v2/` (Java lambdas, runtime)
+5. **Phase 6**: Build comparison test suites -- `mal-lal-v1-v2-checker/` AND `hierarchy-v1-v2-checker/`
+6. **Phase 7**: Switch `server-starter` from v1 to v2 for all three subsystems (MAL, LAL, hierarchy), remove Groovy from runtime classpath
+7. **Eventually**: Remove `mal-lal-v1`, `hierarchy-v1`, and all checker modules once community confidence is established