You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by wu...@apache.org on 2026/08/08 00:47:59 UTC
(skywalking) branch feat/mal-line-attribution created (now f175bbe3ec)
This is an automated email from the ASF dual-hosted git repository.
wu-sheng pushed a change to branch feat/mal-line-attribution
in repository https://gitbox.apache.org/repos/asf/skywalking.git
at f175bbe3ec Fix MAL line attribution: separate the operator and machine coordinates
This branch includes the following new commits:
new f175bbe3ec Fix MAL line attribution: separate the operator and machine coordinates
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
(skywalking) 01/01: Fix MAL line attribution: separate the operator and machine coordinates
Posted by wu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wu-sheng pushed a commit to branch feat/mal-line-attribution
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit f175bbe3ec33d25a54b82552c5c088e581b52086
Author: Wu Sheng <wu...@foxmail.com>
AuthorDate: Fri Aug 7 17:47:41 2026 -0700
Fix MAL line attribution: separate the operator and machine coordinates
A MAL rule's compiled expression lives in two independent coordinate spaces,
and one overloaded `yamlSource` string was serving as class label, file
identifier and line coordinate at once. It was wrong in all three roles:
- the `_L<n>_` segment in a generated class name was the rules-list INDEX,
not a line, so every rule file's first rule was labelled `L0`;
- `SourceFile` embedded that same fake line AND named a file that is never
written (it used the metric name, while the generated source file is named
after the class), so IDE source-attach could never resolve a MAL stack
frame regardless of whether the line was right;
- `LineNumberTable` held statement ordinals matching neither the YAML nor
the generated source;
- MAL debug records carried no line at all: the recorder hardcoded 0 at all
five probe sites, and the wire shape documented `sourceLine` as "omitted
for MAL (no per-line mapping)" -- true only because the mapping had never
been built.
The fix splits the three roles apart.
OPERATOR COORDINATE (the rule .yaml an operator wrote) now reaches the
dsl-debugging API as a per-STAGE `sourceLine`. Per stage rather than per
rule, because `formatExp` splices `expPrefix`, `exp` and `expSuffix` into
one expression: a stage contributed by the file-level `expSuffix:` must
report the suffix's line, not the rule's. A per-rule implementation gets
that silently wrong, since the rule's line does exist in the file.
`MalYamlLineIndex` resolves the four anchors from a second snakeyaml compose
pass over the same text bean binding discards; `MalSourceMap` attributes an
offset to the fragment that authored it, using splice ranges
`injectExpPrefix` already computed but discarded.
MACHINE COORDINATE (the generated .java) is now self-consistent:
`SourceFile` equals the file actually written, and `LineNumberTable` points
at real statements in it. `MalGeneratedSourceLines` derives the boundaries
from the same generated text the bytecode scan walks, so the two cannot
drift; a mismatch logs a warning. Companion closure classes emit no
`LineNumberTable` at all -- no source file is written for them and the
store-scan is not a boundary detector there, so an absent table (JVM reports
"unknown line") beats a degenerate one.
The class-name label keeps the YAML line, but the real one.
`MalSourceRef` carries both coordinates with distinct accessors so
substituting one for the other is a visible mistake rather than the default.
An unresolvable line is `-1`, never 0 or omitted, so a resolution failure
stays visible in the artifact instead of silently degrading; it renders as
`unknown` inside class names, where `-` is not legal.
Contract change: `MALDebugRecorder`'s five `append*` methods take an
additional `sourceLine`.
Rule behaviour is unchanged: 1362 shipped-rule execution tests, 35 filter
tests and 22 parser tests pass, the last confirming the `injectExpPrefix`
rewrite is byte-identical.
---
docs/en/changes/changes.md | 6 +
.../operation/dynamic-code-generation-debugging.md | 9 +-
.../setup/backend/admin-api/dsl-debugging-mal.md | 5 +-
.../skywalking/oap/meter/analyzer/v2/Analyzer.java | 49 +++++-
.../oap/meter/analyzer/v2/MalYamlLineIndex.java | 192 +++++++++++++++++++++
.../oap/meter/analyzer/v2/MetricConvert.java | 37 +++-
.../oap/meter/analyzer/v2/MetricRuleConfig.java | 42 +++++
.../analyzer/v2/compiler/MALBytecodeHelper.java | 89 ++++++++--
.../analyzer/v2/compiler/MALClassGenerator.java | 35 +++-
.../analyzer/v2/compiler/MALClosureCodegen.java | 8 +-
.../analyzer/v2/compiler/MALCodegenHelper.java | 10 +-
.../meter/analyzer/v2/compiler/MALExprCodegen.java | 32 +++-
.../analyzer/v2/compiler/MALExpressionModel.java | 26 ++-
.../v2/compiler/MALMethodChainCodegen.java | 14 +-
.../analyzer/v2/compiler/MALScriptParser.java | 77 +++++++--
.../v2/compiler/MalGeneratedSourceLines.java | 89 ++++++++++
.../meter/analyzer/v2/compiler/MalSourceMap.java | 135 +++++++++++++++
.../meter/analyzer/v2/compiler/MalSourceRef.java | 111 ++++++++++++
.../skywalking/oap/meter/analyzer/v2/dsl/DSL.java | 20 +++
.../oap/meter/analyzer/v2/dsldebug/MALDebug.java | 24 +--
.../analyzer/v2/dsldebug/MALDebugRecorder.java | 12 +-
.../analyzer/v2/prometheus/rule/MetricsRule.java | 14 ++
.../meter/analyzer/v2/prometheus/rule/Rule.java | 16 ++
.../v2/prometheus/rule/RuleSourceLines.java | 71 ++++++++
.../meter/analyzer/v2/prometheus/rule/Rules.java | 9 +-
.../meter/analyzer/v2/MalProbeSourceLineTest.java | 139 +++++++++++++++
.../meter/analyzer/v2/MalYamlLineIndexTest.java | 127 ++++++++++++++
.../v2/compiler/MalGeneratedSourceLinesTest.java | 92 ++++++++++
.../v2/compiler/MalLineAttributionTest.java | 181 +++++++++++++++++++
.../v2/compiler/MalPrefixInjectionRangesTest.java | 83 +++++++++
.../analyzer/v2/compiler/MalProbeEmissionTest.java | 148 ++++++++++++++++
.../analyzer/v2/compiler/MalSourceMapTest.java | 103 +++++++++++
.../dsl/debugging/mal/MALDebugRecorderImpl.java | 23 +--
.../receiver/runtimerule/apply/MalFileApplier.java | 4 +
34 files changed, 1943 insertions(+), 89 deletions(-)
diff --git a/docs/en/changes/changes.md b/docs/en/changes/changes.md
index 0c51ebd786..e4e5b5e3c2 100644
--- a/docs/en/changes/changes.md
+++ b/docs/en/changes/changes.md
@@ -255,6 +255,12 @@
admin-host only" entry above for the public REST retirement.
#### OAP Server
+* Fix MAL generated-class line attribution, which conflated three different coordinate spaces.
+ - The `_L<n>_` segment in a generated class name was the rules-list index, not a line, so every rule file's first rule was reported as `L0`. It now carries the rule's real YAML line (and the `filter:` line for filter classes).
+ - The `SourceFile` attribute named a file that was never written — it used the metric name while the generated source file is named after the class — so IDE source-attach could never resolve a MAL stack frame regardless of the line. It now equals the generated `.java` file.
+ - `LineNumberTable` held statement ordinals matching neither the YAML nor the generated source; it now points at real statements in the generated `.java`. Companion closure classes emit no `LineNumberTable` at all, since no source file is written for them and their entries were degenerate.
+ - MAL debug records now carry a per-stage `sourceLine` (previously hardcoded to 0 and documented as "omitted for MAL"). It is per stage rather than per rule because a compiled expression splices `expPrefix`, `exp` and `expSuffix` together, so a stage contributed by the file-level `expSuffix:` reports the suffix's line rather than the rule's.
+ - **Contract change:** `MALDebugRecorder`'s five `append*` methods take an additional `sourceLine` argument.
* Support runtime rule hot-update and DSL debugging for the `meter-analyzer-config` catalog, bringing native meter (`MeterReportService`) rules to parity with `otel-rules`.
- Meter rules now load through the same `Rules`/`Rule` pipeline `otel-rules` uses, so they participate in `RuleSetMerger`, are recorded in `StaticRuleRegistry`, support the optional `layerDefinitions` block, and generate source-named expression classes instead of falling back to `MalExpr_<N>`.
- `MeterProcessService` now implements `MalConverterRegistry` and publishes debug holders at boot, so a meter rule can be added / overridden / inactivated at runtime, and attached to a DSL debug session, without restarting the OAP.
diff --git a/docs/en/operation/dynamic-code-generation-debugging.md b/docs/en/operation/dynamic-code-generation-debugging.md
index b582f81df1..64e3c80b19 100644
--- a/docs/en/operation/dynamic-code-generation-debugging.md
+++ b/docs/en/operation/dynamic-code-generation-debugging.md
@@ -123,16 +123,17 @@ Reading this:
| DSL | SourceFile Example | Generated Class Name | How to Read |
|-----|-------------------|---------------------|-------------|
| OAL | `(core.oal:20)ServiceRespTimeMetrics.java` | `ServiceRespTimeMetrics` | OAL file `core.oal`, line 20 defines this metric |
-| MAL | `(vm.yaml:25)cpu_total_percentage.java` | `vm_L25_cpu_total_percentage` | YAML file `vm.yaml`, line 25, rule `cpu_total_percentage` |
-| MAL filter | `(vm.yaml:20)filter.java` | `vm_L20_filter` | YAML file `vm.yaml`, line 20, filter expression |
+| MAL | `vm_L25_cpu_total_percentage.java` | `vm_L25_cpu_total_percentage` | YAML file `vm.yaml`, line 25, rule `cpu_total_percentage`. The `SourceFile` equals the generated class file so an IDE can resolve the frame; the YAML provenance is carried by the class name. |
+| MAL filter | `vm_L20_filter.java` | `vm_L20_filter` | YAML file `vm.yaml`, line 20, filter expression |
| LAL | `(default.yaml:3)default.java` | `default_L3_default` | YAML file `default.yaml`, line 3, rule `default` |
| Hierarchy | `(hierarchy-definition.yml:88)name.java` | `hierarchy_definition_L88_name` | Rule `name` at line 88 in `hierarchy-definition.yml` |
**Notes:**
- The class name pattern is `{yamlFileName}_L{lineNo}_{ruleName}` for all DSLs (except OAL).
The yaml file name and line number from `yamlSource` are combined with the rule name or `filter`.
-- The number after `:` in the SourceFile prefix is the line number in the YAML file where the rule is defined (for MAL in production, this may be a 0-based rule index instead of a line number).
-- When source information is unavailable, the class name falls back to `MalExpr_<N>` / `LalExpr_<N>` / `HierarchyRule_<N>` and the SourceFile to just `ClassName.java` without the parenthesized prefix.
+- The number after `_L` in the class name is the 1-based line in the YAML file where the rule (or the `filter:`) is defined.
+- For MAL, the line number in a stack frame (`SourceFile:LineNumber`) is a line in the GENERATED `.java` file, not in the YAML. The two are different coordinate spaces: use the class name for YAML provenance, and the frame line to locate the generated statement. The per-stage YAML line is reported separately by the [DSL debugging API](../setup/backend/admin-api/dsl-debugging-mal.md) as `sourceLine`.
+- When source information is unavailable, the class name falls back to `MalExpr_<N>` / `LalExpr_<N>` / `HierarchyRule_<N>`. For MAL, a line that was expected but could not be resolved renders as `_Lunknown_` rather than being silently omitted, so the failure stays visible.
### Mapping Back to DSL Source
diff --git a/docs/en/setup/backend/admin-api/dsl-debugging-mal.md b/docs/en/setup/backend/admin-api/dsl-debugging-mal.md
index 7c096a43b4..7937b1d472 100644
--- a/docs/en/setup/backend/admin-api/dsl-debugging-mal.md
+++ b/docs/en/setup/backend/admin-api/dsl-debugging-mal.md
@@ -27,7 +27,10 @@ nodes[]
sourceText — verbatim DSL fragment for this probe
continueOn — true (MAL captures kept-only; see overview)
payload — SampleFamily.toJson() at this probe stage
- sourceLine — omitted for MAL (no per-line mapping)
+ sourceLine — 1-based line in the rule `.yaml` this stage was
+ written on. Per STAGE, not per rule: a stage from the
+ file-level `expSuffix:` reports the suffix's line, not
+ the rule's. Omitted when it could not be resolved.
```
Sample types and the probes that emit them:
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/Analyzer.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/Analyzer.java
index 4cdea3c3ee..c3ced59403 100644
--- a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/Analyzer.java
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/Analyzer.java
@@ -42,6 +42,8 @@ import org.apache.skywalking.oap.meter.analyzer.v2.dsl.ExpressionMetadata;
import org.apache.skywalking.oap.meter.analyzer.v2.dsl.FilterExpression;
import org.apache.skywalking.oap.meter.analyzer.v2.dsl.Result;
import org.apache.skywalking.oap.meter.analyzer.v2.dsl.Sample;
+import org.apache.skywalking.oap.meter.analyzer.v2.compiler.MalSourceMap;
+import org.apache.skywalking.oap.meter.analyzer.v2.compiler.MalSourceRef;
import org.apache.skywalking.oap.meter.analyzer.v2.dsl.SampleFamily;
import org.apache.skywalking.oap.meter.analyzer.v2.dsldebug.MALDebug;
import org.apache.skywalking.oap.server.core.analysis.Layer;
@@ -199,7 +201,27 @@ public class Analyzer {
final javassist.ClassPool pool,
final ClassLoader targetClassLoader,
final StorageManipulationOpt storageOpt) {
- Expression e = DSL.parse(metricName, expression, yamlSource, pool, targetClassLoader);
+ return prepare(metricName, filter, expression, meterSystem, yamlSource, pool,
+ targetClassLoader, storageOpt, MalSourceMap.EMPTY);
+ }
+
+ /**
+ * Source-mapped overload — see {@link DSL#parse(String, String, String,
+ * javassist.ClassPool, ClassLoader, MalSourceMap)}.
+ *
+ * @param sourceMap resolves each chain stage to its YAML line
+ */
+ public static Analyzer prepare(final String metricName,
+ final FilterExpression filter,
+ final String expression,
+ final MeterSystem meterSystem,
+ final String yamlSource,
+ final javassist.ClassPool pool,
+ final ClassLoader targetClassLoader,
+ final StorageManipulationOpt storageOpt,
+ final MalSourceMap sourceMap) {
+ Expression e = DSL.parse(metricName, expression, yamlSource, pool, targetClassLoader,
+ sourceMap);
ExpressionMetadata ctx = e.parse();
Analyzer analyzer = new Analyzer(metricName, filter, e, meterSystem, ctx);
analyzer.pool = pool;
@@ -250,6 +272,14 @@ public class Analyzer {
* MetadataRegistry populate without server-side DDL.
*/
private StorageManipulationOpt storageOpt = StorageManipulationOpt.schemaCreateIfAbsent();
+ /**
+ * YAML lines for the two probes this class fires by hand rather than through codegen. Both
+ * are file-level: the filter is declared once per file, and the terminal meterEmit stage is
+ * contributed by the file-level {@code expSuffix:} when one is declared (falling back to the
+ * rule's own {@code exp:} line when it is not).
+ */
+ private int filterYamlLine = MalSourceRef.UNRESOLVED;
+ private int meterEmitYamlLine = MalSourceRef.UNRESOLVED;
/**
* Analyse the full sample family map and produce meter-system metrics.
@@ -302,7 +332,7 @@ public class Analyzer {
MALDebug.captureFilter(debugHolder, metricName,
filterExpression.getLiteral(),
kept ? input : preFilter,
- kept);
+ kept, filterYamlLine);
}
if (input.isEmpty()) {
if (log.isDebugEnabled()) {
@@ -416,6 +446,18 @@ public class Analyzer {
* state. OAL splits build vs aggregation because its entry function mutates
* the {@code Metrics} between them; MAL has nothing equivalent.
*/
+ /**
+ * Supply the file-level lines for the hand-written probes. Called by {@code MetricConvert}
+ * once the rule's anchors are known.
+ *
+ * @param filterLine line of the file-level {@code filter:} key
+ * @param meterEmitLine line to attribute the terminal emit to
+ */
+ public void setProbeSourceLines(final int filterLine, final int meterEmitLine) {
+ this.filterYamlLine = filterLine > 0 ? filterLine : MalSourceRef.UNRESOLVED;
+ this.meterEmitYamlLine = meterEmitLine > 0 ? meterEmitLine : MalSourceRef.UNRESOLVED;
+ }
+
private void captureMeterEmit(final GateHolder debugHolder,
final MeterEntity entity,
final AcceptableValue<?> value,
@@ -424,7 +466,8 @@ public class Analyzer {
return;
}
MALDebug.captureMeterEmit(debugHolder, metricName, entity, metricName,
- value, TimeBucket.getMinuteTimeBucket(sourceTimestamp));
+ value, TimeBucket.getMinuteTimeBucket(sourceTimestamp),
+ meterEmitYamlLine);
}
private long getValue(Sample sample) {
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/MalYamlLineIndex.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/MalYamlLineIndex.java
new file mode 100644
index 0000000000..52bda45fd1
--- /dev/null
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/MalYamlLineIndex.java
@@ -0,0 +1,192 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.meter.analyzer.v2;
+
+import java.io.StringReader;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import lombok.Getter;
+import lombok.extern.slf4j.Slf4j;
+import org.yaml.snakeyaml.LoaderOptions;
+import org.yaml.snakeyaml.Yaml;
+import org.yaml.snakeyaml.nodes.MappingNode;
+import org.yaml.snakeyaml.nodes.Node;
+import org.yaml.snakeyaml.nodes.NodeTuple;
+import org.yaml.snakeyaml.nodes.ScalarNode;
+import org.yaml.snakeyaml.nodes.SequenceNode;
+
+/**
+ * Resolves the 1-based YAML line of each source fragment in a MAL rule file.
+ *
+ * <p>SnakeYAML's bean binding ({@code Yaml.loadAs}) discards positional {@code Mark} data, so the
+ * rule objects the loaders hand to {@link MetricConvert} carry no idea where they came from. This
+ * class runs a second {@code compose} pass over the SAME text and reads the marks off the node
+ * tree, which is why callers must give it the exact bytes they bound from — re-reading the file
+ * could pick up a different revision.
+ *
+ * <p>Four anchors are resolved, because a compiled MAL expression is a splice of up to three
+ * separate source locations plus a separately-compiled filter:
+ * <ul>
+ * <li>{@code filter} / {@code expPrefix} / {@code expSuffix} — file-level, one line each.</li>
+ * <li>per {@code metricsRules} entry — the entry's own line (the {@code - name:} anchor, used
+ * for the generated class label) and its {@code exp:} line.</li>
+ * </ul>
+ *
+ * <p>Every accessor returns {@code 0} for "unknown", which every consumer treats as "omit the line"
+ * rather than "line zero". Resolution is best-effort by design: a malformed or unusual document
+ * degrades to zeros instead of failing the rule load, because a missing line number must never
+ * stop a rule from compiling.
+ */
+@Slf4j
+public final class MalYamlLineIndex {
+
+ /** Rules key in the standard MAL rule file. Zabbix uses {@code metrics} instead. */
+ public static final String DEFAULT_RULES_KEY = "metricsRules";
+
+ private static final MalYamlLineIndex EMPTY =
+ new MalYamlLineIndex(0, 0, 0, Collections.emptyList());
+
+ @Getter
+ private final int filterLine;
+ @Getter
+ private final int expPrefixLine;
+ @Getter
+ private final int expSuffixLine;
+ private final List<RuleLines> rules;
+
+ private MalYamlLineIndex(final int filterLine, final int expPrefixLine,
+ final int expSuffixLine, final List<RuleLines> rules) {
+ this.filterLine = filterLine;
+ this.expPrefixLine = expPrefixLine;
+ this.expSuffixLine = expSuffixLine;
+ this.rules = rules;
+ }
+
+ /** Per-{@code metricsRules}-entry anchors. */
+ @Getter
+ public static final class RuleLines {
+ /** Line of the entry itself — the {@code - name:} anchor. */
+ private final int entryLine;
+ /** Line of the entry's {@code exp:} key. */
+ private final int expLine;
+
+ RuleLines(final int entryLine, final int expLine) {
+ this.entryLine = entryLine;
+ this.expLine = expLine;
+ }
+ }
+
+ /** All-zero index, for callers with no YAML text to inspect. */
+ public static MalYamlLineIndex empty() {
+ return EMPTY;
+ }
+
+ public static MalYamlLineIndex index(final String yamlContent) {
+ return index(yamlContent, DEFAULT_RULES_KEY);
+ }
+
+ /**
+ * @param yamlContent the exact text the rule object was bound from
+ * @param rulesKey key holding the rules sequence — {@link #DEFAULT_RULES_KEY} for standard
+ * MAL files, {@code metrics} for zabbix
+ * @return resolved anchors, or an all-zero index when the document can't be inspected
+ */
+ public static MalYamlLineIndex index(final String yamlContent, final String rulesKey) {
+ if (yamlContent == null || yamlContent.isEmpty()) {
+ return EMPTY;
+ }
+ try (StringReader reader = new StringReader(yamlContent)) {
+ final Node root = new Yaml(new LoaderOptions()).compose(reader);
+ if (!(root instanceof MappingNode)) {
+ return EMPTY;
+ }
+ int filter = 0;
+ int prefix = 0;
+ int suffix = 0;
+ List<RuleLines> ruleLines = Collections.emptyList();
+ for (final NodeTuple tuple : ((MappingNode) root).getValue()) {
+ final String key = scalarKey(tuple.getKeyNode());
+ if (key == null) {
+ continue;
+ }
+ switch (key) {
+ case "filter":
+ filter = lineOf(tuple.getKeyNode());
+ break;
+ case "expPrefix":
+ prefix = lineOf(tuple.getKeyNode());
+ break;
+ case "expSuffix":
+ suffix = lineOf(tuple.getKeyNode());
+ break;
+ default:
+ if (key.equals(rulesKey)) {
+ ruleLines = indexRules(tuple.getValueNode());
+ }
+ break;
+ }
+ }
+ return new MalYamlLineIndex(filter, prefix, suffix, ruleLines);
+ } catch (final RuntimeException e) {
+ // Malformed YAML is the loader's problem to report, not ours — it will surface a far
+ // better message than we could. Degrade to "no lines known".
+ log.debug("MAL YAML line index unavailable: {}", e.getMessage());
+ return EMPTY;
+ }
+ }
+
+ /** Anchors for the rules entry at {@code index}, or an all-zero entry when out of range. */
+ public RuleLines rule(final int index) {
+ if (index < 0 || index >= rules.size()) {
+ return new RuleLines(0, 0);
+ }
+ return rules.get(index);
+ }
+
+ private static List<RuleLines> indexRules(final Node rulesNode) {
+ if (!(rulesNode instanceof SequenceNode)) {
+ return Collections.emptyList();
+ }
+ final List<Node> items = ((SequenceNode) rulesNode).getValue();
+ final List<RuleLines> out = new ArrayList<>(items.size());
+ for (final Node item : items) {
+ int expLine = 0;
+ if (item instanceof MappingNode) {
+ for (final NodeTuple tuple : ((MappingNode) item).getValue()) {
+ if ("exp".equals(scalarKey(tuple.getKeyNode()))) {
+ expLine = lineOf(tuple.getKeyNode());
+ break;
+ }
+ }
+ }
+ out.add(new RuleLines(lineOf(item), expLine));
+ }
+ return out;
+ }
+
+ private static String scalarKey(final Node node) {
+ return node instanceof ScalarNode ? ((ScalarNode) node).getValue() : null;
+ }
+
+ /** SnakeYAML marks are 0-based; every consumer of this class speaks 1-based lines. */
+ private static int lineOf(final Node node) {
+ return node == null || node.getStartMark() == null ? 0 : node.getStartMark().getLine() + 1;
+ }
+}
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/MetricConvert.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/MetricConvert.java
index 4353339755..7e6ce936b0 100644
--- a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/MetricConvert.java
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/MetricConvert.java
@@ -34,6 +34,8 @@ import java.util.stream.Stream;
import lombok.Getter;
import lombok.extern.slf4j.Slf4j;
import org.apache.skywalking.oap.meter.analyzer.v2.compiler.MALScriptParser;
+import org.apache.skywalking.oap.meter.analyzer.v2.compiler.MalSourceMap;
+import org.apache.skywalking.oap.meter.analyzer.v2.compiler.MalSourceRef;
import org.apache.skywalking.oap.meter.analyzer.v2.dsl.FilterExpression;
import org.apache.skywalking.oap.meter.analyzer.v2.dsl.SampleFamily;
import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
@@ -132,8 +134,20 @@ public class MetricConvert {
final List<Analyzer> prepared = IntStream.range(0, rules.size()).mapToObj(
i -> {
final MetricRuleConfig.RuleConfig r = rules.get(i);
- final String yamlSource = sourceName != null
- ? sourceName + ".yaml:" + i : null;
+ // The rule's REAL line, not its position in the list. The two only ever agreed by
+ // accident, which is why a stack trace used to point at the top of the file.
+ // A line is always expected here; -1 marks a resolution failure so it stays
+ // visible in the artifact rather than silently degrading.
+ final String yamlSource = sourceName == null ? null
+ : sourceName + ".yaml:" + (r.getLineNo() > 0 ? r.getLineNo() : MalSourceRef.UNRESOLVED);
+ // Build the source map from the SAME splice formatExp performs, so a stage's
+ // offset in the formatted text resolves to the fragment that authored it.
+ final MALScriptParser.PrefixInjection injection =
+ MALScriptParser.injectExpPrefixTracked(r.getExp(), rule.getExpPrefix());
+ final boolean hasExpSuffix = !Strings.isNullOrEmpty(rule.getExpSuffix());
+ final MalSourceMap sourceMap = MalSourceMap.of(
+ injection, hasExpSuffix,
+ r.getExpLine(), rule.getExpPrefixLine(), rule.getExpSuffixLine());
final Analyzer analyzer = prepareAnalyzer(
formatMetricName(rule, r.getName()),
filter,
@@ -142,8 +156,13 @@ public class MetricConvert {
yamlSource,
pool,
targetClassLoader,
- storageOpt
+ storageOpt,
+ sourceMap
);
+ // The hand-written probes are file-level: the filter has its own line, and the
+ // terminal emit is contributed by expSuffix when one is declared.
+ analyzer.setProbeSourceLines(rule.getFilterLine(),
+ hasExpSuffix ? rule.getExpSuffixLine() : r.getExpLine());
// Stamp the rule's structured fields onto the per-metric
// GateHolder so dsl-debugging records render the rule as
// {metricPrefix, name, filter, exp, expSuffix} alongside
@@ -268,7 +287,8 @@ public class MetricConvert {
final String yamlSource,
final javassist.ClassPool pool,
final ClassLoader targetClassLoader,
- final StorageManipulationOpt storageOpt) {
+ final StorageManipulationOpt storageOpt,
+ final MalSourceMap sourceMap) {
return Analyzer.prepare(
metricsName,
filter,
@@ -277,7 +297,8 @@ public class MetricConvert {
yamlSource,
pool,
targetClassLoader,
- storageOpt
+ storageOpt,
+ sourceMap
);
}
@@ -289,8 +310,10 @@ public class MetricConvert {
return null;
}
final String sourceName = rule.getSourceName();
- final String yamlSource = sourceName != null
- ? sourceName + ".yaml" : null;
+ // The filter has its own line, distinct from any rule's — it is file-level.
+ final String yamlSource = sourceName == null ? null
+ : sourceName + ".yaml:"
+ + (rule.getFilterLine() > 0 ? rule.getFilterLine() : MalSourceRef.UNRESOLVED);
return new FilterExpression(filterText, "filter", yamlSource, pool, targetClassLoader);
}
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/MetricRuleConfig.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/MetricRuleConfig.java
index 42f55eebeb..b95fd32b37 100644
--- a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/MetricRuleConfig.java
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/MetricRuleConfig.java
@@ -58,6 +58,31 @@ public interface MetricRuleConfig {
return null;
}
+ /**
+ * 1-based line of the file-level {@code filter:} key in the source YAML, or {@code 0} when
+ * unknown. A compiled MAL expression splices up to three separate source locations together,
+ * so each needs its own anchor; see {@link MalYamlLineIndex}.
+ *
+ * @return the filter's line, or {@code 0}
+ */
+ default int getFilterLine() {
+ return 0;
+ }
+
+ /**
+ * @return 1-based line of the file-level {@code expPrefix:} key, or {@code 0} when unknown
+ */
+ default int getExpPrefixLine() {
+ return 0;
+ }
+
+ /**
+ * @return 1-based line of the file-level {@code expSuffix:} key, or {@code 0} when unknown
+ */
+ default int getExpSuffixLine() {
+ return 0;
+ }
+
interface RuleConfig {
/**
* Get definition metrics name
@@ -68,5 +93,22 @@ public interface MetricRuleConfig {
* Build metrics MAL
*/
String getExp();
+
+ /**
+ * 1-based line of this rule's entry (its {@code - name:} anchor) in the source YAML, or
+ * {@code 0} when unknown. Used as the provenance label in the generated class name.
+ *
+ * @return the entry's line, or {@code 0}
+ */
+ default int getLineNo() {
+ return 0;
+ }
+
+ /**
+ * @return 1-based line of this rule's {@code exp:} key, or {@code 0} when unknown
+ */
+ default int getExpLine() {
+ return 0;
+ }
}
}
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALBytecodeHelper.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALBytecodeHelper.java
index 7f7322d5c6..90423a19d8 100644
--- a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALBytecodeHelper.java
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALBytecodeHelper.java
@@ -118,7 +118,15 @@ final class MALBytecodeHelper {
final StringBuilder sb = new StringBuilder();
sb.append(MALCodegenHelper.sanitizeName(yamlBase));
if (lineNo != null) {
- sb.append("_L").append(lineNo);
+ // The unresolved marker is -1, which is not legal in a Java identifier; render it as
+ // "unknown" so the failure stays visible in the class name instead of vanishing.
+ int parsed;
+ try {
+ parsed = Integer.parseInt(lineNo.trim());
+ } catch (final NumberFormatException e) {
+ parsed = MalSourceRef.UNRESOLVED;
+ }
+ sb.append("_L").append(MalSourceRef.toIdentifierSegment(parsed));
}
sb.append('_').append(hint);
return sb.toString();
@@ -146,15 +154,21 @@ final class MALBytecodeHelper {
// ==================== Debug output ====================
/**
- * Builds the SourceFile name for a generated class.
- * Example: {@code "(vm.yaml:25)cpu_total.java"}
+ * Builds the {@code SourceFile} name for a generated class.
+ *
+ * <p>This MUST equal the file {@link #writeSourceFile} actually writes — that is the entire
+ * contract an IDE relies on to resolve a stack frame to that source. It previously returned
+ * {@code "(vm.yaml:25)cpu_total.java"} while the file on disk was named after the class
+ * (e.g. {@code vm_L25_cpu_total.java}), so source-attach could never resolve, independent of
+ * whether the embedded line was correct.
+ *
+ * <p>The YAML provenance is NOT repeated here. It already lives in the class name (and hence
+ * in this file name), and the per-statement YAML lines belong to the operator coordinate
+ * carried on {@link MalSourceRef}, not to a per-file bytecode attribute that can only hold
+ * one value for a rule spliced from three source locations.
*/
- String formatSourceFileName(final String metricName) {
- final String classFile = metricName + ".java";
- if (yamlSource != null) {
- return "(" + yamlSource + ")" + classFile;
- }
- return classFile;
+ static String sourceFileNameOf(final CtClass ctClass) {
+ return ctClass.getSimpleName() + ".java";
}
/**
@@ -232,12 +246,28 @@ final class MALBytecodeHelper {
// ==================== Bytecode attributes ====================
/**
- * Adds a {@code LineNumberTable} attribute to the method.
- * Scans bytecode for store instructions to local variable slots
- * >= {@code firstResultSlot}, assigning sequential line numbers.
+ * Adds a {@code LineNumberTable} mapping each bytecode boundary to its line in the
+ * {@code .java} source written for the generated class.
+ *
+ * <p>Boundaries are the first instruction plus the instruction after every store to a local
+ * slot {@code >= firstResultSlot}, which for the variable-per-expression codegen is one per
+ * emitted statement. {@code boundaryLines} must list the corresponding method-relative source
+ * lines in the same order — {@link MalGeneratedSourceLines} derives them from the same
+ * generated text, so the two agree by construction.
+ *
+ * <p>This previously emitted {@code 1,2,3…} — statement ordinals that matched neither the generated
+ * class source nor the rule YAML, so no IDE could resolve a frame.
+ *
+ * @param method the generated method
+ * @param firstResultSlot lowest local slot that holds a statement result
+ * @param statementLines 1-based lines WITHIN THE GENERATED METHOD, in boundary order
+ * @param methodSignatureLineInClass line the method signature occupies in the generated
+ * class source, used to convert method-relative to class-relative
*/
void addLineNumberTable(final javassist.CtMethod method,
- final int firstResultSlot) {
+ final int firstResultSlot,
+ final java.util.List<Integer> statementLines,
+ final int methodSignatureLineInClass) {
try {
final javassist.bytecode.MethodInfo mi = method.getMethodInfo();
final javassist.bytecode.CodeAttribute code =
@@ -246,14 +276,15 @@ final class MALBytecodeHelper {
return;
}
final List<int[]> entries = new ArrayList<>();
- int line = 1;
+ int boundary = 0;
boolean nextIsNewLine = true;
final javassist.bytecode.CodeIterator ci = code.iterator();
while (ci.hasNext()) {
final int pc = ci.next();
if (nextIsNewLine) {
- entries.add(new int[]{pc, line++});
+ entries.add(new int[]{pc, lineInGeneratedClass(statementLines, boundary, methodSignatureLineInClass)});
+ boundary++;
nextIsNewLine = false;
}
final int op = ci.byteAt(pc) & 0xFF;
@@ -271,6 +302,14 @@ final class MALBytecodeHelper {
if (entries.isEmpty()) {
return;
}
+ if (statementLines != null && !statementLines.isEmpty()
+ && entries.size() != statementLines.size()) {
+ // A desync means a codegen change added or removed a statement shape without the
+ // scanner learning about it. Surface it: the numbers would still look plausible.
+ log.warn("MAL LineNumberTable desync for {}: {} bytecode boundaries vs {} source "
+ + "lines. Some frames will resolve to the wrong line in the generated class.",
+ method.getName(), entries.size(), statementLines.size());
+ }
final javassist.bytecode.ConstPool cp = mi.getConstPool();
final byte[] info = new byte[2 + entries.size() * 4];
info[0] = (byte) (entries.size() >> 8);
@@ -290,6 +329,26 @@ final class MALBytecodeHelper {
}
}
+ /**
+ * Sidecar line for boundary {@code i}, or {@link MalSourceRef#UNRESOLVED} when the scanner
+ * produced no line for it. {@code -1} is deliberate: a bogus-but-plausible line is worse than
+ * an obviously absent one, and the JVM tolerates it in a {@code LineNumberTable}.
+ */
+ private static int lineInGeneratedClass(final java.util.List<Integer> statementLines,
+ final int index,
+ final int methodSignatureLineInClass) {
+ if (statementLines == null || index >= statementLines.size()) {
+ return MalSourceRef.UNRESOLVED;
+ }
+ final Integer lineInMethod = statementLines.get(index);
+ if (lineInMethod == null || lineInMethod <= 0) {
+ return MalSourceRef.UNRESOLVED;
+ }
+ // Method-relative line 1 IS the signature line, so the class-relative line counts from
+ // there rather than adding a bare offset.
+ return methodSignatureLineInClass + lineInMethod - 1;
+ }
+
/**
* Adds a {@code LocalVariableTable} attribute for debug info.
*/
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALClassGenerator.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALClassGenerator.java
index 5fdb8c273a..9e83fcccc1 100644
--- a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALClassGenerator.java
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALClassGenerator.java
@@ -114,6 +114,8 @@ public final class MALClassGenerator {
* don't track per-rule hashes).
*/
private String content = "";
+ /** Per-compile source map; see {@link #setSourceMap}. */
+ private MalSourceMap sourceMap = MalSourceMap.EMPTY;
public void setClassOutputDir(final File dir) {
bytecodeHelper.setClassOutputDir(dir);
@@ -123,6 +125,17 @@ public final class MALClassGenerator {
bytecodeHelper.setClassNameHint(hint);
}
+ /**
+ * Per-compile source map, mirroring {@link #setYamlSource}. Resolves each chain stage to the
+ * YAML line it was written on, which differs across a rule spliced from
+ * {@code expPrefix} / {@code exp} / {@code expSuffix}.
+ *
+ * @param sourceMap the rule's map, or {@link MalSourceMap#EMPTY} when lines are unknown
+ */
+ public void setSourceMap(final MalSourceMap sourceMap) {
+ this.sourceMap = sourceMap == null ? MalSourceMap.EMPTY : sourceMap;
+ }
+
public void setYamlSource(final String yamlSource) {
bytecodeHelper.setYamlSource(yamlSource);
}
@@ -200,11 +213,12 @@ public final class MALClassGenerator {
ctClass.addMethod(testMethod);
bytecodeHelper.addLocalVariableTable(testMethod, className,
new String[][]{{paramName, "Ljava/util/Map;"}});
- bytecodeHelper.addLineNumberTable(testMethod, 2);
+ // wrapMalFilterSource has no leading blank before the body, so the signature sits one
+ // line higher than the expression class's: file line 9.
+ bytecodeHelper.addLineNumberTable(testMethod, 2,
+ MalGeneratedSourceLines.statementLinesOf(filterBody), 9);
MALBytecodeHelper.setSourceFile(ctClass,
- bytecodeHelper.formatSourceFileName(
- bytecodeHelper.getClassNameHint() != null
- ? bytecodeHelper.getClassNameHint() : "filter"));
+ MALBytecodeHelper.sourceFileNameOf(ctClass));
bytecodeHelper.writeClassFile(ctClass);
bytecodeHelper.writeSourceFile(ctClass, wrapMalFilterSource(ctClass, filterBody));
@@ -323,6 +337,7 @@ public final class MALClassGenerator {
// 3. Generate run() method — pass metricName as ruleName so probe call sites
// emitted by MALExprCodegen and MALMethodChainCodegen carry it as their first arg.
final MALExprCodegen exprCodegen = new MALExprCodegen(closureFieldNames, metricName);
+ exprCodegen.setSourceLines(sourceMap, sourceMap.getExpYamlLine());
final String runBody = exprCodegen.generateRunMethod(ast);
// 4. Extract metadata
@@ -344,7 +359,15 @@ public final class MALClassGenerator {
ctClass.addMethod(runMethod);
bytecodeHelper.addRunLocalVariableTable(
runMethod, className, exprCodegen.getDeclaredVars());
- bytecodeHelper.addLineNumberTable(runMethod, 1);
+ // Sidecar geometry: writeSourceFile adds a 4-line preamble, then wrapMalExpressionSource
+ // emits package(2) + class decl(2) + one line per closure field + 2 injection members when
+ // enabled + 1 blank, so the run() signature lands at line 10 + C + I of the generated
+ // class source. The helper converts method-relative lines from there.
+ final int runMethodSignatureLineInClass = 10 + closureFieldNames.size()
+ + (org.apache.skywalking.oap.server.core.dsldebug.DSLDebugCodegenSwitch
+ .isInjectionEnabled() ? 2 : 0);
+ bytecodeHelper.addLineNumberTable(runMethod, 1,
+ MalGeneratedSourceLines.statementLinesOf(runBody), runMethodSignatureLineInClass);
final javassist.CtMethod metaMethod =
CtNewMethod.make(metadataBody, ctClass);
@@ -357,7 +380,7 @@ public final class MALClassGenerator {
{"_pct", "[I"}
});
MALBytecodeHelper.setSourceFile(ctClass,
- bytecodeHelper.formatSourceFileName(metricName));
+ MALBytecodeHelper.sourceFileNameOf(ctClass));
// 6. Load companions, then main class
for (final CtClass companion : companionClasses) {
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALClosureCodegen.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALClosureCodegen.java
index e12a6b7fdc..7120f256fb 100644
--- a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALClosureCodegen.java
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALClosureCodegen.java
@@ -626,7 +626,13 @@ final class MALClosureCodegen {
final javassist.CtMethod m = CtNewMethod.make(methodBody, companion);
companion.addMethod(m);
addCompanionLocalVariableTable(m, info);
- bytecodeHelper.addLineNumberTable(m, firstResultSlot(info));
+ // No LineNumberTable for companions, deliberately. Two reasons, both verified against
+ // real artifacts: (1) no `.java` source is written for a companion, so any line would
+ // address a file that does not exist; (2) the store-scan is not a statement-boundary
+ // detector here — a closure body of `tags.put(...)` / `if` / `return` emits no store to
+ // the result slot, so the whole method collapsed to a single entry spanning 200+ bytes.
+ // An absent table lets the JVM report "unknown line", which is honest; the previous
+ // degenerate one attributed every frame in the closure to line 1.
return companion;
}
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALCodegenHelper.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALCodegenHelper.java
index 428b230a76..640486a42f 100644
--- a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALCodegenHelper.java
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALCodegenHelper.java
@@ -114,14 +114,15 @@ final class MALCodegenHelper {
* @param var the local variable holding the {@code SampleFamily} after the stage
*/
static void emitCaptureStage(final StringBuilder sb, final String ruleName,
- final String sourceText, final String var) {
+ final String sourceText, final String var,
+ final int sourceLine) {
if (!DSLDebugCodegenSwitch.isInjectionEnabled()) {
return;
}
sb.append(" if (this.debug.isGateOn()) ").append(MAL_DEBUG_FQCN)
.append(".captureStage(this.debug, \"").append(escapeJava(ruleName))
.append("\", \"").append(escapeJava(sourceText))
- .append("\", ").append(var).append(");\n");
+ .append("\", ").append(var).append(", ").append(sourceLine).append(");\n");
}
/**
@@ -129,14 +130,15 @@ final class MALCodegenHelper {
* metric reference is read from the input {@code samples} map.
*/
static void emitCaptureInput(final StringBuilder sb, final String ruleName,
- final String metricRef, final String var) {
+ final String metricRef, final String var,
+ final int sourceLine) {
if (!DSLDebugCodegenSwitch.isInjectionEnabled()) {
return;
}
sb.append(" if (this.debug.isGateOn()) ").append(MAL_DEBUG_FQCN)
.append(".captureInput(this.debug, \"").append(escapeJava(ruleName))
.append("\", \"").append(escapeJava(metricRef))
- .append("\", ").append(var).append(");\n");
+ .append("\", ").append(var).append(", ").append(sourceLine).append(");\n");
}
// ---- Method classification sets ----
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALExprCodegen.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALExprCodegen.java
index 79f0ef0a78..883fae7127 100644
--- a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALExprCodegen.java
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALExprCodegen.java
@@ -49,6 +49,17 @@ final class MALExprCodegen {
private final List<String> closureFieldNames;
private final MALMethodChainCodegen chainCodegen;
private final String ruleName;
+ /**
+ * Resolves a chain stage's offset in the formatted expression to the YAML line it was
+ * written on — which differs per stage, because {@code exp} / {@code expPrefix} /
+ * {@code expSuffix} are spliced together.
+ */
+ private MalSourceMap sourceMap = MalSourceMap.EMPTY;
+ /**
+ * Line for captures with no offset of their own. A metric reference is always written in
+ * the rule's own {@code exp:}, so that is the right answer for input captures.
+ */
+ private int defaultYamlLine = MalSourceRef.UNRESOLVED;
private int closureFieldIndex;
private int varCounter;
private final Set<String> declaredVars = new HashSet<>();
@@ -130,6 +141,19 @@ final class MALExprCodegen {
/**
* Generates the complete {@code run(Map)} method source.
*/
+ void setSourceLines(final MalSourceMap map, final int expYamlLine) {
+ this.sourceMap = map == null ? MalSourceMap.EMPTY : map;
+ this.defaultYamlLine = expYamlLine > 0 ? expYamlLine : MalSourceRef.UNRESOLVED;
+ }
+
+ MalSourceMap getSourceMap() {
+ return sourceMap;
+ }
+
+ int getDefaultYamlLine() {
+ return defaultYamlLine;
+ }
+
String generateRunMethod(final MALExpressionModel.Expr ast) {
varCounter = 0;
closureFieldIndex = 0;
@@ -197,7 +221,8 @@ final class MALExprCodegen {
.append("\", ").append(SF).append(".EMPTY));\n");
// First read of an input metric — capture the SampleFamily as it enters the chain.
// The metric name itself is the sourceText: the UI shows "input: <metric>".
- MALCodegenHelper.emitCaptureInput(sb, ruleName, expr.getMetricName(), var);
+ MALCodegenHelper.emitCaptureInput(sb, ruleName, expr.getMetricName(), var,
+ defaultYamlLine);
chainCodegen.emitChainStatements(sb, var, expr.getMethodChain());
return var;
}
@@ -243,7 +268,8 @@ final class MALExprCodegen {
// in the same waterfall column as named chain stages. The operator's method name
// (plus, minus, multiply, divide) is the verbatim DSL fragment.
MALCodegenHelper.emitCaptureStage(sb, ruleName,
- MALCodegenHelper.opMethodName(op), resultVar);
+ MALCodegenHelper.opMethodName(op), resultVar,
+ defaultYamlLine);
return resultVar;
}
@@ -318,7 +344,7 @@ final class MALExprCodegen {
final String var = emitExpr(sb, expr.getOperand());
sb.append(" ").append(var).append(" = ")
.append(var).append(".negative();\n");
- MALCodegenHelper.emitCaptureStage(sb, ruleName, "negative", var);
+ MALCodegenHelper.emitCaptureStage(sb, ruleName, "negative", var, defaultYamlLine);
return var;
}
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALExpressionModel.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALExpressionModel.java
index 06ca2b1468..8be02a2d69 100644
--- a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALExpressionModel.java
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALExpressionModel.java
@@ -162,29 +162,49 @@ public final class MALExpressionModel {
* — the probe path tolerates an empty source-text gracefully.
*/
private final String sourceText;
+ /**
+ * Character offset of this call within the FORMATTED expression that was parsed — the
+ * concatenation of {@code expPrefix}, {@code exp} and {@code expSuffix}, not any one of
+ * them. {@code MalSourceMap} turns it into a YAML line by asking which fragment the
+ * offset falls in.
+ *
+ * <p>An offset is used rather than {@link #sourceLine} because that field counts lines
+ * within the concatenated expression, which is almost always 1 and so cannot distinguish
+ * a stage written in {@code exp:} from one written in {@code expSuffix:}.
+ *
+ * <p>{@code -1} for synthesised calls with no source position.
+ */
+ private final int sourceStartIndex;
public MethodCall(final String name, final List<Argument> arguments) {
- this(null, name, arguments, 0, "");
+ this(null, name, arguments, 0, "", -1);
}
public MethodCall(final String namespace, final String name,
final List<Argument> arguments) {
- this(namespace, name, arguments, 0, "");
+ this(namespace, name, arguments, 0, "", -1);
}
public MethodCall(final String namespace, final String name,
final List<Argument> arguments, final int sourceLine) {
- this(namespace, name, arguments, sourceLine, "");
+ this(namespace, name, arguments, sourceLine, "", -1);
}
public MethodCall(final String namespace, final String name,
final List<Argument> arguments, final int sourceLine,
final String sourceText) {
+ this(namespace, name, arguments, sourceLine, sourceText, -1);
+ }
+
+ public MethodCall(final String namespace, final String name,
+ final List<Argument> arguments, final int sourceLine,
+ final String sourceText, final int sourceStartIndex) {
this.namespace = namespace;
this.name = name;
this.arguments = Collections.unmodifiableList(arguments);
this.sourceLine = sourceLine;
this.sourceText = sourceText == null ? "" : sourceText;
+ this.sourceStartIndex = sourceStartIndex;
}
public boolean isExtension() {
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALMethodChainCodegen.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALMethodChainCodegen.java
index 7a2b4bc87c..176b0cbe11 100644
--- a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALMethodChainCodegen.java
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALMethodChainCodegen.java
@@ -52,6 +52,16 @@ final class MALMethodChainCodegen {
this.exprCodegen = exprCodegen;
}
+ /**
+ * YAML line this chain stage was written on. A stage contributed by the file-level
+ * {@code expSuffix:} resolves to the suffix's line, not the rule's — which is the whole
+ * point of carrying an offset on {@link MALExpressionModel.MethodCall}.
+ */
+ private int yamlLineOf(final MALExpressionModel.MethodCall mc) {
+ final int resolved = exprCodegen.getSourceMap().yamlLineOf(mc.getSourceStartIndex());
+ return resolved > 0 ? resolved : exprCodegen.getDefaultYamlLine();
+ }
+
// ==================== Chain codegen ====================
/**
@@ -86,7 +96,7 @@ final class MALMethodChainCodegen {
if (mc.isExtension()) {
emitExtensionCall(sb, var, mc);
- MALCodegenHelper.emitCaptureStage(sb, rule, stageText, var);
+ MALCodegenHelper.emitCaptureStage(sb, rule, stageText, var, yamlLineOf(mc));
continue;
}
// Pre-compute complex expression arguments to temp variables
@@ -98,7 +108,7 @@ final class MALMethodChainCodegen {
sb.append(");\n");
preComputedArgs = null;
// Capture the SampleFamily after this chain stage runs.
- MALCodegenHelper.emitCaptureStage(sb, rule, stageText, var);
+ MALCodegenHelper.emitCaptureStage(sb, rule, stageText, var, yamlLineOf(mc));
}
}
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALScriptParser.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALScriptParser.java
index 43897451be..63f0bbb454 100644
--- a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALScriptParser.java
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MALScriptParser.java
@@ -111,9 +111,70 @@ public final class MALScriptParser {
* {@code expPrefix} is null or empty
*/
public static String injectExpPrefix(final String exp, final String expPrefix) {
+ return injectExpPrefixTracked(exp, expPrefix).getText();
+ }
+
+ /**
+ * Same rewrite as {@link #injectExpPrefix}, but also reports where the injected
+ * {@code expPrefix} text ended up in the result.
+ *
+ * <p>Those ranges are what lets a captured chain stage be attributed to the
+ * {@code expPrefix:} line rather than the {@code exp:} line it is spliced into. The splice is
+ * built left-to-right with a running offset: the original right-to-left {@code replace} loop
+ * produced the same string, but a region's FINAL position shifts by the growth of every
+ * splice before it, so its end position was not recoverable afterwards.
+ *
+ * @param exp the rule's own expression
+ * @param expPrefix file-level prefix to inject at each metric reference
+ * @return the rewritten text plus the final position of each injected prefix
+ */
+ public static PrefixInjection injectExpPrefixTracked(final String exp, final String expPrefix) {
if (expPrefix == null || expPrefix.isEmpty()) {
- return exp;
+ return new PrefixInjection(exp, Collections.emptyList());
+ }
+ final List<int[]> ranges = new ArrayList<>();
+ collectMetricSourceRanges(parseForInjection(exp), ranges);
+ ranges.sort(Comparator.comparingInt((int[] r) -> r[0]));
+
+ final StringBuilder sb = new StringBuilder();
+ final List<int[]> prefixRanges = new ArrayList<>();
+ int cursor = 0;
+ for (final int[] range : ranges) {
+ sb.append(exp, cursor, range[0]);
+ final String name = exp.substring(range[0], range[1] + 1);
+ sb.append('(').append(name).append('.');
+ // The prefix text starts here in the OUTPUT, which is what callers need.
+ final int prefixStart = sb.length();
+ sb.append(expPrefix);
+ prefixRanges.add(new int[]{prefixStart, sb.length()});
+ sb.append(')');
+ cursor = range[1] + 1;
+ }
+ sb.append(exp, cursor, exp.length());
+ return new PrefixInjection(sb.toString(), prefixRanges);
+ }
+
+ /** Rewritten expression plus the output positions of each injected prefix. */
+ public static final class PrefixInjection {
+ private final String text;
+ private final List<int[]> prefixRanges;
+
+ PrefixInjection(final String text, final List<int[]> prefixRanges) {
+ this.text = text;
+ this.prefixRanges = prefixRanges;
+ }
+
+ public String getText() {
+ return text;
+ }
+
+ /** Half-open {@code [start, end)} spans of injected prefix text, ascending. */
+ public List<int[]> getPrefixRanges() {
+ return prefixRanges;
}
+ }
+
+ private static MALParser.ExpressionContext parseForInjection(final String exp) {
final MALLexer lexer = new MALLexer(CharStreams.fromString(exp));
final CommonTokenStream tokens = new CommonTokenStream(lexer);
final MALParser parser = new MALParser(tokens);
@@ -136,16 +197,7 @@ public final class MALScriptParser {
"MAL expression parsing failed while injecting expPrefix: "
+ String.join("; ", errors) + " in expression: " + exp);
}
- final List<int[]> ranges = new ArrayList<>();
- collectMetricSourceRanges(tree, ranges);
- // Splice from right to left so earlier indices remain valid.
- ranges.sort(Comparator.comparingInt((int[] r) -> r[0]).reversed());
- final StringBuilder sb = new StringBuilder(exp);
- for (final int[] range : ranges) {
- final String name = sb.substring(range[0], range[1] + 1);
- sb.replace(range[0], range[1] + 1, "(" + name + "." + expPrefix + ")");
- }
- return sb.toString();
+ return tree;
}
private static void collectMetricSourceRanges(final ParseTree node,
@@ -360,7 +412,8 @@ public final class MALScriptParser {
// Pulled from the input stream so whitespace and identifier spelling are
// exactly what the operator wrote. Used as the captured stage's sourceText.
return new MethodCall(namespace, name, args,
- ctx.getStart().getLine(), rawTextOf(ctx));
+ ctx.getStart().getLine(), rawTextOf(ctx),
+ ctx.getStart().getStartIndex());
}
/**
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalGeneratedSourceLines.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalGeneratedSourceLines.java
new file mode 100644
index 0000000000..e32d0d1fe8
--- /dev/null
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalGeneratedSourceLines.java
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.meter.analyzer.v2.compiler;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+/**
+ * Finds the method-body lines that {@code addLineNumberTable}'s bytecode scan will mark as
+ * boundaries, so the two agree by construction.
+ *
+ * <p>The alternative was recording a line at each of the dozen places the codegen appends a
+ * statement. That drifts: a new emit site added later silently desynchronises the list from the
+ * bytecode, and because the numbers still look plausible the damage is invisible. Deriving both
+ * from the same generated text keeps them in lockstep — the source we scan is source we wrote, and
+ * it is fully deterministic.
+ *
+ * <p><b>What counts as a boundary.</b> {@code addLineNumberTable} emits one entry for the first
+ * instruction, then one after every store to a result slot. So the boundary lines are, in order:
+ * every statement that assigns a result variable, followed by the terminating {@code return}. Any
+ * line that emits no such store — notably the {@code MALDebug.captureXxx(...)} probe calls, which
+ * are present only when debug injection is enabled — is deliberately skipped. That is exactly why
+ * a fixed "statement N is at line N+1" offset cannot be used: enabling injection doubles the
+ * source lines without adding boundaries.
+ */
+final class MalGeneratedSourceLines {
+
+ private MalGeneratedSourceLines() {
+ }
+
+ /**
+ * @param generatedMethodBody the generated method source, starting at its signature line
+ * @return 1-based lines WITHIN THAT METHOD for each statement the bytecode scan will mark,
+ * in order
+ */
+ static List<Integer> statementLinesOf(final String generatedMethodBody) {
+ if (generatedMethodBody == null || generatedMethodBody.isEmpty()) {
+ return Collections.emptyList();
+ }
+ final String[] lines = generatedMethodBody.split("\n", -1);
+ final List<Integer> out = new ArrayList<>();
+ for (int i = 0; i < lines.length; i++) {
+ final String trimmed = lines[i].trim();
+ if (isResultAssignment(trimmed) || trimmed.startsWith("return ")) {
+ // 1-based, and the array index already excludes nothing — line 1 is the signature.
+ out.add(i + 1);
+ }
+ }
+ return out;
+ }
+
+ /**
+ * A result assignment is {@code _var = ...;} or its declaring form {@code SampleFamily _var =
+ * ...;}. Both compile to a store into the result slot; a probe call or a bare expression
+ * statement does not.
+ */
+ private static boolean isResultAssignment(final String trimmed) {
+ if (!trimmed.endsWith(";") || trimmed.startsWith("return ")) {
+ return false;
+ }
+ final int eq = trimmed.indexOf(" = ");
+ if (eq <= 0) {
+ return false;
+ }
+ // The assignment target is the last token before " = ", and every result variable the
+ // codegen mints is prefixed with '_' (MALCodegenHelper's variable-per-expression scheme).
+ final String lhs = trimmed.substring(0, eq);
+ final int lastSpace = lhs.lastIndexOf(' ');
+ final String target = lastSpace < 0 ? lhs : lhs.substring(lastSpace + 1);
+ return target.startsWith("_");
+ }
+}
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalSourceMap.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalSourceMap.java
new file mode 100644
index 0000000000..d0a6501d17
--- /dev/null
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalSourceMap.java
@@ -0,0 +1,135 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.meter.analyzer.v2.compiler;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+/**
+ * Maps a character offset in a formatted MAL expression back to the YAML line it was written on.
+ *
+ * <p>A rule's compiled expression is not one source location. {@code MetricConvert.formatExp}
+ * produces {@code "(" + injectExpPrefix(exp, expPrefix) + ")." + expSuffix}, so a single generated
+ * method can contain stages authored on three different lines of the rule file:
+ *
+ * <pre>
+ * vm.yaml:32 expSuffix: service(['host'], Layer.OS_LINUX)
+ * vm.yaml:38 exp: (node_cpu * 100).sum(['host']).rate('PT1M')
+ *
+ * formatted: ((node_cpu * 100).sum(['host']).rate('PT1M')).service(['host'], ...)
+ * └──────────────── line 38 ─────────────────┘ └──── line 32 ────┘
+ * </pre>
+ *
+ * <p>Without this, every stage would report the rule's line, which is right for the {@code exp:}
+ * body and wrong for everything spliced around it — and wrong in a way that looks plausible,
+ * because the reported line does exist in the file.
+ *
+ * <p>Segments are half-open {@code [start, end)} over the formatted text and are searched in
+ * order, so the {@code expPrefix} spans (which sit INSIDE the {@code exp} span) must be added
+ * first. An offset matching nothing yields {@link MalSourceRef#UNRESOLVED}.
+ */
+public final class MalSourceMap {
+
+ /** Empty map — every lookup is unresolved. For callers with no line information. */
+ public static final MalSourceMap EMPTY =
+ new MalSourceMap(Collections.emptyList(), MalSourceRef.UNRESOLVED);
+
+ private final List<Segment> segments;
+ /**
+ * The rule's own {@code exp:} line. Carried here so callers that need a fallback for a
+ * capture with no offset of its own don't have to thread a second parameter alongside the
+ * map through the whole compile chain.
+ */
+ private final int expYamlLine;
+
+ private MalSourceMap(final List<Segment> segments, final int expYamlLine) {
+ this.segments = segments;
+ this.expYamlLine = expYamlLine;
+ }
+
+ public int getExpYamlLine() {
+ return expYamlLine > 0 ? expYamlLine : MalSourceRef.UNRESOLVED;
+ }
+
+ private static final class Segment {
+ private final int start;
+ private final int end;
+ private final int yamlLine;
+
+ Segment(final int start, final int end, final int yamlLine) {
+ this.start = start;
+ this.end = end;
+ this.yamlLine = yamlLine;
+ }
+ }
+
+ /**
+ * Build the map for one rule.
+ *
+ * @param injection result of splicing {@code expPrefix} into {@code exp}
+ * @param hasExpSuffix whether {@code formatExp} appended {@code ").<expSuffix>"}
+ * @param expLine YAML line of the rule's {@code exp:} key
+ * @param expPrefixLine YAML line of the file-level {@code expPrefix:} key
+ * @param expSuffixLine YAML line of the file-level {@code expSuffix:} key
+ * @return a map over the formatted expression's offsets
+ */
+ public static MalSourceMap of(final MALScriptParser.PrefixInjection injection,
+ final boolean hasExpSuffix,
+ final int expLine,
+ final int expPrefixLine,
+ final int expSuffixLine) {
+ if (injection == null) {
+ return EMPTY;
+ }
+ // formatExp wraps the injected text in "(...)" when a suffix follows, shifting every
+ // offset right by one. Keeping that arithmetic here, next to the layout it mirrors, is
+ // why this class takes the injection rather than the finished string.
+ final int shift = hasExpSuffix ? 1 : 0;
+ final int injectedLength = injection.getText().length();
+
+ final List<Segment> segments = new ArrayList<>();
+ // expPrefix spans first: they are nested inside the exp span and must win the lookup.
+ for (final int[] range : injection.getPrefixRanges()) {
+ segments.add(new Segment(range[0] + shift, range[1] + shift, expPrefixLine));
+ }
+ segments.add(new Segment(shift, shift + injectedLength, expLine));
+ if (hasExpSuffix) {
+ // Everything past the closing ")." belongs to the file-level suffix.
+ segments.add(new Segment(shift + injectedLength + 2, Integer.MAX_VALUE, expSuffixLine));
+ }
+ return new MalSourceMap(segments, expLine);
+ }
+
+ /**
+ * @param offset character offset in the formatted expression, or negative when unknown
+ * @return the 1-based YAML line, or {@link MalSourceRef#UNRESOLVED}
+ */
+ public int yamlLineOf(final int offset) {
+ if (offset < 0) {
+ return MalSourceRef.UNRESOLVED;
+ }
+ for (final Segment segment : segments) {
+ if (offset >= segment.start && offset < segment.end) {
+ return segment.yamlLine > 0 ? segment.yamlLine : MalSourceRef.UNRESOLVED;
+ }
+ }
+ return MalSourceRef.UNRESOLVED;
+ }
+}
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalSourceRef.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalSourceRef.java
new file mode 100644
index 0000000000..77b78391e0
--- /dev/null
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalSourceRef.java
@@ -0,0 +1,111 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.meter.analyzer.v2.compiler;
+
+import lombok.Getter;
+
+/**
+ * The two source coordinates of one generated statement, carried together.
+ *
+ * <p>A compiled MAL rule exists in two independent coordinate spaces, and they must never be
+ * substituted for one another:
+ *
+ * <ul>
+ * <li><b>{@link #getYamlLine() yamlLine}</b> — the OPERATOR coordinate: a line in the rule
+ * {@code .yaml} the operator wrote. Consumed by the dsl-debugging API so a captured sample
+ * can say which line of which rule file produced it. A single generated method mixes
+ * several of these, because {@code expPrefix} / {@code exp} / {@code expSuffix} are
+ * separate locations spliced into one expression.</li>
+ * <li><b>{@link #getGenLine() genLine}</b> — the MACHINE coordinate: a 1-based line within the
+ * generated METHOD BODY, counted as the codegen appends to its buffer. Consumed by the
+ * bytecode {@code LineNumberTable} so an IDE or {@code javap} can resolve a stack frame
+ * back to the generated class's {@code .java} source. It is relative to the METHOD, not the
+ * class file — the class wrapper above the method body is added later, so
+ * {@code MALBytecodeHelper.addLineNumberTable} converts it using the method signature's
+ * line in the class. Converting in exactly one place is what keeps this field meaning one
+ * thing.</li>
+ * </ul>
+ *
+ * <p>They are paired here deliberately. Before this type existed, one overloaded {@code yamlSource}
+ * string served as class label, file identifier and line coordinate at once, which is precisely
+ * how the two spaces got conflated: the {@code LineNumberTable} ended up holding neither a YAML
+ * line nor a line in the generated class, but a statement ordinal that matched no file at all. Keeping both
+ * numbers in one value with distinct accessors makes a mix-up a visible mistake rather than a
+ * silent default.
+ *
+ * <p><b>Unresolved lines.</b> In practice a line is ALWAYS available — every rule is bound from a
+ * YAML document snakeyaml can re-compose, and every statement is emitted into a source buffer we
+ * build ourselves. {@link #UNRESOLVED} therefore marks a defensive fallback, not a normal state,
+ * and is deliberately {@code -1} rather than {@code 0} or an omitted value: a silently missing
+ * line hides the failure, whereas {@code -1} propagates visibly all the way into the class name
+ * and the {@code SourceFile} attribute, where it can be spotted and grepped for. If you see
+ * {@code -1} in a generated artifact, line resolution broke — that is a bug to chase, not a
+ * supported mode.
+ */
+@Getter
+public final class MalSourceRef {
+
+ /** Marker for "a line was expected here but could not be resolved". */
+ public static final int UNRESOLVED = -1;
+
+ /** Neither coordinate resolved. */
+ public static final MalSourceRef UNKNOWN = new MalSourceRef(UNRESOLVED, UNRESOLVED);
+
+ private final int yamlLine;
+ private final int genLine;
+
+ private MalSourceRef(final int yamlLine, final int genLine) {
+ this.yamlLine = yamlLine;
+ this.genLine = genLine;
+ }
+
+ /**
+ * Non-positive inputs normalise to {@link #UNRESOLVED}, so callers may pass through a
+ * {@code 0} from an older accessor without it being mistaken for a real line.
+ *
+ * @param yamlLine 1-based line in the source rule YAML
+ * @param genLine 1-based line within the generated method body
+ * @return the paired coordinates
+ */
+ public static MalSourceRef of(final int yamlLine, final int genLine) {
+ final int yaml = yamlLine > 0 ? yamlLine : UNRESOLVED;
+ final int gen = genLine > 0 ? genLine : UNRESOLVED;
+ if (yaml == UNRESOLVED && gen == UNRESOLVED) {
+ return UNKNOWN;
+ }
+ return new MalSourceRef(yaml, gen);
+ }
+
+ /**
+ * Render a line for use inside a generated Java identifier. {@code -1} is not legal in an
+ * identifier, so an unresolved line becomes the literal {@code unknown} — still visible and
+ * greppable in a class name, but syntactically valid.
+ *
+ * @param line a line in either coordinate space
+ * @return the identifier-safe rendering
+ */
+ public static String toIdentifierSegment(final int line) {
+ return line > 0 ? Integer.toString(line) : "unknown";
+ }
+
+ @Override
+ public String toString() {
+ return "MalSourceRef(yaml=" + yamlLine + ", gen=" + genLine + ")";
+ }
+}
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/DSL.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/DSL.java
index 65a252b5eb..4cf669b0a4 100644
--- a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/DSL.java
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsl/DSL.java
@@ -18,6 +18,7 @@
package org.apache.skywalking.oap.meter.analyzer.v2.dsl;
import javassist.ClassPool;
+import org.apache.skywalking.oap.meter.analyzer.v2.compiler.MalSourceMap;
import lombok.extern.slf4j.Slf4j;
import org.apache.skywalking.oap.meter.analyzer.v2.compiler.MALClassGenerator;
import org.apache.skywalking.oap.meter.analyzer.v2.compiler.MALExpressionModel;
@@ -95,6 +96,23 @@ public final class DSL {
final String yamlSource,
final ClassPool pool,
final ClassLoader targetClassLoader) {
+ return parse(metricName, expression, yamlSource, pool, targetClassLoader,
+ MalSourceMap.EMPTY);
+ }
+
+ /**
+ * Source-mapped overload. {@code sourceMap} resolves each chain stage to the YAML line it was
+ * written on, so a stage contributed by the file-level {@code expSuffix:} is reported against
+ * the suffix's line rather than the rule's.
+ *
+ * @param sourceMap per-rule map, or {@link MalSourceMap#EMPTY} when lines are unknown
+ */
+ public static Expression parse(final String metricName,
+ final String expression,
+ final String yamlSource,
+ final ClassPool pool,
+ final ClassLoader targetClassLoader,
+ final MalSourceMap sourceMap) {
try {
final MalExpression malExpr;
if (pool != null && targetClassLoader != null) {
@@ -103,6 +121,7 @@ public final class DSL {
// yamlSource state that the shared GENERATOR carries between calls.
final MALClassGenerator perFile = new MALClassGenerator(pool, targetClassLoader);
perFile.setYamlSource(yamlSource);
+ perFile.setSourceMap(sourceMap);
// The verbatim MAL expression text is the rule's "content" — threaded
// into the GateHolder so dsl-debugging records carry the rule source
// inline. Same string ANTLR parsed; no extra plumbing.
@@ -110,6 +129,7 @@ public final class DSL {
malExpr = perFile.compile(metricName, expression);
} else {
GENERATOR.setYamlSource(yamlSource);
+ GENERATOR.setSourceMap(sourceMap);
GENERATOR.setContent(expression);
malExpr = GENERATOR.compile(metricName, expression);
}
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsldebug/MALDebug.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsldebug/MALDebug.java
index 4e8fe4f862..45ba0bea19 100644
--- a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsldebug/MALDebug.java
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsldebug/MALDebug.java
@@ -85,8 +85,9 @@ public final class MALDebug {
}
public static void captureInput(final GateHolder holder, final String rule,
- final String metricRef, final SampleFamily family) {
- fanOut(holder, r -> r.appendInput(rule, metricRef, family));
+ final String metricRef, final SampleFamily family,
+ final int sourceLine) {
+ fanOut(holder, r -> r.appendInput(rule, metricRef, family, sourceLine));
}
/**
@@ -103,25 +104,28 @@ public final class MALDebug {
public static void captureFilter(final GateHolder holder, final String rule,
final String filterText,
final Map<String, SampleFamily> surviving,
- final boolean kept) {
- fanOut(holder, r -> r.appendFilter(rule, filterText, surviving, kept));
+ final boolean kept, final int sourceLine) {
+ fanOut(holder, r -> r.appendFilter(rule, filterText, surviving, kept, sourceLine));
}
public static void captureStage(final GateHolder holder, final String rule,
- final String sourceText, final SampleFamily family) {
- fanOut(holder, r -> r.appendStage(rule, sourceText, family));
+ final String sourceText, final SampleFamily family,
+ final int sourceLine) {
+ fanOut(holder, r -> r.appendStage(rule, sourceText, family, sourceLine));
}
public static void captureDownsample(final GateHolder holder, final String rule,
final String function, final String origin,
- final SampleFamily family) {
- fanOut(holder, r -> r.appendDownsample(rule, function, origin, family));
+ final SampleFamily family, final int sourceLine) {
+ fanOut(holder, r -> r.appendDownsample(rule, function, origin, family, sourceLine));
}
public static void captureMeterEmit(final GateHolder holder, final String rule,
final MeterEntity entity, final String metricName,
- final AcceptableValue<?> value, final long timeBucket) {
- fanOut(holder, r -> r.appendMeterEmit(rule, entity, metricName, value, timeBucket));
+ final AcceptableValue<?> value, final long timeBucket,
+ final int sourceLine) {
+ fanOut(holder, r -> r.appendMeterEmit(rule, entity, metricName, value, timeBucket,
+ sourceLine));
}
private static void fanOut(final GateHolder holder, final Dispatch dispatch) {
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsldebug/MALDebugRecorder.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsldebug/MALDebugRecorder.java
index ecd67a84fa..1759e5b61e 100644
--- a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsldebug/MALDebugRecorder.java
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/dsldebug/MALDebugRecorder.java
@@ -68,7 +68,7 @@ public interface MALDebugRecorder extends DebugRecorder {
}
/** First read of an input metric reference at the top of run(). */
- void appendInput(String rule, String metricRef, SampleFamily family);
+ void appendInput(String rule, String metricRef, SampleFamily family, int sourceLine);
/**
* File-level filter probe. {@code surviving} carries every
@@ -78,13 +78,15 @@ public interface MALDebugRecorder extends DebugRecorder {
* kept = true} when at least one family survived, {@code false}
* when the rule short-circuits post-filter.
*/
- void appendFilter(String rule, String filterText, Map<String, SampleFamily> surviving, boolean kept);
+ void appendFilter(String rule, String filterText, Map<String, SampleFamily> surviving,
+ boolean kept, int sourceLine);
/** Per-stage capture (chain method, binary op, unary neg, etc.). */
- void appendStage(String rule, String sourceText, SampleFamily family);
+ void appendStage(String rule, String sourceText, SampleFamily family, int sourceLine);
/** Downsample stage (rate / irate / increase / etc.). */
- void appendDownsample(String rule, String function, String origin, SampleFamily family);
+ void appendDownsample(String rule, String function, String origin, SampleFamily family,
+ int sourceLine);
/**
* Captures the {@code (value, timeBucket)} pair the ingest path is about to push —
@@ -97,5 +99,5 @@ public interface MALDebugRecorder extends DebugRecorder {
* (unlike OAL where {@code appendBuild} fires before the entry function runs).
*/
void appendMeterEmit(String rule, MeterEntity entity, String metricName,
- AcceptableValue<?> value, long timeBucket);
+ AcceptableValue<?> value, long timeBucket, int sourceLine);
}
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/prometheus/rule/MetricsRule.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/prometheus/rule/MetricsRule.java
index 0a10499f9c..6f610967d3 100644
--- a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/prometheus/rule/MetricsRule.java
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/prometheus/rule/MetricsRule.java
@@ -21,6 +21,8 @@ package org.apache.skywalking.oap.meter.analyzer.v2.prometheus.rule;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
+import lombok.EqualsAndHashCode;
+import lombok.ToString;
import lombok.NoArgsConstructor;
import org.apache.skywalking.oap.meter.analyzer.v2.MetricRuleConfig;
@@ -34,4 +36,16 @@ import org.apache.skywalking.oap.meter.analyzer.v2.MetricRuleConfig;
public class MetricsRule implements MetricRuleConfig.RuleConfig {
private String name;
private String exp;
+ /**
+ * Source anchors, stamped by the loader from {@code MalYamlLineIndex} — not YAML-bound keys.
+ * Excluded from equals/hashCode/toString so a rule's identity stays its content: the
+ * runtime-rule delta classifier compares parsed rules, and a rule that merely moved down the
+ * file must not read as changed.
+ */
+ @EqualsAndHashCode.Exclude
+ @ToString.Exclude
+ private int lineNo;
+ @EqualsAndHashCode.Exclude
+ @ToString.Exclude
+ private int expLine;
}
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/prometheus/rule/Rule.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/prometheus/rule/Rule.java
index 47497a964f..827fb33697 100644
--- a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/prometheus/rule/Rule.java
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/prometheus/rule/Rule.java
@@ -19,6 +19,8 @@
package org.apache.skywalking.oap.meter.analyzer.v2.prometheus.rule;
import lombok.Data;
+import lombok.EqualsAndHashCode;
+import lombok.ToString;
import lombok.NoArgsConstructor;
import org.apache.skywalking.oap.meter.analyzer.v2.MetricRuleConfig;
import org.apache.skywalking.oap.server.core.analysis.LayerDefinition;
@@ -43,6 +45,20 @@ public class Rule implements MetricRuleConfig {
* rule file is self-describing for any custom layers it references.
*/
private List<LayerDefinition> layerDefinitions;
+ /**
+ * File-level source anchors, stamped by the loader from {@code MalYamlLineIndex} — not
+ * YAML-bound keys. Excluded from equals/hashCode/toString for the same reason as
+ * {@link MetricsRule}'s: a rule file that merely shifted must not read as content-changed.
+ */
+ @EqualsAndHashCode.Exclude
+ @ToString.Exclude
+ private int filterLine;
+ @EqualsAndHashCode.Exclude
+ @ToString.Exclude
+ private int expPrefixLine;
+ @EqualsAndHashCode.Exclude
+ @ToString.Exclude
+ private int expSuffixLine;
@Override
public String getSourceName() {
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/prometheus/rule/RuleSourceLines.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/prometheus/rule/RuleSourceLines.java
new file mode 100644
index 0000000000..2a56d84143
--- /dev/null
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/prometheus/rule/RuleSourceLines.java
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.meter.analyzer.v2.prometheus.rule;
+
+import java.util.List;
+import org.apache.skywalking.oap.meter.analyzer.v2.MalYamlLineIndex;
+
+/**
+ * Stamps YAML source anchors onto a parsed {@link Rule}.
+ *
+ * <p>Exists so every loader that binds a rule file — the disk loader, the runtime-rule hot-update
+ * applier, and the class-generation tooling — assigns lines the same way. They previously agreed
+ * only by coincidence: the test harness scanned for {@code name:} text while production used the
+ * rules-list index, so the two produced different numbers for the same rule.
+ *
+ * <p>Callers MUST pass the exact text the rule was bound from. Re-reading the file would risk
+ * indexing a different revision than the one in memory.
+ */
+public final class RuleSourceLines {
+
+ private RuleSourceLines() {
+ }
+
+ /**
+ * Assign file-level and per-rule anchors. No-op when either argument is null, and individual
+ * anchors stay {@code 0} when the document doesn't declare them.
+ *
+ * @param rule parsed rule to stamp in place
+ * @param yamlContent the exact text {@code rule} was bound from
+ */
+ public static void assign(final Rule rule, final String yamlContent) {
+ if (rule == null || yamlContent == null) {
+ return;
+ }
+ final MalYamlLineIndex index = MalYamlLineIndex.index(yamlContent);
+ rule.setFilterLine(index.getFilterLine());
+ rule.setExpPrefixLine(index.getExpPrefixLine());
+ rule.setExpSuffixLine(index.getExpSuffixLine());
+
+ final List<MetricsRule> rules = rule.getMetricsRules();
+ if (rules == null) {
+ return;
+ }
+ // Positional: the index walks the same sequence snakeyaml bound, in the same order.
+ for (int i = 0; i < rules.size(); i++) {
+ final MetricsRule metricsRule = rules.get(i);
+ if (metricsRule == null) {
+ continue;
+ }
+ final MalYamlLineIndex.RuleLines lines = index.rule(i);
+ metricsRule.setLineNo(lines.getEntryLine());
+ metricsRule.setExpLine(lines.getExpLine());
+ }
+ }
+}
diff --git a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/prometheus/rule/Rules.java b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/prometheus/rule/Rules.java
index 4b0e369e1e..439c0e55d3 100644
--- a/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/prometheus/rule/Rules.java
+++ b/oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/v2/prometheus/rule/Rules.java
@@ -18,11 +18,10 @@
package org.apache.skywalking.oap.meter.analyzer.v2.prometheus.rule;
-import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.IOException;
-import java.io.InputStreamReader;
+import java.io.StringReader;
import java.io.Reader;
import java.nio.charset.StandardCharsets;
import java.nio.file.FileSystems;
@@ -145,12 +144,16 @@ public class Rules {
}
private static Rule parseRule(final String ruleName, final byte[] bytes) {
- try (Reader r = new InputStreamReader(new ByteArrayInputStream(bytes), StandardCharsets.UTF_8)) {
+ // Decode once and bind from the String: RuleSourceLines re-composes the SAME text to read
+ // positional marks, which snakeyaml's bean binding discards.
+ final String text = new String(bytes, StandardCharsets.UTF_8);
+ try (Reader r = new StringReader(text)) {
Rule rule = new Yaml().loadAs(r, Rule.class);
if (rule == null) {
return null;
}
rule.setName(ruleName);
+ RuleSourceLines.assign(rule, text);
registerInlineLayers(ruleName, rule);
return rule;
} catch (IOException e) {
diff --git a/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/MalProbeSourceLineTest.java b/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/MalProbeSourceLineTest.java
new file mode 100644
index 0000000000..8c9cf76f90
--- /dev/null
+++ b/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/MalProbeSourceLineTest.java
@@ -0,0 +1,139 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.meter.analyzer.v2;
+
+import java.util.List;
+import org.apache.skywalking.oap.meter.analyzer.v2.compiler.MALScriptParser;
+import org.apache.skywalking.oap.meter.analyzer.v2.compiler.MalSourceMap;
+import org.apache.skywalking.oap.meter.analyzer.v2.compiler.MalSourceRef;
+import org.apache.skywalking.oap.meter.analyzer.v2.prometheus.rule.Rule;
+import org.apache.skywalking.oap.meter.analyzer.v2.prometheus.rule.RuleSourceLines;
+import org.junit.jupiter.api.Test;
+import org.yaml.snakeyaml.Yaml;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * End-to-end check that a real rule file yields the right YAML line for each stage of a compiled
+ * expression — the operator coordinate that the dsl-debugging API reports.
+ *
+ * <p>This is the case the whole milestone turns on. A rule's compiled expression is a splice of
+ * up to three source locations, so "the rule's line" is the right answer for the {@code exp:}
+ * body and the WRONG answer for anything contributed by the file-level {@code expPrefix:} or
+ * {@code expSuffix:} — wrong in a way that looks correct, because the line it names does exist.
+ */
+class MalProbeSourceLineTest {
+
+ /** Same shape as a shipped rule file. Line numbers are in the comments and are asserted. */
+ private static final String YAML =
+ "filter: \"{ tags -> tags.job == 'vm' }\"\n" // 1
+ + "expSuffix: service(['host'], Layer.GENERAL)\n" // 2
+ + "metricPrefix: meter_vm\n" // 3
+ + "metricsRules:\n" // 4
+ + " - name: cpu_total\n" // 5
+ + " exp: node_cpu.sum(['host']).rate('PT1M')\n" // 6
+ + " - name: mem_used\n" // 7
+ + " exp: node_mem.sum(['host'])\n"; // 8
+
+ private static Rule parse(final String yaml) {
+ final Rule rule = new Yaml().loadAs(yaml, Rule.class);
+ rule.setName("vm");
+ RuleSourceLines.assign(rule, yaml);
+ return rule;
+ }
+
+ /** Rebuilds exactly what MetricConvert does, so the assertions bind to production behaviour. */
+ private static MalSourceMap mapFor(final Rule rule, final int ruleIndex) {
+ final MetricRuleConfig.RuleConfig r = rule.getMetricsRules().get(ruleIndex);
+ final MALScriptParser.PrefixInjection injection =
+ MALScriptParser.injectExpPrefixTracked(r.getExp(), rule.getExpPrefix());
+ final boolean hasSuffix = rule.getExpSuffix() != null && !rule.getExpSuffix().isEmpty();
+ return MalSourceMap.of(injection, hasSuffix,
+ r.getExpLine(), rule.getExpPrefixLine(), rule.getExpSuffixLine());
+ }
+
+ @Test
+ void anchorsComeFromTheFileNotTheRuleOrder() {
+ final Rule rule = parse(YAML);
+
+ assertEquals(1, rule.getFilterLine());
+ assertEquals(2, rule.getExpSuffixLine());
+ // Second rule: its exp is on line 8, not "index 1".
+ assertEquals(6, rule.getMetricsRules().get(0).getExpLine());
+ assertEquals(8, rule.getMetricsRules().get(1).getExpLine());
+ assertEquals(5, rule.getMetricsRules().get(0).getLineNo());
+ assertEquals(7, rule.getMetricsRules().get(1).getLineNo());
+ }
+
+ @Test
+ void stagesInExpReportTheRuleLineAndSuffixStagesReportTheSuffixLine() {
+ final Rule rule = parse(YAML);
+ final MalSourceMap map = mapFor(rule, 0);
+ final String exp = rule.getMetricsRules().get(0).getExp();
+
+ // formatExp = "(" + exp + ")." + expSuffix, so exp offsets shift by 1.
+ assertEquals(6, map.yamlLineOf(1 + exp.indexOf("sum")), ".sum() is written in exp:");
+ assertEquals(6, map.yamlLineOf(1 + exp.indexOf("rate")), ".rate() is written in exp:");
+ // The stage the file-level suffix contributes must NOT claim the rule's line.
+ assertEquals(2, map.yamlLineOf(1 + exp.length() + 2), ".service() comes from expSuffix:");
+ }
+
+ @Test
+ void eachRuleResolvesToItsOwnLine() {
+ final Rule rule = parse(YAML);
+
+ assertEquals(6, mapFor(rule, 0).getExpYamlLine());
+ assertEquals(8, mapFor(rule, 1).getExpYamlLine());
+ }
+
+ @Test
+ void prefixDerivedStagesReportThePrefixLine() {
+ final String withPrefix =
+ "expPrefix: tag({tags -> tags.k = 'v'})\n" // 1
+ + "expSuffix: service(['host'], Layer.GENERAL)\n" // 2
+ + "metricPrefix: meter_vm\n" // 3
+ + "metricsRules:\n" // 4
+ + " - name: cpu_total\n" // 5
+ + " exp: node_cpu.sum(['host'])\n"; // 6
+ final Rule rule = parse(withPrefix);
+ assertEquals(1, rule.getExpPrefixLine());
+
+ final MalSourceMap map = mapFor(rule, 0);
+ final List<int[]> prefixRanges = MALScriptParser
+ .injectExpPrefixTracked(rule.getMetricsRules().get(0).getExp(), rule.getExpPrefix())
+ .getPrefixRanges();
+ assertTrue(prefixRanges.size() > 0, "prefix should be injected at the metric ref");
+
+ // +1 for the paren formatExp adds ahead of the suffix.
+ assertEquals(1, map.yamlLineOf(prefixRanges.get(0)[0] + 1),
+ "a stage from expPrefix must report the prefix's line, not the rule's");
+ }
+
+ @Test
+ void aRuleFileWithoutResolvableLinesReportsUnresolvedNotZero() {
+ final Rule rule = new Yaml().loadAs(
+ "metricPrefix: m\nmetricsRules:\n - name: a\n exp: x.sum(['h'])\n", Rule.class);
+ rule.setName("vm");
+ // Deliberately NOT stamped — simulates a loader that could not resolve marks.
+
+ assertEquals(MalSourceRef.UNRESOLVED, mapFor(rule, 0).getExpYamlLine());
+ assertEquals(MalSourceRef.UNRESOLVED, mapFor(rule, 0).yamlLineOf(0));
+ }
+}
diff --git a/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/MalYamlLineIndexTest.java b/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/MalYamlLineIndexTest.java
new file mode 100644
index 0000000000..79917612ab
--- /dev/null
+++ b/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/MalYamlLineIndexTest.java
@@ -0,0 +1,127 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.meter.analyzer.v2;
+
+import org.junit.jupiter.api.Test;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+/**
+ * Pins the 1-based line contract. SnakeYAML marks are 0-based, so an off-by-one here would
+ * silently mis-attribute every generated class label and every debug sample.
+ */
+class MalYamlLineIndexTest {
+
+ /** Mirrors the shape of a shipped rule file: file-level keys, then a rules sequence. */
+ private static final String YAML =
+ "# comment line 1\n" // 1
+ + "\n" // 2
+ + "filter: \"{ tags -> true }\"\n" // 3
+ + "expSuffix: service(['host'])\n" // 4
+ + "metricPrefix: meter_vm\n" // 5
+ + "metricsRules:\n" // 6
+ + "\n" // 7
+ + " # a comment inside the list\n" // 8
+ + " - name: cpu_total\n" // 9
+ + " exp: node_cpu.sum(['host'])\n" // 10
+ + " - name: mem_used\n" // 11
+ + " exp: node_mem.sum(['host'])\n"; // 12
+
+ @Test
+ void resolvesFileLevelAnchors() {
+ final MalYamlLineIndex index = MalYamlLineIndex.index(YAML);
+
+ assertEquals(3, index.getFilterLine());
+ assertEquals(4, index.getExpSuffixLine());
+ // Not declared in this document.
+ assertEquals(0, index.getExpPrefixLine());
+ }
+
+ @Test
+ void resolvesPerRuleAnchorsSkippingBlanksAndComments() {
+ final MalYamlLineIndex index = MalYamlLineIndex.index(YAML);
+
+ // The entry anchor is the `- name:` line, not the `metricsRules:` key and not the
+ // blank/comment lines between them.
+ assertEquals(9, index.rule(0).getEntryLine());
+ assertEquals(10, index.rule(0).getExpLine());
+ assertEquals(11, index.rule(1).getEntryLine());
+ assertEquals(12, index.rule(1).getExpLine());
+ }
+
+ @Test
+ void outOfRangeRuleIsZeroNotAnError() {
+ final MalYamlLineIndex index = MalYamlLineIndex.index(YAML);
+
+ assertEquals(0, index.rule(99).getEntryLine());
+ assertEquals(0, index.rule(-1).getExpLine());
+ }
+
+ @Test
+ void honoursAnAlternateRulesKey() {
+ // Zabbix rule files hold their rules under `metrics:` rather than `metricsRules:`.
+ final String zabbix = "metricPrefix: meter_zb\n" // 1
+ + "metrics:\n" // 2
+ + " - name: cpu\n" // 3
+ + " exp: agent_cpu.sum(['host'])\n"; // 4
+
+ assertEquals(3, MalYamlLineIndex.index(zabbix, "metrics").rule(0).getEntryLine());
+ assertEquals(4, MalYamlLineIndex.index(zabbix, "metrics").rule(0).getExpLine());
+ // The default key finds nothing in that document — zeros, not an exception.
+ assertEquals(0, MalYamlLineIndex.index(zabbix).rule(0).getEntryLine());
+ }
+
+ @Test
+ void malformedOrEmptyInputDegradesToZeros() {
+ // A missing line number must never stop a rule from compiling.
+ assertEquals(0, MalYamlLineIndex.index(null).getFilterLine());
+ assertEquals(0, MalYamlLineIndex.index("").getFilterLine());
+ assertEquals(0, MalYamlLineIndex.index("- not: a mapping root").getFilterLine());
+ assertEquals(0, MalYamlLineIndex.index("key: [unclosed").getFilterLine());
+ }
+
+ @Test
+ void blockScalarsAnchorOnTheirKeyLineNotTheirContent() {
+ // Shipped rules use both folded (`>`) and literal (`|-`) scalars, and a real one spans
+ // 30 lines (meter-analyzer-config/network-profiling.yaml). Everything the fragment
+ // contributes is attributed to the KEY line, which is the stable anchor an operator
+ // navigates to — the content lines have no individual meaning to a chain stage.
+ final String yaml =
+ "expSuffix: |-\n" // 1 key
+ + " service(['host'],\n" // 2 content
+ + " Layer.GENERAL)\n" // 3 content
+ + "metricPrefix: meter_vm\n" // 4
+ + "metricsRules:\n" // 5
+ + " - name: cpu\n" // 6
+ + " exp: >\n" // 7 key
+ + " node_cpu\n" // 8 content
+ + " .sum(['host'])\n" // 9 content
+ + " - name: mem\n" // 10
+ + " exp: node_mem\n"; // 11
+
+ final MalYamlLineIndex index = MalYamlLineIndex.index(yaml);
+
+ assertEquals(1, index.getExpSuffixLine(), "multi-line expSuffix anchors on its key");
+ assertEquals(6, index.rule(0).getEntryLine());
+ assertEquals(7, index.rule(0).getExpLine(), "folded exp anchors on its key, not line 8");
+ // The rule AFTER a multi-line scalar must not be shifted by the scalar's height.
+ assertEquals(10, index.rule(1).getEntryLine());
+ assertEquals(11, index.rule(1).getExpLine());
+ }
+}
diff --git a/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalGeneratedSourceLinesTest.java b/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalGeneratedSourceLinesTest.java
new file mode 100644
index 0000000000..5900974e1b
--- /dev/null
+++ b/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalGeneratedSourceLinesTest.java
@@ -0,0 +1,92 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.meter.analyzer.v2.compiler;
+
+import java.util.Arrays;
+import java.util.List;
+import org.junit.jupiter.api.Test;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * The boundary list must match, one-for-one and in order, the entries
+ * {@code MALBytecodeHelper.addLineNumberTable} derives from the bytecode. A mismatch would put
+ * real-looking but wrong line numbers in the table, which is worse than the sequential ordinals
+ * it replaces.
+ */
+class MalGeneratedSourceLinesTest {
+
+ /** Verbatim shape of a real generated run() body — the vm.yaml cpu_total_percentage rule. */
+ private static final String RUN_BODY =
+ "public SampleFamily run(java.util.Map samples) {\n" // 1 signature
+ + " SampleFamily _node_cpu = ((SampleFamily) samples.get(\"x\"));\n" // 2 declare+store
+ + " _node_cpu = _node_cpu.multiply(Long.valueOf(100L));\n" // 3 store
+ + " _node_cpu = _node_cpu.tagNotEqual(new String[]{\"m\", \"i\"});\n" // 4 store
+ + " _node_cpu = _node_cpu.sum(java.util.Arrays.asList(new String[]{\"h\"}));\n" // 5
+ + " _node_cpu = _node_cpu.rate(\"PT1M\");\n" // 6 store
+ + " _node_cpu = _node_cpu.service(java.util.Arrays.asList(x), y);\n" // 7 store
+ + " return _node_cpu;\n" // 8 return
+ + "}\n"; // 9
+
+ @Test
+ void findsEveryAssignmentPlusTheReturn() {
+ final List<Integer> lines = MalGeneratedSourceLines.statementLinesOf(RUN_BODY);
+
+ // 6 assignments (lines 2-7) + the return (line 8) = 7 boundaries, matching the 7 entries
+ // javap shows for this rule today.
+ assertEquals(Arrays.asList(2, 3, 4, 5, 6, 7, 8), lines);
+ }
+
+ @Test
+ void skipsProbeCallsWhichEmitNoStore() {
+ // With SW_DSL_DEBUGGING_INJECTION_ENABLED the codegen interleaves probe calls. They add
+ // source lines but NO bytecode boundary, which is precisely why a fixed "statement N is at
+ // line N+1" offset cannot be used.
+ final String withProbes =
+ "public SampleFamily run(java.util.Map samples) {\n" // 1
+ + " MALDebug.captureInput(this.debug, \"r\", \"m\", _v);\n" // 2 no store
+ + " SampleFamily _v = ((SampleFamily) samples.get(\"x\"));\n" // 3 store
+ + " MALDebug.captureStage(this.debug, \"r\", \"sum\", _v);\n" // 4 no store
+ + " _v = _v.sum(java.util.Arrays.asList(new String[]{\"h\"}));\n" // 5 store
+ + " return _v;\n" // 6 return
+ + "}\n";
+
+ assertEquals(Arrays.asList(3, 5, 6), MalGeneratedSourceLines.statementLinesOf(withProbes));
+ }
+
+ @Test
+ void ignoresNonResultLocalsAndBareExpressions() {
+ final String mixed =
+ "public SampleFamily run(java.util.Map samples) {\n" // 1
+ + " java.util.List other = new java.util.ArrayList();\n" // 2 not a _ target
+ + " other.add(\"x\");\n" // 3 bare expression
+ + " SampleFamily _v = SampleFamily.EMPTY;\n" // 4 store
+ + " return _v;\n" // 5 return
+ + "}\n";
+
+ assertEquals(Arrays.asList(4, 5), MalGeneratedSourceLines.statementLinesOf(mixed));
+ }
+
+ @Test
+ void emptyInputIsEmptyNotAnError() {
+ assertTrue(MalGeneratedSourceLines.statementLinesOf(null).isEmpty());
+ assertTrue(MalGeneratedSourceLines.statementLinesOf("").isEmpty());
+ }
+}
diff --git a/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalLineAttributionTest.java b/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalLineAttributionTest.java
new file mode 100644
index 0000000000..363f5efcec
--- /dev/null
+++ b/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalLineAttributionTest.java
@@ -0,0 +1,181 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.meter.analyzer.v2.compiler;
+
+import java.io.BufferedInputStream;
+import java.io.DataInputStream;
+import java.io.File;
+import java.io.FileInputStream;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.util.ArrayList;
+import java.util.List;
+import javassist.ClassPool;
+import javassist.bytecode.ClassFile;
+import javassist.bytecode.CodeAttribute;
+import javassist.bytecode.LineNumberAttribute;
+import javassist.bytecode.MethodInfo;
+import org.apache.skywalking.oap.meter.analyzer.v2.dsl.MalExpression;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * End-to-end guard on the generated artifact's line attribution.
+ *
+ * <p>The unit tests for {@link MalYamlLineIndex} and {@link MalGeneratedSourceLines} check those
+ * pieces in isolation. This one checks the thing that actually ships: that the {@code SourceFile}
+ * attribute names the file that was really written, and that every {@code LineNumberTable} entry
+ * indexes a line of THAT file which genuinely holds a statement.
+ *
+ * <p>It exists because the arithmetic converting a method-relative line to a class-relative one
+ * depends on the class wrapper's shape — the number of closure fields and whether debug injection
+ * is enabled. That is a constant derived by hand, so it needs a test that would fail if the
+ * wrapper ever grows a line. Before this work the same code emitted {@code SourceFile} naming a
+ * file that never existed and a table of statement ordinals, and nothing noticed.
+ */
+class MalLineAttributionTest {
+
+ @TempDir
+ File outputDir;
+
+ private MALClassGenerator generator;
+
+ @BeforeEach
+ void setUp() {
+ org.apache.skywalking.oap.server.core.dsldebug.DSLDebugCodegenSwitch.resetInjection();
+ generator = new MALClassGenerator(new ClassPool(true));
+ generator.setClassOutputDir(outputDir);
+ }
+
+ @Test
+ void sourceFileNamesTheFileThatWasActuallyWritten() throws Exception {
+ compileMultiStageRule();
+
+ final File javaFile = writtenFile(".java");
+ final ClassFile classFile = readClassFile(writtenFile(".class"));
+
+ // The whole contract of SourceFile: an IDE resolves a frame by opening exactly this name
+ // next to the class. It used to read "(vm.yaml:0)cpu_total.java" while the file on disk
+ // was "vm_L0_cpu_total.java", so it could never resolve.
+ assertEquals(javaFile.getName(), classFile.getSourceFile());
+ }
+
+ @Test
+ void classNameCarriesTheRealYamlLine() throws Exception {
+ compileMultiStageRule();
+
+ // 37 is what the caller supplied as the rule's line; it must survive into the label
+ // rather than being replaced by the rule's position in the list.
+ assertTrue(writtenFile(".class").getName().contains("_L37_"),
+ "expected the YAML line in the class name, got: " + writtenFile(".class").getName());
+ }
+
+ @Test
+ void everyLineNumberEntryPointsAtAStatementInThatFile() throws Exception {
+ compileMultiStageRule();
+
+ final List<String> sourceLines =
+ Files.readAllLines(writtenFile(".java").toPath(), StandardCharsets.UTF_8);
+ final List<Integer> tableLines = lineNumbersOf(readClassFile(writtenFile(".class")), "run");
+
+ assertTrue(tableLines.size() >= 2,
+ "expected several boundaries for a multi-stage rule, got " + tableLines.size());
+
+ for (final int line : tableLines) {
+ assertTrue(line >= 1 && line <= sourceLines.size(),
+ "line " + line + " is outside the generated file (1.." + sourceLines.size() + ")");
+ final String text = sourceLines.get(line - 1).trim();
+ // A boundary is either a result assignment or the terminating return — never a blank,
+ // a brace, or the class declaration, which is what an off-by-N would land on.
+ assertTrue(text.endsWith(";"),
+ "line " + line + " should hold a statement but was: '" + text + "'");
+ }
+ }
+
+ @Test
+ void theLastBoundaryIsTheReturn() throws Exception {
+ compileMultiStageRule();
+
+ final List<String> sourceLines =
+ Files.readAllLines(writtenFile(".java").toPath(), StandardCharsets.UTF_8);
+ final List<Integer> tableLines = lineNumbersOf(readClassFile(writtenFile(".class")), "run");
+
+ // Pins the ordering end-to-end: the final bytecode boundary must correspond to the final
+ // statement of the method, so any drift in the middle would shift this off the return.
+ final String last = sourceLines.get(tableLines.get(tableLines.size() - 1) - 1).trim();
+ assertTrue(last.startsWith("return "),
+ "last boundary should be the return, was: '" + last + "'");
+ }
+
+ /** A chain with several stages, so the table has more than one entry to get wrong. */
+ private void compileMultiStageRule() throws Exception {
+ generator.setClassNameHint("cpu_total");
+ generator.setYamlSource("vm.yaml:37");
+ try {
+ final MalExpression expr = generator.compile(
+ "meter_vm_cpu_total",
+ "(node_cpu_seconds_total * 100).tagNotEqual('mode', 'idle')"
+ + ".sum(['host']).rate('PT1M')");
+ assertNotNull(expr);
+ } finally {
+ generator.setClassNameHint(null);
+ generator.setYamlSource(null);
+ }
+ }
+
+ private File writtenFile(final String suffix) {
+ final File[] matches = outputDir.listFiles(
+ (dir, name) -> name.endsWith(suffix) && name.contains("cpu_total"));
+ assertNotNull(matches, "no files written to " + outputDir);
+ assertTrue(matches.length > 0, "no " + suffix + " written to " + outputDir);
+ return matches[0];
+ }
+
+ private static ClassFile readClassFile(final File file) throws Exception {
+ try (DataInputStream in =
+ new DataInputStream(new BufferedInputStream(new FileInputStream(file)))) {
+ return new ClassFile(in);
+ }
+ }
+
+ private static List<Integer> lineNumbersOf(final ClassFile classFile, final String methodName) {
+ final List<Integer> out = new ArrayList<>();
+ for (final MethodInfo method : classFile.getMethods()) {
+ if (!methodName.equals(method.getName())) {
+ continue;
+ }
+ final CodeAttribute code = method.getCodeAttribute();
+ if (code == null) {
+ continue;
+ }
+ final LineNumberAttribute table =
+ (LineNumberAttribute) code.getAttribute(LineNumberAttribute.tag);
+ assertNotNull(table, "no LineNumberTable on " + methodName + "()");
+ for (int i = 0; i < table.tableLength(); i++) {
+ out.add(table.lineNumber(i));
+ }
+ }
+ return out;
+ }
+}
diff --git a/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalPrefixInjectionRangesTest.java b/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalPrefixInjectionRangesTest.java
new file mode 100644
index 0000000000..e7b53e4a59
--- /dev/null
+++ b/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalPrefixInjectionRangesTest.java
@@ -0,0 +1,83 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.meter.analyzer.v2.compiler;
+
+import java.util.List;
+import org.junit.jupiter.api.Test;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * The rewrite was changed from a right-to-left {@code replace} loop to a left-to-right build so
+ * the injected prefix's FINAL position is recoverable. The output text must not have changed —
+ * it feeds the compiler — and the reported ranges must actually contain the prefix.
+ */
+class MalPrefixInjectionRangesTest {
+
+ private static final String PREFIX = "tag({tags -> tags.k = 'v'})";
+
+ @Test
+ void nullOrEmptyPrefixIsAPassThrough() {
+ assertEquals("a.sum(['s'])",
+ MALScriptParser.injectExpPrefix("a.sum(['s'])", null));
+ assertEquals("a.sum(['s'])",
+ MALScriptParser.injectExpPrefix("a.sum(['s'])", ""));
+ assertTrue(MALScriptParser.injectExpPrefixTracked("a.sum(['s'])", "")
+ .getPrefixRanges().isEmpty());
+ }
+
+ @Test
+ void injectsAtEveryMetricReferenceNotJustTheFirst() {
+ // The case the walker exists for: a second metric nested in an argument.
+ final String out = MALScriptParser.injectExpPrefix("a.sum(['s']).safeDiv(b.sum(['s']))",
+ PREFIX);
+
+ assertEquals("(a." + PREFIX + ").sum(['s']).safeDiv((b." + PREFIX + ").sum(['s']))", out);
+ }
+
+ @Test
+ void reportedRangesLandExactlyOnTheInjectedPrefixText() {
+ final MALScriptParser.PrefixInjection injected =
+ MALScriptParser.injectExpPrefixTracked("a.sum(['s']).safeDiv(b.sum(['s']))", PREFIX);
+
+ final String text = injected.getText();
+ final List<int[]> ranges = injected.getPrefixRanges();
+
+ // One per metric reference, ascending, each slicing out exactly the prefix.
+ assertEquals(2, ranges.size());
+ assertTrue(ranges.get(0)[0] < ranges.get(1)[0], "ranges must be ascending");
+ for (final int[] range : ranges) {
+ assertEquals(PREFIX, text.substring(range[0], range[1]));
+ }
+ }
+
+ @Test
+ void downsamplingConstantsAreNotTreatedAsMetrics() {
+ // SUM/AVG/... parse as bare identifiers but are enum values; wrapping them would
+ // produce a bogus SampleFamily reference.
+ final String out = MALScriptParser.injectExpPrefix("a.histogram().histogram_percentile([50])",
+ PREFIX);
+
+ assertTrue(out.startsWith("(a." + PREFIX + ")"), "metric should be wrapped: " + out);
+ assertEquals(1, MALScriptParser
+ .injectExpPrefixTracked("a.histogram().histogram_percentile([50])", PREFIX)
+ .getPrefixRanges().size());
+ }
+}
diff --git a/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalProbeEmissionTest.java b/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalProbeEmissionTest.java
new file mode 100644
index 0000000000..193be84789
--- /dev/null
+++ b/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalProbeEmissionTest.java
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.meter.analyzer.v2.compiler;
+
+import java.io.File;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.util.List;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import javassist.ClassPool;
+import org.apache.skywalking.oap.server.core.dsldebug.DSLDebugCodegenSwitch;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * Runs the codegen with debug injection ENABLED and asserts the YAML line actually reaches the
+ * emitted probe call.
+ *
+ * <p>Every other test of the operator coordinate stops at {@code MalSourceMap.yamlLineOf(...)} —
+ * they check the map, not what codegen does with it. The probe-emission path only executes when
+ * injection is on, so without this test that code would ship having never run: it would compile,
+ * and the map would be correct, and the emitted literal could still be wrong.
+ *
+ * <p>Asserting on the generated source rather than on captured records keeps the test inside this
+ * module — the recorder that turns these arguments into a debug record lives in
+ * {@code dsl-debugging}, downstream of here.
+ */
+class MalProbeEmissionTest {
+
+ private static final int EXP_LINE = 38;
+ private static final int SUFFIX_LINE = 32;
+
+ @TempDir
+ File outputDir;
+
+ private MALClassGenerator generator;
+
+ @BeforeEach
+ void enableInjection() {
+ DSLDebugCodegenSwitch.enableInjection();
+ generator = new MALClassGenerator(new ClassPool(true));
+ generator.setClassOutputDir(outputDir);
+ }
+
+ @AfterEach
+ void resetInjection() {
+ // JVM-wide switch — leaking it on would change every sibling test's generated shape.
+ DSLDebugCodegenSwitch.resetInjection();
+ }
+
+ @Test
+ void everyEmittedProbeCarriesAResolvedLine() throws Exception {
+ final String generated = compileWithSourceMap();
+
+ final List<int[]> probes = probeLines(generated);
+ assertTrue(probes.size() >= 2,
+ "expected several probe call sites with injection on, got " + probes.size());
+ for (final int[] probe : probes) {
+ assertTrue(probe[0] > 0,
+ "a probe was emitted with a non-positive line: " + probe[0]);
+ }
+ }
+
+ @Test
+ void stagesFromExpAndFromExpSuffixEmitTheirOwnLines() throws Exception {
+ final String generated = compileWithSourceMap();
+
+ // The decisive assertion of the whole operator coordinate: within ONE generated method,
+ // stages authored in `exp:` and the stage contributed by the file-level `expSuffix:`
+ // must emit DIFFERENT lines. An implementation that resolved per-rule instead of
+ // per-stage would emit EXP_LINE everywhere and still look plausible.
+ assertTrue(generated.contains(", " + EXP_LINE + ");"),
+ "no probe reported the exp line " + EXP_LINE + " in:\n" + generated);
+ assertTrue(generated.contains(", " + SUFFIX_LINE + ");"),
+ "no probe reported the expSuffix line " + SUFFIX_LINE + " in:\n" + generated);
+ }
+
+ @Test
+ void withoutASourceMapProbesReportUnresolvedRatherThanAWrongLine() throws Exception {
+ generator.setClassNameHint("no_map");
+ generator.setYamlSource("vm.yaml:" + EXP_LINE);
+ // Deliberately no setSourceMap: simulates a caller that could not resolve lines.
+ generator.compile("meter_vm_no_map", "node_cpu.sum(['host']).rate('PT1M')");
+
+ final String generated = readGenerated("no_map");
+ for (final int[] probe : probeLines(generated)) {
+ assertEquals(MalSourceRef.UNRESOLVED, probe[0],
+ "an unmapped probe must report -1, not a plausible-looking line");
+ }
+ }
+
+ /** Compiles a rule whose expSuffix is file-level, mirroring MetricConvert's formatExp. */
+ private String compileWithSourceMap() throws Exception {
+ final String exp = "node_cpu.sum(['host']).rate('PT1M')";
+ final String suffix = "service(['host'], Layer.GENERAL)";
+ final MALScriptParser.PrefixInjection injection =
+ MALScriptParser.injectExpPrefixTracked(exp, null);
+
+ generator.setClassNameHint("cpu_total");
+ generator.setYamlSource("vm.yaml:" + EXP_LINE);
+ generator.setSourceMap(MalSourceMap.of(injection, true, EXP_LINE, 0, SUFFIX_LINE));
+ generator.compile("meter_vm_cpu_total", String.format("(%s).%s", exp, suffix));
+
+ return readGenerated("cpu_total");
+ }
+
+ private String readGenerated(final String hint) throws Exception {
+ final File[] matches = outputDir.listFiles(
+ (dir, name) -> name.endsWith(".java") && name.contains(hint));
+ assertNotNull(matches, "nothing written to " + outputDir);
+ assertTrue(matches.length > 0, "no .java written for " + hint);
+ return new String(Files.readAllBytes(matches[0].toPath()), StandardCharsets.UTF_8);
+ }
+
+ /** The trailing int literal of every {@code MALDebug.captureXxx(...)} call site. */
+ private static List<int[]> probeLines(final String generated) {
+ final Matcher m = Pattern.compile("MALDebug\\.capture\\w+\\([^;]*?,\\s*(-?\\d+)\\);")
+ .matcher(generated);
+ final List<int[]> out = new java.util.ArrayList<>();
+ while (m.find()) {
+ out.add(new int[]{Integer.parseInt(m.group(1))});
+ }
+ return out;
+ }
+}
diff --git a/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalSourceMapTest.java b/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalSourceMapTest.java
new file mode 100644
index 0000000000..a2713afb12
--- /dev/null
+++ b/oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/v2/compiler/MalSourceMapTest.java
@@ -0,0 +1,103 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.meter.analyzer.v2.compiler;
+
+import org.junit.jupiter.api.Test;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+/**
+ * Pins the worked example this whole change exists for: in a rule whose {@code expSuffix} is
+ * file-level, the stages written in {@code exp:} must report the rule's line while the suffix's
+ * stage reports the SUFFIX's line — not the rule's.
+ */
+class MalSourceMapTest {
+
+ private static final int EXP_LINE = 38;
+ private static final int PREFIX_LINE = 18;
+ private static final int SUFFIX_LINE = 32;
+
+ @Test
+ void expStagesReportTheRuleLineAndSuffixStagesReportTheSuffixLine() {
+ final String exp = "node_cpu.sum(['host'])";
+ final MALScriptParser.PrefixInjection injection =
+ MALScriptParser.injectExpPrefixTracked(exp, null);
+ final MalSourceMap map =
+ MalSourceMap.of(injection, true, EXP_LINE, PREFIX_LINE, SUFFIX_LINE);
+
+ // Formatted: "(" + exp + ")." + expSuffix
+ // 0 1..22 23,24 25...
+ assertEquals(EXP_LINE, map.yamlLineOf(1), "start of exp");
+ assertEquals(EXP_LINE, map.yamlLineOf(1 + exp.indexOf("sum")), "a stage inside exp");
+ assertEquals(SUFFIX_LINE, map.yamlLineOf(1 + exp.length() + 2), "first char of expSuffix");
+ assertEquals(SUFFIX_LINE, map.yamlLineOf(1 + exp.length() + 40), "deeper into expSuffix");
+ }
+
+ @Test
+ void prefixDerivedStagesReportThePrefixLineNotTheRuleLine() {
+ // The case an offset-free implementation gets silently wrong: expPrefix is spliced INSIDE
+ // exp, so a naive "before the suffix boundary means exp" rule mis-attributes it.
+ final String prefix = "tag({tags -> tags.k = 'v'})";
+ final MALScriptParser.PrefixInjection injection =
+ MALScriptParser.injectExpPrefixTracked("node_cpu.sum(['host'])", prefix);
+ final MalSourceMap map =
+ MalSourceMap.of(injection, true, EXP_LINE, PREFIX_LINE, SUFFIX_LINE);
+
+ final int[] prefixRange = injection.getPrefixRanges().get(0);
+ // +1 for the wrapping paren formatExp adds when a suffix follows.
+ assertEquals(PREFIX_LINE, map.yamlLineOf(prefixRange[0] + 1));
+ assertEquals(PREFIX_LINE, map.yamlLineOf(prefixRange[1] - 1 + 1));
+
+ // Immediately after the injected prefix we are back in exp.
+ assertEquals(EXP_LINE, map.yamlLineOf(prefixRange[1] + 1 + 1));
+ }
+
+ @Test
+ void withoutASuffixThereIsNoWrappingParenAndNoSuffixSegment() {
+ final String exp = "node_cpu.sum(['host'])";
+ final MalSourceMap map = MalSourceMap.of(
+ MALScriptParser.injectExpPrefixTracked(exp, null),
+ false, EXP_LINE, PREFIX_LINE, SUFFIX_LINE);
+
+ assertEquals(EXP_LINE, map.yamlLineOf(0), "no leading paren, so offset 0 is exp");
+ assertEquals(MalSourceRef.UNRESOLVED, map.yamlLineOf(exp.length() + 5),
+ "nothing lives past exp when there is no suffix");
+ }
+
+ @Test
+ void unknownOffsetsAndEmptyMapsResolveToUnresolved() {
+ assertEquals(MalSourceRef.UNRESOLVED, MalSourceMap.EMPTY.yamlLineOf(0));
+ assertEquals(MalSourceRef.UNRESOLVED,
+ MalSourceMap.of(null, true, EXP_LINE, PREFIX_LINE, SUFFIX_LINE).yamlLineOf(0));
+ assertEquals(MalSourceRef.UNRESOLVED, MalSourceMap.of(
+ MALScriptParser.injectExpPrefixTracked("a.sum(['s'])", null),
+ true, EXP_LINE, PREFIX_LINE, SUFFIX_LINE).yamlLineOf(-1));
+ }
+
+ @Test
+ void anUnresolvedYamlLineStaysUnresolvedRatherThanBecomingZero() {
+ // Phase A could not resolve the exp line: the map must not report 0, which downstream
+ // would treat as "not applicable" and silently omit.
+ final MalSourceMap map = MalSourceMap.of(
+ MALScriptParser.injectExpPrefixTracked("a.sum(['s'])", null),
+ false, 0, 0, 0);
+
+ assertEquals(MalSourceRef.UNRESOLVED, map.yamlLineOf(0));
+ }
+}
diff --git a/oap-server/server-admin/dsl-debugging/src/main/java/org/apache/skywalking/oap/server/admin/dsl/debugging/mal/MALDebugRecorderImpl.java b/oap-server/server-admin/dsl-debugging/src/main/java/org/apache/skywalking/oap/server/admin/dsl/debugging/mal/MALDebugRecorderImpl.java
index 3367228f13..f2906e56f4 100644
--- a/oap-server/server-admin/dsl-debugging/src/main/java/org/apache/skywalking/oap/server/admin/dsl/debugging/mal/MALDebugRecorderImpl.java
+++ b/oap-server/server-admin/dsl-debugging/src/main/java/org/apache/skywalking/oap/server/admin/dsl/debugging/mal/MALDebugRecorderImpl.java
@@ -90,14 +90,15 @@ public final class MALDebugRecorderImpl extends AbstractDebugRecorder
}
@Override
- public void appendInput(final String rule, final String metricRef, final SampleFamily family) {
- addSample(new Sample(Sample.TYPE_INPUT, metricRef, true, payload(family), 0));
+ public void appendInput(final String rule, final String metricRef, final SampleFamily family,
+ final int sourceLine) {
+ addSample(new Sample(Sample.TYPE_INPUT, metricRef, true, payload(family), sourceLine));
}
@Override
public void appendFilter(final String rule, final String filterText,
final Map<String, SampleFamily> surviving,
- final boolean kept) {
+ final boolean kept, final int sourceLine) {
if (!kept) {
// Kept-only capture: a rule whose file-level filter rejected the
// entire input map isn't relevant to this rule's operator-facing
@@ -110,7 +111,7 @@ public final class MALDebugRecorderImpl extends AbstractDebugRecorder
return;
}
addSample(new Sample(Sample.TYPE_FILTER, filterText, true,
- multiFamilyPayload(surviving), 0));
+ multiFamilyPayload(surviving), sourceLine));
}
/**
@@ -141,25 +142,27 @@ public final class MALDebugRecorderImpl extends AbstractDebugRecorder
}
@Override
- public void appendStage(final String rule, final String sourceText, final SampleFamily family) {
- addSample(new Sample(Sample.TYPE_FUNCTION, sourceText, true, payload(family), 0));
+ public void appendStage(final String rule, final String sourceText, final SampleFamily family,
+ final int sourceLine) {
+ addSample(new Sample(Sample.TYPE_FUNCTION, sourceText, true, payload(family), sourceLine));
}
@Override
public void appendDownsample(final String rule, final String function,
- final String origin, final SampleFamily family) {
+ final String origin, final SampleFamily family,
+ final int sourceLine) {
// sourceText is the verbatim downsample function name (AVG / SUM / etc.).
// The `origin` argument tells the UI whether the operator declared
// `downsampling:` explicitly or MAL filled it in based on the metric
// type — record both so the terminal output sample can echo it.
lastDownsample.set(function + "(" + origin + ")");
- addSample(new Sample(Sample.TYPE_FUNCTION, function, true, payload(family), 0));
+ addSample(new Sample(Sample.TYPE_FUNCTION, function, true, payload(family), sourceLine));
}
@Override
public void appendMeterEmit(final String rule, final MeterEntity entity,
final String metricName, final AcceptableValue<?> value,
- final long timeBucket) {
+ final long timeBucket, final int sourceLine) {
// One analysis pass can emit multiple meters (one per entity in
// meterSamples) — they all land in the same record. The pass-level
// appendEndAnalysis is the close point, not this one.
@@ -174,7 +177,7 @@ public final class MALDebugRecorderImpl extends AbstractDebugRecorder
? resolveFunctionName(value)
: captured;
addSample(new Sample(Sample.TYPE_OUTPUT, sourceText, true,
- meterEmitPayload(entity, metricName, value, timeBucket), 0));
+ meterEmitPayload(entity, metricName, value, timeBucket), sourceLine));
}
private static String payload(final SampleFamily family) {
diff --git a/oap-server/server-admin/runtime-rule/src/main/java/org/apache/skywalking/oap/server/receiver/runtimerule/apply/MalFileApplier.java b/oap-server/server-admin/runtime-rule/src/main/java/org/apache/skywalking/oap/server/receiver/runtimerule/apply/MalFileApplier.java
index 1761e913b6..e0b969b3fa 100644
--- a/oap-server/server-admin/runtime-rule/src/main/java/org/apache/skywalking/oap/server/receiver/runtimerule/apply/MalFileApplier.java
+++ b/oap-server/server-admin/runtime-rule/src/main/java/org/apache/skywalking/oap/server/receiver/runtimerule/apply/MalFileApplier.java
@@ -33,6 +33,7 @@ import lombok.extern.slf4j.Slf4j;
import org.apache.skywalking.oap.meter.analyzer.v2.MetricConvert;
import org.apache.skywalking.oap.meter.analyzer.v2.prometheus.rule.MetricsRule;
import org.apache.skywalking.oap.meter.analyzer.v2.prometheus.rule.Rule;
+import org.apache.skywalking.oap.meter.analyzer.v2.prometheus.rule.RuleSourceLines;
import org.apache.skywalking.oap.server.core.CoreModule;
import org.apache.skywalking.oap.server.core.analysis.LayerDefinition;
import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
@@ -279,6 +280,9 @@ public class MalFileApplier {
if (rule.getName() == null || rule.getName().isEmpty()) {
rule.setName(sourceName);
}
+ // Stamp YAML source anchors from the SAME text we just bound, so a hot-updated rule
+ // gets the same line provenance a disk-loaded one does.
+ RuleSourceLines.assign(rule, yamlContent);
// layerDefinitions: are now permitted in runtime MAL rules; they're handled by
// the layer registry on the apply path below. The rejection that used to live
// here was removed when runtime dynamic layers became a first-class feature.