You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by wu...@apache.org on 2026/09/11 03:49:27 UTC
(skywalking-graalvm-distro) branch main updated: Sync upstream to 476afadecd (11.1.0-SNAPSHOT): AI evaluation, AI agent conversations, native-image fixes (#34)
This is an automated email from the ASF dual-hosted git repository.
wu-sheng pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/skywalking-graalvm-distro.git
The following commit(s) were added to refs/heads/main by this push:
new 5ea6df6 Sync upstream to 476afadecd (11.1.0-SNAPSHOT): AI evaluation, AI agent conversations, native-image fixes (#34)
5ea6df6 is described below
commit 5ea6df60d05212d937aade9459fdd17b5715c0a3
Author: 吴晟 Wu Sheng <wu...@foxmail.com>
AuthorDate: Fri Sep 11 11:49:21 2026 +0800
Sync upstream to 476afadecd (11.1.0-SNAPSHOT): AI evaluation, AI agent conversations, native-image fixes (#34)
- Bump the skywalking submodule v11.0.0 -> master 476afadecd (13 commits); skywalking.version 11.1.0-SNAPSHOT.
- Wire the new ai-evaluation (SWIP-16 LLM-as-judge, disabled by default) and ai-agent-conversation
(default + none) modules: startup registration, AcceptedModules, poms, application.yml, config
loaders, provider inventory, ai-evaluation.yml in both distributions.
- Adopt the new/changed rules: lal/ai-agent.yaml, meter-analyzer-config/gen-ai-model.yaml,
otel-rules/ai-agent/*, otel-rules/banyandb/banyandb-trace-sampling.yaml, otel-rules/oap.yaml.
- Native-image fixes found by native e2e: embed every query-protocol/*.graphqls schema; collect
Armeria @Decorator targets; register admin-server Jackson response POJOs (/inspect/* returned 400);
register the OTLP/HTTP JSON Export*ServiceRequest + LogData protobuf descriptor closure.
- E2E: new ai-agent, banyandb-trace-sampling, banyandb-auth-rotation, banyandb-ca-rotation cases;
virtual-genai moves to the published mock-LLM / Spring AI images and asserts LLM-as-judge results;
bump SW_CTL_COMMIT / SW_E2E_SERVICE_COMMIT, add SW_AI_SESSIONIZER_COMMIT.
- Tests, inventories, YAML SHA baseline, docs, changes.md and the sync skill updated.
Claude-Session: https://claude.ai/code/session_01SbgcgeiHpVW7KMainY6L9Z
---
.claude/skills/sync-submodule/SKILL.md | 38 ++-
.github/E2E-CASES.md | 6 +-
.github/workflows/ci.yml | 8 +
CLAUDE.md | 10 +-
.../server/buildtools/common/AcceptedModules.java | 3 +
build-tools/precompiler/pom.xml | 6 +
.../server/buildtools/precompiler/Precompiler.java | 69 +++--
.../resources/META-INF/module-config-classes.txt | 3 +
changes/changes.md | 31 +++
docs/configuration.md | 45 ++-
docs/distro-policy.md | 20 +-
docs/version-mapping.md | 3 +-
.../src/main/assembly/native-distribution.xml | 1 +
.../oap-graalvm-native/reachability-metadata.json | 68 +----
oap-graalvm-server/pom.xml | 8 +
.../src/main/assembly/distribution.xml | 2 +
.../server/graalvm/GraalVMOAPServerStartUp.java | 18 ++
.../server/library/util/YamlConfigLoaderUtils.java | 302 +++++++++++++++------
.../src/main/resources/application.yml | 26 +-
.../oap/server/graalvm/lal/LALAIAgentTest.java | 50 ++++
.../graalvm/mal/AIAgentRuntimeInstanceTest.java | 30 ++
.../graalvm/mal/AIAgentRuntimeServiceTest.java | 30 ++
.../graalvm/mal/BanyandbTraceSamplingTest.java | 30 ++
.../oap/server/graalvm/mal/GenAIModelTest.java | 30 ++
.../skywalking/oap/server/graalvm/mal/OapTest.java | 16 ++
.../resources/precompiled-yaml-sha256.properties | 9 +-
.../test/resources/provider-inventory.properties | 3 +
.../test/resources/rule-file-inventory.properties | 11 +-
pom.xml | 12 +-
skywalking | 2 +-
test/e2e/cases/ai-agent/docker-compose.yml | 213 +++++++++++++++
test/e2e/cases/ai-agent/e2e.yaml | 74 +++++
.../docker-compose.yml | 65 +++--
test/e2e/cases/banyandb-auth-rotation/e2e.yaml | 77 ++++++
.../docker-compose.yml | 64 +++--
test/e2e/cases/banyandb-ca-rotation/e2e.yaml | 69 +++++
.../banyandb-trace-sampling/docker-compose.yml | 102 +++++++
test/e2e/cases/banyandb-trace-sampling/e2e.yaml | 38 +++
test/e2e/cases/virtual-genai/docker-compose.yml | 71 +++--
test/e2e/script/env | 7 +-
40 files changed, 1395 insertions(+), 275 deletions(-)
diff --git a/.claude/skills/sync-submodule/SKILL.md b/.claude/skills/sync-submodule/SKILL.md
index c77838b..7a7fd30 100644
--- a/.claude/skills/sync-submodule/SKILL.md
+++ b/.claude/skills/sync-submodule/SKILL.md
@@ -188,9 +188,41 @@ docker logs <container> 2>&1 | grep "ERROR\|NoSuchMethodException\|ClassNotFound
when a new probe reads a replaced class.
- **Protobuf `JsonFormat` needs reflection metadata**: it prints/parses generated messages through
the accessor table, which looks getters up with `Class.getMethod`. The precompiler registers the
- descriptor closure of every LAL input type (`addProtoMessageEntries`); when upstream starts
- JSON-printing another generated type (a new LAL input type, an OTLP/JSON receiver...), extend the
- root set or the native image answers `Generated message class ... missing method ...`.
+ descriptor closure of every LAL input type plus the OTLP/HTTP JSON receivers' `Export*ServiceRequest`
+ and `LogData` (`protoJsonRoots` → `addProtoMessageEntries`); when upstream starts JSON-parsing or
+ printing another generated type (`git grep "JsonFormat\.(parser|printer)"`), extend the root set or
+ the native image answers `Generated message class ... missing method ...` (11.1: the ai-agent
+ case's OTLP/JSON log posts exposed the missing OTLP roots).
+- **`LALOutputBuilder` implementations are a hand list**: the precompiler resolves `outputType` short
+ names through the SPI at build time, but their reflection entries sit in `Precompiler`'s
+ `configPojos` list. A new builder (11.1: `ConversationFile` from `ai-agent-conversation`) goes on
+ that list, and its module must be a precompiler dependency or the LAL rule fails to compile.
+- **Explicit-input MAL tests go stale when upstream adds a rule to an existing file**: `OapTest`-style
+ tests (hand-built `buildInput`) fail with "both paths returned EMPTY" on the new rule until its
+ sample family is added to the input; auto-discovery tests pick new rules up by themselves.
+- **Classpath resources must be globbed into the native image**: `oap-graalvm-native`'s
+ `reachability-metadata.json` lists the resources the binary embeds. JVM unit tests never notice a
+ missing one; the native OAP dies at boot (11.1: `classpath:query-protocol/gen-ai-evaluation-record.graphqls`,
+ now covered by the `query-protocol/*.graphqls` glob). When upstream starts reading a new
+ classpath resource (not a `config/` file), add a glob there and boot the native image.
+- **Armeria instantiates handler-referenced classes reflectively**: `@ExceptionHandler`,
+ `@RequestConverter` and `@Decorator` targets need constructor metadata. The precompiler's
+ `scanArmeriaHandlers` collects all three (11.1 added `@Decorator` for the AI agent conversation
+ view's `CompressResponse`; without it the native OAP dies at boot with "cannot inject the
+ dependency for ..."). A new Armeria annotation that names a class needs the same treatment.
+- **Jackson response POJOs need reflection metadata**: Armeria `HttpResponse.ofJson(pojo)` /
+ `@ProducesJson` serialize through Jackson, which finds no properties in the native image without
+ metadata and answers HTTP 400 ("No serializer found for class ..."). The precompiler's
+ `scanQueryEntityClasses` covers `org.apache.skywalking.oap.query.*.entity.*` and
+ `org.apache.skywalking.oap.server.admin.*.response.*` (the latter added in 11.1 after the storage
+ suite's `admin inspect` cases exposed it); a POJO outside those packages must be registered by hand.
+- **Native OAP boots faster than upstream assumes**: a case whose BanyanDB has no healthcheck
+ (`service_started` only, e.g. the distroless `-plugins` image) lets the native OAP connect before
+ BanyanDB listens, and it exits on `UNAVAILABLE`. Wrap the OAP entrypoint in `until nc -z ...`.
+- **Upstream e2e cases that bind-mount `java-test-service` jars**: don't build them in CI; every module
+ is published as `ghcr.io/apache/skywalking/<module>:${SW_E2E_SERVICE_COMMIT}` with the jar at
+ `/app.jar` — run the image directly, or copy the jar out through an init container when an agent
+ image must run it (`virtual-genai` spring-ai-service, `banyandb-trace-sampling` trace-mocker).
- **Reflection errors at native image runtime**: New classes instantiated via `Class.forName().newInstance()` need entries in `reflect-config.json` or `reachability-metadata.json`
- **Config loading failures**: New `ModuleConfig` subclasses need config-generator regeneration AND may need `@Setter` same-FQCN replacement
diff --git a/.github/E2E-CASES.md b/.github/E2E-CASES.md
index 030ef12..f8fed8b 100644
--- a/.github/E2E-CASES.md
+++ b/.github/E2E-CASES.md
@@ -52,13 +52,17 @@ Every case below is in the CI matrix (`.github/workflows/ci.yml`) and runs again
| AWS API Gateway | `aws-api-gateway/` | AWS API Gateway (Firehose) |
| Self Observability | `so11y/` | OAP self-observability (Prometheus telemetry via OTel collector) |
| MQE | `mqe/` | Metrics Query Engine |
-| Virtual GenAI | `virtual-genai/` | GenAI provider/model metrics (`gen-ai-analyzer`) |
+| Virtual GenAI | `virtual-genai/` | GenAI provider/model metrics (`gen-ai-analyzer`), LLM-as-judge evaluation (`ai-evaluation`, `gen-ai-model` MAL) |
| TraceQL SkyWalking | `traceql-skywalking/` | `traceQL` (SkyWalking datasource) |
| DSL Management | `dsl-management/` | Read-only DSL catalogs (`/runtime/rule/*`, `/runtime/oal/*`) + HTTP 501 for runtime-rule mutations, via curl and `swctl admin` |
| DSL Debugging | `dsl-debugging/` | SWIP-13 live debugger: status + session cycles on bundled MAL and LAL rules |
| Airflow | `airflow/` | Airflow layer OTel rules (upstream mock replay sender) |
| BanyanDB Self Observability | `banyandb-so11y/` | BanyanDB self-observability (SWIP-15: liaison + data node, endpoint + instance-relation rules) |
| DSL Debugging OAL | `dsl-debugging-oal/` | Upstream OAL live-debug flow (real samples on `service_relation_server_cpm`) |
+| AI Agent Conversations | `ai-agent/` | `ai-agent-conversation` (AI Sessionizer files via `lal/ai-agent.yaml`, `asz.view` fold), `ai-agent/*` OTel rules |
+| BanyanDB Trace Sampling | `banyandb-trace-sampling/` | `banyandb-trace-sampling` OTel rules (plugin-capable BanyanDB running the sampler chain) |
+| BanyanDB Auth Rotation | `banyandb-auth-rotation/` | BanyanDB credential hot-reload (`secretsManagementFile`) + storage suite |
+| BanyanDB CA Rotation | `banyandb-ca-rotation/` | BanyanDB TLS trust CA hot-reload + storage suite |
---
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 2e3fe94..09db7f1 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -390,6 +390,14 @@ jobs:
case: banyandb-so11y
- name: DSL Debugging OAL
case: dsl-debugging-oal
+ - name: AI Agent Conversations
+ case: ai-agent
+ - name: BanyanDB Trace Sampling
+ case: banyandb-trace-sampling
+ - name: BanyanDB Auth Rotation
+ case: banyandb-auth-rotation
+ - name: BanyanDB CA Rotation
+ case: banyandb-ca-rotation
steps:
- name: Checkout with submodules
uses: actions/checkout@v5
diff --git a/CLAUDE.md b/CLAUDE.md
index 82fb890..e01f36e 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -21,10 +21,10 @@
## Technical Notes
- **V2 DSL engines**: All four DSL compilers (OAL, MAL, LAL, Hierarchy) use the same pipeline: ANTLR4 parse → immutable AST → Javassist bytecode. Upstream PR #13723 removed Groovy from all production code. The precompiler runs these v2 engines at build time via `setClassOutputDir()` / `setClassNameHint()` / `setSourceRef()`, capturing `.class` files into the output JAR. Since 11.0.0 generated class names carry the YAML line (`{yaml}_L{line}_{hint}`, via `DslSourceRef` + `DslClassNaming`) [...]
- **OAL engine**: Generates ~1330 metrics/builder/dispatcher classes via Javassist at startup. Run at build time via `OALEngineV2.start()` with `setGeneratedFilePath()`. Uses ANTLR4 + FreeMarker + Javassist.
-- **MAL compiler**: Compiles ~1470 MAL expressions from 90 YAML rule files (incl. `meter-analyzer-config`, which upstream loads through `Rules` since 11.0.0) via `MALClassGenerator`. Each expression becomes a `MalExpression` implementation class. Closures (TagFunction, ForEachFunction, PropertiesExtractor, DecorateFunction) compile into companion classes next to the expression class. **Per-file manifests**: `META-INF/mal-v2/` mirrors original YAML directory structure; each config file co [...]
-- **LAL compiler**: Compiles 15 LAL rules from 11 YAML files via `LALClassGenerator`. Each rule becomes a `LalExpression` implementation class. `META-INF/lal-v2-rules.txt` maps `lal/<file>.yaml:<line>` (the `DslSourceRef` upstream passes) to the class and records the effective input type, which upstream `LogFilterListener` uses to route Envoy HTTP vs TCP access logs.
+- **MAL compiler**: Compiles ~1540 MAL expressions from 94 YAML rule files (incl. `meter-analyzer-config`, which upstream loads through `Rules` since 11.0.0) via `MALClassGenerator`. Each expression becomes a `MalExpression` implementation class. Closures (TagFunction, ForEachFunction, PropertiesExtractor, DecorateFunction) compile into companion classes next to the expression class. **Per-file manifests**: `META-INF/mal-v2/` mirrors original YAML directory structure; each config file co [...]
+- **LAL compiler**: Compiles 16 LAL rules from 12 YAML files via `LALClassGenerator`. Each rule becomes a `LalExpression` implementation class. `META-INF/lal-v2-rules.txt` maps `lal/<file>.yaml:<line>` (the `DslSourceRef` upstream passes) to the class and records the effective input type, which upstream `LogFilterListener` uses to route Envoy HTTP vs TCP access logs.
- **Hierarchy compiler**: Compiles 4 hierarchy matching rules via `HierarchyRuleClassGenerator`. Each rule becomes a `BiFunction<Service, Service, Boolean>` implementation class, listed by name in `META-INF/hierarchy-v2-rules.txt`.
-- **MeterSystem**: `MeterSystem.create()` uses Javassist to generate ~1430 meter function subclasses. Run at build time, export `.class` files. Separate from MAL DSL compilation.
+- **MeterSystem**: `MeterSystem.create()` uses Javassist to generate ~1500 meter function subclasses. Run at build time, export `.class` files. Separate from MAL DSL compilation.
- **Same-FQCN replacement**: Classes in `oap-libs-for-graalvm/*/src/main/java/` with the same fully-qualified class name as upstream classes are repackaged via `maven-shade-plugin` (original `.class` excluded). Used for v2 `DSL.java` (MAL/LAL), `MeterSystem.java`, `HierarchyDefinitionService.java`, etc.
- **Classpath scanning**: Guava `ClassPath.from()` used in multiple places. Run at build-time pre-compilation as verification gate, export static class index.
- **Config loading**: `YamlConfigLoaderUtils.copyProperties()` replaced with same-FQCN version that uses Lombok setters instead of `Field.setAccessible()`. See [docs/internals/config-init-immigration.md](docs/internals/config-init-immigration.md).
@@ -33,8 +33,8 @@
- **Native image**: `oap-graalvm-native` uses `native-maven-plugin` with `-Pnative` profile. Console-only `log4j2.xml` avoids RollingFile reflection chain. ~203MB binary, boots to full module init.
## Test Suites
-- **MAL**: Verify pre-compiled `MalExpression` classes from build-time JAR produce identical results to fresh v2 compilation. Covers all 90 MAL YAML rule files.
-- **LAL**: Verify pre-compiled `LalExpression` classes produce identical results. Covers all 11 LAL YAML files.
+- **MAL**: Verify pre-compiled `MalExpression` classes from build-time JAR produce identical results to fresh v2 compilation. Covers all 94 MAL YAML rule files.
+- **LAL**: Verify pre-compiled `LalExpression` classes produce identical results. Covers all 12 LAL YAML files.
- **Hierarchy**: Verify pre-compiled hierarchy rule classes match fresh v2 compilation.
See [docs/internals/dsl-immigration.md](docs/internals/dsl-immigration.md) for details.
diff --git a/build-tools/build-common/src/main/java/org/apache/skywalking/oap/server/buildtools/common/AcceptedModules.java b/build-tools/build-common/src/main/java/org/apache/skywalking/oap/server/buildtools/common/AcceptedModules.java
index 06b2249..58549a7 100644
--- a/build-tools/build-common/src/main/java/org/apache/skywalking/oap/server/buildtools/common/AcceptedModules.java
+++ b/build-tools/build-common/src/main/java/org/apache/skywalking/oap/server/buildtools/common/AcceptedModules.java
@@ -57,6 +57,9 @@ public final class AcceptedModules {
new ModuleProviderPair("log-analyzer", "default"),
new ModuleProviderPair("event-analyzer", "default"),
new ModuleProviderPair("gen-ai-analyzer", "default"),
+ new ModuleProviderPair("ai-evaluation", "default"),
+ new ModuleProviderPair("ai-agent-conversation", "default"),
+ new ModuleProviderPair("ai-agent-conversation", "none"),
// Receivers
new ModuleProviderPair("receiver-sharing-server", "default"),
new ModuleProviderPair("receiver-register", "default"),
diff --git a/build-tools/precompiler/pom.xml b/build-tools/precompiler/pom.xml
index c2df906..1fbfb24 100644
--- a/build-tools/precompiler/pom.xml
+++ b/build-tools/precompiler/pom.xml
@@ -94,6 +94,12 @@
<artifactId>agent-analyzer</artifactId>
</dependency>
+ <!-- LAL output builder `ConversationFile` (lal/ai-agent.yaml), resolved through the LALOutputBuilder SPI -->
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>ai-agent-conversation</artifactId>
+ </dependency>
+
<!-- Query plugins (GraphQL resolvers, HTTP handlers, adapter types) -->
<dependency>
<groupId>org.apache.skywalking</groupId>
diff --git a/build-tools/precompiler/src/main/java/org/apache/skywalking/oap/server/buildtools/precompiler/Precompiler.java b/build-tools/precompiler/src/main/java/org/apache/skywalking/oap/server/buildtools/precompiler/Precompiler.java
index 7c2d8a2..73ab3ce 100644
--- a/build-tools/precompiler/src/main/java/org/apache/skywalking/oap/server/buildtools/precompiler/Precompiler.java
+++ b/build-tools/precompiler/src/main/java/org/apache/skywalking/oap/server/buildtools/precompiler/Precompiler.java
@@ -858,7 +858,7 @@ public class Precompiler {
/**
* Scan for Armeria HTTP handler classes — classes with methods annotated with
* {@code @Post}, {@code @Get}, or {@code @Path} from {@code com.linecorp.armeria.server.annotation}.
- * Also collects classes referenced in {@code @ExceptionHandler} and {@code @RequestConverter}.
+ * Also collects classes referenced in {@code @ExceptionHandler}, {@code @RequestConverter} and {@code @Decorator}.
* Armeria's annotatedService().build(handler) uses reflection to discover these annotations;
* without reflection metadata, routes are silently not registered (returning 404).
*/
@@ -900,6 +900,7 @@ public class Precompiler {
if (matched) {
result.add(aClass.getName());
collectExceptionHandlerClasses(aClass, result);
+ collectDecoratorClasses(aClass, result);
}
} catch (NoClassDefFoundError | Exception ignored) {
}
@@ -956,10 +957,35 @@ public class Precompiler {
}
/**
- * Auto-discover all classes under {@code org.apache.skywalking.oap.query.*.entity} packages.
+ * Collect {@code @Decorator} targets on the handler class and its methods; Armeria instantiates
+ * them reflectively while building the annotated service, and a miss fails the boot.
+ */
+ private static void collectDecoratorClasses(Class<?> handlerClass, Set<String> result) {
+ Class<? extends Annotation> decoratorAnno = loadAnnotation(
+ "com.linecorp.armeria.server.annotation.Decorator");
+ if (decoratorAnno == null) {
+ return;
+ }
+ List<Annotation> decorators = new ArrayList<>(List.of(handlerClass.getAnnotationsByType(decoratorAnno)));
+ for (Method method : handlerClass.getMethods()) {
+ decorators.addAll(List.of(method.getAnnotationsByType(decoratorAnno)));
+ }
+ for (Annotation anno : decorators) {
+ try {
+ Class<?> decorator = (Class<?>) anno.annotationType().getMethod("value").invoke(anno);
+ result.add(decorator.getName());
+ } catch (Exception ignored) {
+ }
+ }
+ }
+
+ /**
+ * Auto-discover all classes under {@code org.apache.skywalking.oap.query.*.entity} and
+ * {@code org.apache.skywalking.oap.server.admin.*.response} packages.
* These are Jackson-serialized POJOs (Lombok {@code @Data}) returned by Armeria HTTP handlers
- * in query plugins (PromQL, LogQL, TraceQL, etc.). Inner static classes and codec serializers
- * are included. Enums are excluded (Jackson handles them without reflection metadata).
+ * in query plugins (PromQL, LogQL, TraceQL, etc.) and the admin-server family (inspect,
+ * ui-management). Inner static classes and codec serializers are included. Enums are excluded
+ * (Jackson handles them without reflection metadata).
*
* <p>This replaces hardcoded class lists — new query plugins are picked up automatically.
*/
@@ -967,15 +993,14 @@ public class Precompiler {
ImmutableSet<ClassPath.ClassInfo> allClasses) {
// Match: org.apache.skywalking.oap.query.<plugin>.entity[.<sub>].<ClassName>
+ // org.apache.skywalking.oap.server.admin.<module>.response.<ClassName>
Set<String> result = new HashSet<>();
for (ClassPath.ClassInfo classInfo : allClasses) {
String name = classInfo.getName();
- if (!name.startsWith("org.apache.skywalking.oap.query.")) {
- continue;
- }
- // Check that the package contains ".entity."
- String afterQuery = name.substring("org.apache.skywalking.oap.query.".length());
- if (afterQuery.indexOf(".entity.") < 0) {
+ boolean queryEntity = name.startsWith("org.apache.skywalking.oap.query.") && name.contains(".entity.");
+ boolean adminResponse = name.startsWith("org.apache.skywalking.oap.server.admin.")
+ && name.contains(".response.");
+ if (!queryEntity && !adminResponse) {
continue;
}
try {
@@ -1494,6 +1519,7 @@ public class Precompiler {
"org.apache.skywalking.oap.server.analyzer.provider.trace.parser.listener.DatabaseSlowStatementBuilder",
"org.apache.skywalking.oap.server.analyzer.provider.trace.parser.listener.SampledTraceBuilder",
"org.apache.skywalking.oap.server.receiver.envoy.persistence.EnvoyAccessLogBuilder",
+ "org.apache.skywalking.oap.server.ai.agent.conversation.ingest.ConversationFileBuilder",
// LALSourceTypeProvider SPI: ServiceLoader instantiates for per-layer input/output type resolution
"org.apache.skywalking.oap.server.receiver.envoy.EnvoyHTTPLALSourceTypeProvider",
// TTL status REST endpoint: Jackson serializes TTLDefinition returned by /status/config/ttl
@@ -1544,25 +1570,30 @@ public class Precompiler {
// LAL v2 expression classes — one FQCN per line, constructor-only
addConstructorEntries(entries, metaInf.resolve("lal-v2-classes.txt"));
- // LAL effective input types (proto classes). The runtime DSL replacement resolves them by
- // name to route mixed-type inputs, and the LAL debug dump / EnvoyAccessLogBuilder print them
- // through protobuf's JsonFormat, whose accessor table finds the generated getters and setters
- // by reflection on the message, its Builder and every nested message and enum type. Register
- // the whole descriptor closure with full access.
+ // Proto messages that go through protobuf's JsonFormat, whose accessor table finds the
+ // generated getters and setters by reflection on the message, its Builder and every nested
+ // message and enum type. Register the whole descriptor closure with full access. Roots: the
+ // LAL effective input types (the runtime DSL replacement resolves them by name to route
+ // mixed-type inputs; the LAL debug dump / EnvoyAccessLogBuilder print them), the OTLP/HTTP
+ // JSON receivers' export requests, and LogData parsed by ProtoBufJsonUtils.
+ Set<String> protoJsonRoots = new TreeSet<>(List.of(
+ "io.opentelemetry.proto.collector.logs.v1.ExportLogsServiceRequest",
+ "io.opentelemetry.proto.collector.metrics.v1.ExportMetricsServiceRequest",
+ "io.opentelemetry.proto.collector.trace.v1.ExportTraceServiceRequest",
+ "org.apache.skywalking.apm.network.logging.v3.LogData"));
Path lalRules = metaInf.resolve("lal-v2-rules.txt");
if (Files.exists(lalRules)) {
Properties lalProps = new Properties();
try (InputStream is = Files.newInputStream(lalRules)) {
lalProps.load(is);
}
- Set<String> inputTypes = new TreeSet<>();
for (String key : lalProps.stringPropertyNames()) {
if (key.endsWith(".inputType") && !lalProps.getProperty(key).isBlank()) {
- inputTypes.add(lalProps.getProperty(key).trim());
+ protoJsonRoots.add(lalProps.getProperty(key).trim());
}
}
- addProtoMessageEntries(entries, inputTypes);
}
+ addProtoMessageEntries(entries, protoJsonRoots);
// Hierarchy v2 rule classes — one FQCN per line, constructor-only
addConstructorEntries(entries, metaInf.resolve("hierarchy-v2-classes.txt"));
@@ -1620,7 +1651,7 @@ public class Precompiler {
try {
root = Class.forName(rootType);
} catch (ClassNotFoundException e) {
- throw new IllegalStateException("LAL input type is not on the precompiler classpath: " + rootType, e);
+ throw new IllegalStateException("Proto JSON root type is not on the precompiler classpath: " + rootType, e);
}
if (Message.class.isAssignableFrom(root)) {
collectProtoClosure(root, closure);
diff --git a/build-tools/precompiler/src/main/resources/META-INF/module-config-classes.txt b/build-tools/precompiler/src/main/resources/META-INF/module-config-classes.txt
index f6395af..f392c75 100644
--- a/build-tools/precompiler/src/main/resources/META-INF/module-config-classes.txt
+++ b/build-tools/precompiler/src/main/resources/META-INF/module-config-classes.txt
@@ -12,6 +12,8 @@ org.apache.skywalking.oap.server.admin.inspect.InspectModuleConfig
org.apache.skywalking.oap.server.admin.server.module.AdminServerModuleConfig
org.apache.skywalking.oap.server.admin.status.StatusModuleConfig
org.apache.skywalking.oap.server.admin.uimanagement.UIManagementModuleConfig
+org.apache.skywalking.oap.server.ai.agent.conversation.AIAgentConversationConfig
+org.apache.skywalking.oap.server.ai.evaluation.AIEvaluationConfig
org.apache.skywalking.oap.server.ai.pipeline.AIPipelineConfig
org.apache.skywalking.oap.server.analyzer.agent.kafka.module.KafkaFetcherConfig
org.apache.skywalking.oap.server.analyzer.provider.AnalyzerModuleConfig
@@ -41,6 +43,7 @@ org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageConfig$M
org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageConfig$MetricsHour
org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageConfig$MetricsMin
org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageConfig$Property
+org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageConfig$RecordsAIAgent
org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageConfig$RecordsBrowserErrorLog
org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageConfig$RecordsLog
org.apache.skywalking.oap.server.storage.plugin.banyandb.BanyanDBStorageConfig$RecordsNormal
diff --git a/changes/changes.md b/changes/changes.md
index 986937f..0e41dc2 100644
--- a/changes/changes.md
+++ b/changes/changes.md
@@ -1,5 +1,36 @@
# Changes
+## 0.5.0
+
+### Upstream Sync
+
+- Sync SkyWalking submodule to upstream master `476afadecd` (11.1.0-SNAPSHOT), 13 commits past the v11.0.0 tag.
+- Wire the two new analyzer modules: `ai-evaluation` (SWIP-16 LLM-as-judge over sampled GenAI spans; disabled by default, configured through the new `config/ai-evaluation.yml`) and `ai-agent-conversation` (AI agent conversations landed by the AI Sessionizer; on by default because the GraphQL query module requires it, turned off with its `none` provider).
+- Adopt every rule file upstream added or changed: `lal/ai-agent.yaml` (`ConversationFile` output builder), `meter-analyzer-config/gen-ai-model.yaml` (judge scores), `otel-rules/ai-agent/runtime-{instance,service}.yaml`, `otel-rules/banyandb/banyandb-trace-sampling.yaml`, and the AI-evaluation counters in `otel-rules/oap.yaml` — all enabled with the upstream defaults.
+- Mirror the new `application.yml` sections and defaults, and pick up upstream's BanyanDB credential / trust-CA hot reload (`secretsManagementFile`), the `recordsAIAgent` BanyanDB group, the Zipkin cold-stage query, and the ports opening once boot completes (`ModuleProvider#notifyBootCompleted`, which the fixed module manager reaches through `BootstrapFlow`).
+
+### GraalVM Native Image Compatibility
+
+- Embed every `query-protocol/*.graphqls` schema with one glob instead of a per-file list; upstream's new `gen-ai-evaluation-record` and `ai-agent-conversation` schemas were missing and the native OAP died at boot.
+- The precompiler's Armeria scan collects `@Decorator` targets (the AI agent conversation view's `CompressResponse`): Armeria instantiates them reflectively, and without the metadata the native OAP died at boot with "cannot inject the dependency".
+- Register the admin-server family's Jackson response POJOs (`org.apache.skywalking.oap.server.admin.*.response.*`) for reflection: `/inspect/*` answered HTTP 400 (`No serializer found for class ...MetricsResponse`) in the native image since the 11.0.0 sync.
+- Register the protobuf descriptor closure of the OTLP/HTTP JSON receivers' `Export{Logs,Metrics,Trace}ServiceRequest` and of `LogData` (parsed by `ProtoBufJsonUtils`): OTLP/HTTP JSON requests failed natively with `Generated message class ... missing method getResourceLogsList`.
+- Register the `ConversationFile` LAL output builder for reflection, regenerate the config loaders (`AIEvaluationConfig`, `AIAgentConversationConfig`, `BanyanDBStorageConfig$RecordsAIAgent`), and package `ai-evaluation.yml` in both distributions.
+
+### Documentation
+
+- Document the new modules in `distro-policy.md` and `configuration.md` (new `ai-evaluation` / `ai-agent-conversation` sections; refreshed `lalFiles`, `malFiles` and `meterAnalyzerActiveFiles` defaults), refresh the build-time counts, move the `version-mapping.md` dev row to `0.5.0-SNAPSHOT` and add `0.4.0` → `11.0.0`, and record the native-image pitfalls in the sync skill.
+
+### Testing
+
+- Add MAL comparison tests for the AI agent runtime, BanyanDB trace-sampling and gen-ai-model rules and a LAL pre-compilation test for `ai-agent.yaml`; `OapTest` feeds the new `ai_evaluation_*` families; the provider / rule inventories and the precompiled-YAML staleness baseline track the new files.
+
+### E2E Tests
+
+- New cases: `ai-agent` (AI Sessionizer files over OTLP logs, the `asz.view` document equal to the Sessionizer's own, list / filter / raw-file / metrics checks), `banyandb-trace-sampling` (a plugin-capable BanyanDB running the sampler chain), and `banyandb-auth-rotation` / `banyandb-ca-rotation` (credential and trust-CA hot reload, then the shared storage suite).
+- `virtual-genai` follows upstream's split: the mock LLM and the Spring AI service run from the published `e2e-mock-llm-server` / `e2e-spring-ai-service` images, and the case also asserts the LLM-as-judge evaluation records and metrics.
+- Bump `SW_CTL_COMMIT` to `1b6837da` (ai-agent commands) and `SW_E2E_SERVICE_COMMIT` to `51e72735`; add `SW_AI_SESSIONIZER_COMMIT`.
+
## 0.4.0
### Upstream Sync
diff --git a/docs/configuration.md b/docs/configuration.md
index f99cbac..e999d87 100644
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -37,6 +37,10 @@ corresponding environment variable:
| `kafka-fetcher` | `SW_KAFKA_FETCHER=default` | disabled |
| `cilium-fetcher` | `SW_CILIUM_FETCHER=default` | disabled |
| `exporter` | `SW_EXPORTER=default` | disabled |
+| `ai-evaluation` | `SW_AI_EVALUATION=default` | disabled |
+
+`ai-agent-conversation` is on by default and cannot be removed with `-` (the GraphQL query module
+requires it); `SW_AI_AGENT_CONVERSATION=none` turns it off.
## Module Configuration Reference
@@ -142,7 +146,7 @@ for details on those files.
| forceSampleErrorSegment | `SW_FORCE_SAMPLE_ERROR_SEGMENT` | `true` |
| segmentStatusAnalysisStrategy | `SW_SEGMENT_STATUS_ANALYSIS_STRATEGY` | `FROM_SPAN_STATUS` |
| noUpstreamRealAddressAgents | `SW_NO_UPSTREAM_REAL_ADDRESS` | `6000,9000` |
-| meterAnalyzerActiveFiles | `SW_METER_ANALYZER_ACTIVE_FILES` | `datasource,threadpool,satellite,go-runtime,python-runtime,continuous-profiling,java-agent,go-agent,ruby-runtime,php-runtime,nodejs-runtime` |
+| meterAnalyzerActiveFiles | `SW_METER_ANALYZER_ACTIVE_FILES` | `datasource,threadpool,satellite,go-runtime,python-runtime,continuous-profiling,java-agent,go-agent,ruby-runtime,php-runtime,nodejs-runtime,gen-ai-model` |
| slowCacheReadThreshold | `SW_SLOW_CACHE_SLOW_READ_THRESHOLD` | `default:20,redis:10` |
| slowCacheWriteThreshold | `SW_SLOW_CACHE_SLOW_WRITE_THRESHOLD` | `default:20,redis:10` |
@@ -154,8 +158,8 @@ for details on those files.
| Setting | Environment Variable | Default |
|---------|---------------------|---------|
-| lalFiles | `SW_LOG_LAL_FILES` | `envoy-als,mesh-dp,mysql-slowsql,pgsql-slowsql,redis-slowsql,k8s-service,nginx,default` |
-| malFiles | `SW_LOG_MAL_FILES` | `nginx` |
+| lalFiles | `SW_LOG_LAL_FILES` | `envoy-als,mesh-dp,mysql-slowsql,pgsql-slowsql,redis-slowsql,k8s-service,nginx,envoy-ai-gateway,miniprogram,ai-agent,default` |
+| malFiles | `SW_LOG_MAL_FILES` | `nginx,miniprogram-wechat,miniprogram-alipay` |
---
@@ -167,6 +171,41 @@ No additional settings.
---
+### ai-evaluation (disabled by default)
+
+**Selector**: `SW_AI_EVALUATION` (default: `-`)
+
+LLM-as-judge evaluation of sampled GenAI spans (SWIP-16). The judge endpoint, model, API key, system
+prompt and tasks come from `config/ai-evaluation.yml`; with an incomplete judge section the module
+stays inactive.
+
+| Setting | Environment Variable | Default |
+|---------|---------------------|---------|
+| sampleRate | `SW_AI_EVALUATION_SAMPLE_RATE` | `1000000` |
+| bufferSize | `SW_AI_EVALUATION_BUFFER_SIZE` | `100` |
+| consumerThreads | `SW_AI_EVALUATION_CONSUMER_THREADS` | `8` |
+| maxContentLength | `SW_AI_EVALUATION_MAX_CONTENT_LENGTH` | `16384` |
+
+---
+
+### ai-agent-conversation
+
+**Selector**: `SW_AI_AGENT_CONVERSATION` (default: `default`; `none` turns it off)
+
+AI agent conversations landed by the AI Sessionizer as OTLP logs under the `AI_AGENT` layer
+(`lal/ai-agent.yaml`), folded into `asz.view` documents on read.
+
+| Setting | Environment Variable | Default |
+|---------|---------------------|---------|
+| fileReadWindow | `SW_AI_AGENT_CONVERSATION_FILE_READ_WINDOW` | `16` |
+| roundReadWindow | `SW_AI_AGENT_CONVERSATION_ROUND_READ_WINDOW` | `16` |
+| maxListLimit | `SW_AI_AGENT_CONVERSATION_MAX_LIST_LIMIT` | `10000` |
+| viewRequestTimeout | `SW_AI_AGENT_CONVERSATION_VIEW_REQUEST_TIMEOUT` | `120` |
+| maxFileBytes | `SW_AI_AGENT_CONVERSATION_MAX_FILE_BYTES` | `15728640` |
+| maxResponseBytes | `SW_AI_AGENT_CONVERSATION_MAX_RESPONSE_BYTES` | `104857600` |
+
+---
+
### receiver-sharing-server
**Selector**: `SW_RECEIVER_SHARING_SERVER` (default: `default`)
diff --git a/docs/distro-policy.md b/docs/distro-policy.md
index 8923d97..9c397c4 100644
--- a/docs/distro-policy.md
+++ b/docs/distro-policy.md
@@ -17,7 +17,7 @@ Build and package Apache SkyWalking OAP server as a GraalVM native image on JDK
| **Cluster** | ClusterModule | Standalone, Kubernetes |
| **Configuration** | ConfigurationModule | Kubernetes |
| **Receivers** | SharingServerModule, TraceModule, JVMModule, MeterReceiverModule, LogModule, RegisterModule, ProfileModule, BrowserModule, EventModule, OtelMetricReceiverModule, MeshReceiverModule, EnvoyMetricReceiverModule, ZipkinReceiverModule, ZabbixReceiverModule, TelegrafReceiverModule, AWSFirehoseReceiverModule, CiliumFetcherModule, EBPFReceiverModule, AsyncProfilerModule, PprofModule, CLRModule, ConfigurationDiscoveryModule, KafkaFetcherModule | default providers |
-| **Analyzers** | AnalyzerModule, LogAnalyzerModule, EventAnalyzerModule, GenAIAnalyzerModule | default providers |
+| **Analyzers** | AnalyzerModule, LogAnalyzerModule, EventAnalyzerModule, GenAIAnalyzerModule, AIEvaluationModule, AIAgentConversationModule | default providers (AIEvaluationModule disabled by default; AIAgentConversationModule also offers `none`) |
| **Query** | QueryModule (GraphQL), PromQLModule, LogQLModule, TraceQLModule, ZipkinQueryModule | default providers |
| **Admin** | AdminServerModule, StatusModule, InspectModule, UIManagementModule, DSLDebuggingModule | default; runtime-rule excluded (mutations return 501) |
| **Alarm** | AlarmModule | default |
@@ -69,13 +69,13 @@ Run all four v2 compilers at build time via their native exposed APIs (`setClass
- Same-FQCN `OALEngineLoaderService` loads pre-compiled classes from manifests.
### MAL
-- `MALClassGenerator` compiles ~1470 MAL expressions from 90 YAML rule files at build time.
+- `MALClassGenerator` compiles ~1540 MAL expressions from 94 YAML rule files at build time.
- Deterministic class naming via `DslSourceRef` (`{yaml}_L{line}_{metricName}`), the same names the JVM distro generates.
- Same-FQCN v2 `DSL.java` loads pre-compiled `MalExpression` classes by computed name.
-- `MeterSystem.create()` also runs at build time → exports ~1430 Javassist meter classes.
+- `MeterSystem.create()` also runs at build time → exports ~1500 Javassist meter classes.
### LAL
-- `LALClassGenerator` compiles 15 LAL rules (11 YAML files) at build time.
+- `LALClassGenerator` compiles 16 LAL rules (12 YAML files) at build time.
- Deterministic class naming via `DslSourceRef` (`{yaml}_L{line}_{ruleName}`).
- Same-FQCN v2 `DSL.java` loads pre-compiled `LalExpression` classes by source coordinates from `META-INF/lal-v2-rules.txt`, which also records each rule's effective input type (upstream routes Envoy HTTP vs TCP access logs by it).
@@ -218,10 +218,12 @@ YAML parsing. No reflection involved — safe for GraalVM native image as-is.
| `metadata-service-mapping.yaml` | 1 | `ResourceUtils.read()` | Metadata service mapping |
| `service-apdex-threshold.yml` | 1 | `ApdexThresholdConfig` via `ResourceUtils.read()` | APDEX thresholds |
| `trace-sampling-policy-settings.yml` | 1 | `TraceSamplingPolicyWatcher` via `ResourceUtils.read()` | Trace sampling |
+| `gen-ai-config.yml` | 1 | `GenAIConfigLoader` via `ResourceUtils.read()` | GenAI provider/model pricing |
+| `ai-evaluation.yml` | 1 | `AIEvaluationConfigLoader` via `ResourceUtils.read()` | LLM-as-judge endpoint, prompt and tasks (read only when `ai-evaluation` is enabled) |
| `cilium-rules/**` | 2 | `CiliumFetcherProvider` via `ResourceUtils.getPathFiles()` | Cilium flow rules |
| `openapi-definitions/**` | 1 | `EndpointNameGrouping` via `ResourceUtils.getPathFiles()` | OpenAPI grouping definitions |
-**Total: 15 files** included in the distro `config/` directory. (Upstream 11.0.0 removed the
+**Total: 17 files** included in the distro `config/` directory. (Upstream 11.0.0 removed the
bundled `ui-initialized-templates/` + `UITemplateInitializer`; dashboard templates are now managed
via the `ui-management` REST surface.)
@@ -234,16 +236,16 @@ packaged in JARs. The YAML source files are not needed at runtime.
| Category | Count | Pre-compiled Into | Tool |
|---|---|---|---|
| `oal/*.oal` | 10 | ~640 metrics + ~640 builders + ~47 dispatchers (Javassist) | OAL v2 engine |
-| `meter-analyzer-config/*.yaml` | 13 | ~180 `MalExpression` classes (ANTLR4+Javassist) + meter classes | MAL v2 compiler |
-| `otel-rules/**/*.yaml` | 69 | ~1220 `MalExpression` classes + meter classes | MAL v2 compiler |
+| `meter-analyzer-config/*.yaml` | 14 | ~180 `MalExpression` classes (ANTLR4+Javassist) + meter classes | MAL v2 compiler |
+| `otel-rules/**/*.yaml` | 72 | ~1290 `MalExpression` classes + meter classes | MAL v2 compiler |
| `log-mal-rules/*.yaml` | 4 | ~4 `MalExpression` classes | MAL v2 compiler |
| `envoy-metrics-rules/*.yaml` | 2 | ~26 `MalExpression` classes + meter classes | MAL v2 compiler |
| `telegraf-rules/*.yaml` | 1 | ~20 `MalExpression` classes + meter classes | MAL v2 compiler |
| `zabbix-rules/*.yaml` | 1 | ~15 `MalExpression` classes + meter classes | MAL v2 compiler |
-| `lal/*.yaml` | 11 | 15 `LalExpression` classes (ANTLR4+Javassist) | LAL v2 compiler |
+| `lal/*.yaml` | 12 | 16 `LalExpression` classes (ANTLR4+Javassist) | LAL v2 compiler |
| `hierarchy-definition.yml` | 1 | ~4 `BiFunction` hierarchy rule classes | Hierarchy v2 compiler |
-**Total: 112 files** consumed at build time, producing ~1330 OAL classes, ~1470 MAL expression classes, ~1430 meter classes, 15 LAL expression classes, and 4 hierarchy rule classes.
+**Total: 117 files** consumed at build time, producing ~1330 OAL classes, ~1540 MAL expression classes, ~1500 meter classes, 16 LAL expression classes, and 4 hierarchy rule classes.
Additionally, the precompiler serializes parsed config POJOs as JSON manifests in
`META-INF/config-data/` (7 JSON files for meter-analyzer-config, otel-rules,
diff --git a/docs/version-mapping.md b/docs/version-mapping.md
index 6257a99..6b6f96f 100644
--- a/docs/version-mapping.md
+++ b/docs/version-mapping.md
@@ -8,7 +8,8 @@ Otherwise, it is shown as `{commit-id}-SNAPSHOT` indicating a development build
<!-- DOC-CHECK: version-mapping-table -->
| Distro Version | Apache SkyWalking Version |
|---|---|
-| 0.4.0-SNAPSHOT (dev) | 11.0.0 |
+| 0.5.0-SNAPSHOT (dev) | `476afadecdc0`-SNAPSHOT |
+| 0.4.0 | 11.0.0 |
| 0.3.0 | 10.4.0 |
| 0.2.1 | `64a1795d8a58`-SNAPSHOT |
| 0.2.0 | `64a1795d8a58`-SNAPSHOT |
diff --git a/oap-graalvm-native/src/main/assembly/native-distribution.xml b/oap-graalvm-native/src/main/assembly/native-distribution.xml
index e0c47a3..3798541 100644
--- a/oap-graalvm-native/src/main/assembly/native-distribution.xml
+++ b/oap-graalvm-native/src/main/assembly/native-distribution.xml
@@ -81,6 +81,7 @@
<include>service-apdex-threshold.yml</include>
<include>trace-sampling-policy-settings.yml</include>
<include>gen-ai-config.yml</include>
+ <include>ai-evaluation.yml</include>
</includes>
</fileSet>
<!-- LICENSE, NOTICE, and third-party licenses from upstream dist-material -->
diff --git a/oap-graalvm-native/src/main/resources/META-INF/native-image/org.apache.skywalking/oap-graalvm-native/reachability-metadata.json b/oap-graalvm-native/src/main/resources/META-INF/native-image/org.apache.skywalking/oap-graalvm-native/reachability-metadata.json
index 05ac271..2d00bcf 100644
--- a/oap-graalvm-native/src/main/resources/META-INF/native-image/org.apache.skywalking/oap-graalvm-native/reachability-metadata.json
+++ b/oap-graalvm-native/src/main/resources/META-INF/native-image/org.apache.skywalking/oap-graalvm-native/reachability-metadata.json
@@ -19861,73 +19861,7 @@
"glob": "org/slf4j/impl/StaticLoggerBinder.class"
},
{
- "glob": "query-protocol/aggregation.graphqls"
- },
- {
- "glob": "query-protocol/alarm.graphqls"
- },
- {
- "glob": "query-protocol/async-profiler.graphqls"
- },
- {
- "glob": "query-protocol/browser-log.graphqls"
- },
- {
- "glob": "query-protocol/common.graphqls"
- },
- {
- "glob": "query-protocol/continuous-profiling.graphqls"
- },
- {
- "glob": "query-protocol/ebpf-profiling.graphqls"
- },
- {
- "glob": "query-protocol/event.graphqls"
- },
- {
- "glob": "query-protocol/hierarchy.graphqls"
- },
- {
- "glob": "query-protocol/log.graphqls"
- },
- {
- "glob": "query-protocol/metadata-v2.graphqls"
- },
- {
- "glob": "query-protocol/metadata.graphqls"
- },
- {
- "glob": "query-protocol/metric.graphqls"
- },
- {
- "glob": "query-protocol/metrics-v2.graphqls"
- },
- {
- "glob": "query-protocol/metrics-v3.graphqls"
- },
- {
- "glob": "query-protocol/pprof.graphqls"
- },
- {
- "glob": "query-protocol/profile.graphqls"
- },
- {
- "glob": "query-protocol/record.graphqls"
- },
- {
- "glob": "query-protocol/top-n-records.graphqls"
- },
- {
- "glob": "query-protocol/topology.graphqls"
- },
- {
- "glob": "query-protocol/trace-v2.graphqls"
- },
- {
- "glob": "query-protocol/trace.graphqls"
- },
- {
- "glob": "query-protocol/ui-configuration.graphqls"
+ "glob": "query-protocol/*.graphqls"
},
{
"glob": "service-apdex-threshold.yml"
diff --git a/oap-graalvm-server/pom.xml b/oap-graalvm-server/pom.xml
index 30ef2fd..2f32ee0 100644
--- a/oap-graalvm-server/pom.xml
+++ b/oap-graalvm-server/pom.xml
@@ -246,6 +246,14 @@
<groupId>org.apache.skywalking</groupId>
<artifactId>gen-ai-analyzer</artifactId>
</dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>ai-evaluation</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>ai-agent-conversation</artifactId>
+ </dependency>
<!-- Receivers -->
<dependency>
diff --git a/oap-graalvm-server/src/main/assembly/distribution.xml b/oap-graalvm-server/src/main/assembly/distribution.xml
index dcb06a1..c8ff83c 100644
--- a/oap-graalvm-server/src/main/assembly/distribution.xml
+++ b/oap-graalvm-server/src/main/assembly/distribution.xml
@@ -103,6 +103,8 @@
<include>trace-sampling-policy-settings.yml</include>
<!-- GenAI pricing configuration (loaded by GenAIConfigLoader) -->
<include>gen-ai-config.yml</include>
+ <!-- AI evaluation judge/tasks (loaded by AIEvaluationConfigLoader) -->
+ <include>ai-evaluation.yml</include>
</includes>
</fileSet>
<!-- logs/ — empty directory for runtime logs -->
diff --git a/oap-graalvm-server/src/main/java/org/apache/skywalking/oap/server/graalvm/GraalVMOAPServerStartUp.java b/oap-graalvm-server/src/main/java/org/apache/skywalking/oap/server/graalvm/GraalVMOAPServerStartUp.java
index 4faa7d3..884ff29 100644
--- a/oap-graalvm-server/src/main/java/org/apache/skywalking/oap/server/graalvm/GraalVMOAPServerStartUp.java
+++ b/oap-graalvm-server/src/main/java/org/apache/skywalking/oap/server/graalvm/GraalVMOAPServerStartUp.java
@@ -57,6 +57,11 @@ import org.apache.skywalking.oap.server.analyzer.event.EventAnalyzerModule;
import org.apache.skywalking.oap.server.analyzer.event.EventAnalyzerModuleProvider;
import org.apache.skywalking.oap.analyzer.genai.module.GenAIAnalyzerModule;
import org.apache.skywalking.oap.analyzer.genai.GenAIAnalyzerModuleProvider;
+import org.apache.skywalking.oap.server.ai.evaluation.AIEvaluationModule;
+import org.apache.skywalking.oap.server.ai.evaluation.AIEvaluationProvider;
+import org.apache.skywalking.oap.server.ai.agent.conversation.AIAgentConversationModule;
+import org.apache.skywalking.oap.server.ai.agent.conversation.AIAgentConversationProvider;
+import org.apache.skywalking.oap.server.ai.agent.conversation.NoneAIAgentConversationProvider;
// Receivers
import org.apache.skywalking.oap.server.receiver.sharing.server.SharingServerModule;
import org.apache.skywalking.oap.server.receiver.sharing.server.SharingServerModuleProvider;
@@ -226,6 +231,19 @@ public class GraalVMOAPServerStartUp {
manager.register(new LogAnalyzerModule(), new LogAnalyzerModuleProvider());
manager.register(new EventAnalyzerModule(), new EventAnalyzerModuleProvider());
manager.register(new GenAIAnalyzerModule(), new GenAIAnalyzerModuleProvider());
+ // AI evaluation (optional, disabled by default with selector: -)
+ if (configuration.has("ai-evaluation")) {
+ manager.register(new AIEvaluationModule(), new AIEvaluationProvider());
+ }
+ // AI agent conversations: always registered because the GraphQL query module requires it,
+ // so the `none` provider is how the feature is turned off.
+ ApplicationConfiguration.ModuleConfiguration conversationConfig =
+ configuration.getModuleConfiguration("ai-agent-conversation");
+ if (conversationConfig != null && conversationConfig.has("none")) {
+ manager.register(new AIAgentConversationModule(), new NoneAIAgentConversationProvider());
+ } else {
+ manager.register(new AIAgentConversationModule(), new AIAgentConversationProvider());
+ }
// Receivers
manager.register(new SharingServerModule(), new SharingServerModuleProvider());
diff --git a/oap-graalvm-server/src/main/java/org/apache/skywalking/oap/server/library/util/YamlConfigLoaderUtils.java b/oap-graalvm-server/src/main/java/org/apache/skywalking/oap/server/library/util/YamlConfigLoaderUtils.java
index 0865492..f60d185 100644
--- a/oap-graalvm-server/src/main/java/org/apache/skywalking/oap/server/library/util/YamlConfigLoaderUtils.java
+++ b/oap-graalvm-server/src/main/java/org/apache/skywalking/oap/server/library/util/YamlConfigLoaderUtils.java
@@ -38,6 +38,9 @@ import org.apache.skywalking.oap.server.admin.inspect.InspectModuleConfig;
import org.apache.skywalking.oap.server.admin.server.module.AdminServerModuleConfig;
import org.apache.skywalking.oap.server.admin.status.StatusModuleConfig;
import org.apache.skywalking.oap.server.admin.uimanagement.UIManagementModuleConfig;
+import org.apache.skywalking.oap.server.ai.agent.conversation.AIAgentConversationConfig;
+import org.apache.skywalking.oap.server.ai.evaluation.AIEvaluationConfig;
+import org.apache.skywalking.oap.server.ai.evaluation.level.EvaluationLevelConfig;
import org.apache.skywalking.oap.server.ai.pipeline.AIPipelineConfig;
import org.apache.skywalking.oap.server.analyzer.agent.kafka.module.KafkaFetcherConfig;
import org.apache.skywalking.oap.server.analyzer.provider.AnalyzerModuleConfig;
@@ -124,14 +127,18 @@ public class YamlConfigLoaderUtils {
}
if (dest instanceof CoreModuleConfig) {
copyToCoreModuleConfig((CoreModuleConfig) dest, src, moduleName, providerName);
- } else if (dest instanceof ClusterModuleKubernetesConfig) {
- copyToClusterModuleKubernetesConfig((ClusterModuleKubernetesConfig) dest, src, moduleName, providerName);
- } else if (dest instanceof SharingServerConfig) {
- copyToSharingServerConfig((SharingServerConfig) dest, src, moduleName, providerName);
+ } else if (dest instanceof AIEvaluationConfig) {
+ copyToAIEvaluationConfig((AIEvaluationConfig) dest, src, moduleName, providerName);
} else if (dest instanceof AnalyzerModuleConfig) {
copyToAnalyzerModuleConfig((AnalyzerModuleConfig) dest, src, moduleName, providerName);
} else if (dest instanceof GenAIConfig) {
copyToGenAIConfig((GenAIConfig) dest, src, moduleName, providerName);
+ } else if (dest instanceof AIAgentConversationConfig) {
+ copyToAIAgentConversationConfig((AIAgentConversationConfig) dest, src, moduleName, providerName);
+ } else if (dest instanceof ClusterModuleKubernetesConfig) {
+ copyToClusterModuleKubernetesConfig((ClusterModuleKubernetesConfig) dest, src, moduleName, providerName);
+ } else if (dest instanceof SharingServerConfig) {
+ copyToSharingServerConfig((SharingServerConfig) dest, src, moduleName, providerName);
} else if (dest instanceof ZipkinReceiverConfig) {
copyToZipkinReceiverConfig((ZipkinReceiverConfig) dest, src, moduleName, providerName);
} else if (dest instanceof ZipkinQueryConfig) {
@@ -212,6 +219,8 @@ public class YamlConfigLoaderUtils {
copyToRecordsLog((BanyanDBStorageConfig.RecordsLog) dest, src, moduleName, providerName);
} else if (dest instanceof BanyanDBStorageConfig.RecordsBrowserErrorLog) {
copyToRecordsBrowserErrorLog((BanyanDBStorageConfig.RecordsBrowserErrorLog) dest, src, moduleName, providerName);
+ } else if (dest instanceof BanyanDBStorageConfig.RecordsAIAgent) {
+ copyToRecordsAIAgent((BanyanDBStorageConfig.RecordsAIAgent) dest, src, moduleName, providerName);
} else if (dest instanceof BanyanDBStorageConfig.MetricsMin) {
copyToMetricsMin((BanyanDBStorageConfig.MetricsMin) dest, src, moduleName, providerName);
} else if (dest instanceof BanyanDBStorageConfig.MetricsHour) {
@@ -431,8 +440,8 @@ public class YamlConfigLoaderUtils {
}
@SuppressWarnings("unchecked")
- private static void copyToClusterModuleKubernetesConfig(
- final ClusterModuleKubernetesConfig cfg, final Properties src,
+ private static void copyToAIEvaluationConfig(
+ final AIEvaluationConfig cfg, final Properties src,
final String moduleName, final String providerName) {
final Enumeration<?> propertyNames = src.propertyNames();
while (propertyNames.hasMoreElements()) {
@@ -440,89 +449,29 @@ public class YamlConfigLoaderUtils {
final Object value = src.get(key);
log.debug("{}.{} config: {} = {}", moduleName, providerName, key, value);
switch (key) {
- case "namespace":
- cfg.setNamespace((String) value);
- break;
- case "labelSelector":
- cfg.setLabelSelector((String) value);
- break;
- case "uidEnvName":
- cfg.setUidEnvName((String) value);
- break;
- default:
- log.warn("{} setting is not supported in {} provider of {} module",
- key, providerName, moduleName);
- break;
- }
- }
- }
-
- @SuppressWarnings("unchecked")
- private static void copyToSharingServerConfig(
- final SharingServerConfig cfg, final Properties src,
- final String moduleName, final String providerName) {
- final Enumeration<?> propertyNames = src.propertyNames();
- while (propertyNames.hasMoreElements()) {
- final String key = (String) propertyNames.nextElement();
- final Object value = src.get(key);
- log.debug("{}.{} config: {} = {}", moduleName, providerName, key, value);
- switch (key) {
- case "restHost":
- cfg.setRestHost((String) value);
- break;
- case "restPort":
- cfg.setRestPort(((Number) value).intValue());
- break;
- case "restContextPath":
- cfg.setRestContextPath((String) value);
- break;
- case "restIdleTimeOut":
- cfg.setRestIdleTimeOut(((Number) value).longValue());
- break;
- case "restAcceptQueueSize":
- cfg.setRestAcceptQueueSize(((Number) value).intValue());
- break;
- case "restSSLEnabled":
- cfg.setRestSSLEnabled((boolean) value);
- break;
- case "restSSLKeyPath":
- cfg.setRestSSLKeyPath((String) value);
- break;
- case "restSSLCertChainPath":
- cfg.setRestSSLCertChainPath((String) value);
- break;
- case "gRPCHost":
- cfg.setGRPCHost((String) value);
- break;
- case "gRPCPort":
- cfg.setGRPCPort(((Number) value).intValue());
+ case "sampleRate":
+ cfg.setSampleRate(((Number) value).intValue());
break;
- case "maxConcurrentCallsPerConnection":
- cfg.setMaxConcurrentCallsPerConnection(((Number) value).intValue());
+ case "bufferSize":
+ cfg.setBufferSize(((Number) value).intValue());
break;
- case "maxMessageSize":
- cfg.setMaxMessageSize(((Number) value).intValue());
+ case "consumerThreads":
+ cfg.setConsumerThreads(((Number) value).intValue());
break;
- case "gRPCThreadPoolSize":
- cfg.setGRPCThreadPoolSize(((Number) value).intValue());
+ case "maxContentLength":
+ cfg.setMaxContentLength(((Number) value).intValue());
break;
- case "authentication":
- cfg.setAuthentication((String) value);
+ case "judge":
+ cfg.setJudge((Properties) value);
break;
- case "gRPCSslEnabled":
- cfg.setGRPCSslEnabled((boolean) value);
+ case "systemPrompt":
+ cfg.setSystemPrompt((String) value);
break;
- case "gRPCSslKeyPath":
- cfg.setGRPCSslKeyPath((String) value);
+ case "tasks":
+ cfg.setTasks((List) value);
break;
- case "gRPCSslCertChainPath":
- cfg.setGRPCSslCertChainPath((String) value);
- break;
- case "gRPCSslTrustedCAsPath":
- cfg.setGRPCSslTrustedCAsPath((String) value);
- break;
- case "httpMaxRequestHeaderSize":
- cfg.setHttpMaxRequestHeaderSize(((Number) value).intValue());
+ case "level":
+ cfg.setLevel((EvaluationLevelConfig) value);
break;
default:
log.warn("{} setting is not supported in {} provider of {} module",
@@ -622,6 +571,144 @@ public class YamlConfigLoaderUtils {
}
}
+ @SuppressWarnings("unchecked")
+ private static void copyToAIAgentConversationConfig(
+ final AIAgentConversationConfig cfg, final Properties src,
+ final String moduleName, final String providerName) {
+ final Enumeration<?> propertyNames = src.propertyNames();
+ while (propertyNames.hasMoreElements()) {
+ final String key = (String) propertyNames.nextElement();
+ final Object value = src.get(key);
+ log.debug("{}.{} config: {} = {}", moduleName, providerName, key, value);
+ switch (key) {
+ case "fileReadWindow":
+ cfg.setFileReadWindow(((Number) value).intValue());
+ break;
+ case "roundReadWindow":
+ cfg.setRoundReadWindow(((Number) value).intValue());
+ break;
+ case "maxResponseBytes":
+ cfg.setMaxResponseBytes(((Number) value).intValue());
+ break;
+ case "viewRequestTimeout":
+ cfg.setViewRequestTimeout(((Number) value).intValue());
+ break;
+ case "maxListLimit":
+ cfg.setMaxListLimit(((Number) value).intValue());
+ break;
+ case "maxFileBytes":
+ cfg.setMaxFileBytes(((Number) value).intValue());
+ break;
+ default:
+ log.warn("{} setting is not supported in {} provider of {} module",
+ key, providerName, moduleName);
+ break;
+ }
+ }
+ }
+
+ @SuppressWarnings("unchecked")
+ private static void copyToClusterModuleKubernetesConfig(
+ final ClusterModuleKubernetesConfig cfg, final Properties src,
+ final String moduleName, final String providerName) {
+ final Enumeration<?> propertyNames = src.propertyNames();
+ while (propertyNames.hasMoreElements()) {
+ final String key = (String) propertyNames.nextElement();
+ final Object value = src.get(key);
+ log.debug("{}.{} config: {} = {}", moduleName, providerName, key, value);
+ switch (key) {
+ case "namespace":
+ cfg.setNamespace((String) value);
+ break;
+ case "labelSelector":
+ cfg.setLabelSelector((String) value);
+ break;
+ case "uidEnvName":
+ cfg.setUidEnvName((String) value);
+ break;
+ default:
+ log.warn("{} setting is not supported in {} provider of {} module",
+ key, providerName, moduleName);
+ break;
+ }
+ }
+ }
+
+ @SuppressWarnings("unchecked")
+ private static void copyToSharingServerConfig(
+ final SharingServerConfig cfg, final Properties src,
+ final String moduleName, final String providerName) {
+ final Enumeration<?> propertyNames = src.propertyNames();
+ while (propertyNames.hasMoreElements()) {
+ final String key = (String) propertyNames.nextElement();
+ final Object value = src.get(key);
+ log.debug("{}.{} config: {} = {}", moduleName, providerName, key, value);
+ switch (key) {
+ case "restHost":
+ cfg.setRestHost((String) value);
+ break;
+ case "restPort":
+ cfg.setRestPort(((Number) value).intValue());
+ break;
+ case "restContextPath":
+ cfg.setRestContextPath((String) value);
+ break;
+ case "restIdleTimeOut":
+ cfg.setRestIdleTimeOut(((Number) value).longValue());
+ break;
+ case "restAcceptQueueSize":
+ cfg.setRestAcceptQueueSize(((Number) value).intValue());
+ break;
+ case "restSSLEnabled":
+ cfg.setRestSSLEnabled((boolean) value);
+ break;
+ case "restSSLKeyPath":
+ cfg.setRestSSLKeyPath((String) value);
+ break;
+ case "restSSLCertChainPath":
+ cfg.setRestSSLCertChainPath((String) value);
+ break;
+ case "gRPCHost":
+ cfg.setGRPCHost((String) value);
+ break;
+ case "gRPCPort":
+ cfg.setGRPCPort(((Number) value).intValue());
+ break;
+ case "maxConcurrentCallsPerConnection":
+ cfg.setMaxConcurrentCallsPerConnection(((Number) value).intValue());
+ break;
+ case "maxMessageSize":
+ cfg.setMaxMessageSize(((Number) value).intValue());
+ break;
+ case "gRPCThreadPoolSize":
+ cfg.setGRPCThreadPoolSize(((Number) value).intValue());
+ break;
+ case "authentication":
+ cfg.setAuthentication((String) value);
+ break;
+ case "gRPCSslEnabled":
+ cfg.setGRPCSslEnabled((boolean) value);
+ break;
+ case "gRPCSslKeyPath":
+ cfg.setGRPCSslKeyPath((String) value);
+ break;
+ case "gRPCSslCertChainPath":
+ cfg.setGRPCSslCertChainPath((String) value);
+ break;
+ case "gRPCSslTrustedCAsPath":
+ cfg.setGRPCSslTrustedCAsPath((String) value);
+ break;
+ case "httpMaxRequestHeaderSize":
+ cfg.setHttpMaxRequestHeaderSize(((Number) value).intValue());
+ break;
+ default:
+ log.warn("{} setting is not supported in {} provider of {} module",
+ key, providerName, moduleName);
+ break;
+ }
+ }
+ }
+
@SuppressWarnings("unchecked")
private static void copyToZipkinReceiverConfig(
final ZipkinReceiverConfig cfg, final Properties src,
@@ -1250,6 +1337,9 @@ public class YamlConfigLoaderUtils {
case "recordsBrowserErrorLog":
cfg.setRecordsBrowserErrorLog((BanyanDBStorageConfig.RecordsBrowserErrorLog) value);
break;
+ case "recordsAIAgent":
+ cfg.setRecordsAIAgent((BanyanDBStorageConfig.RecordsAIAgent) value);
+ break;
case "metricsMin":
cfg.setMetricsMin((BanyanDBStorageConfig.MetricsMin) value);
break;
@@ -1877,6 +1967,9 @@ public class YamlConfigLoaderUtils {
case "sslTrustCAPath":
cfg.setSslTrustCAPath((String) value);
break;
+ case "secretsManagementFile":
+ cfg.setSecretsManagementFile((String) value);
+ break;
case "asyncProfilerTaskQueryMaxSize":
cfg.setAsyncProfilerTaskQueryMaxSize(((Number) value).intValue());
break;
@@ -2227,6 +2320,51 @@ public class YamlConfigLoaderUtils {
}
}
+ @SuppressWarnings("unchecked")
+ private static void copyToRecordsAIAgent(
+ final BanyanDBStorageConfig.RecordsAIAgent cfg, final Properties src,
+ final String moduleName, final String providerName) {
+ final Enumeration<?> propertyNames = src.propertyNames();
+ while (propertyNames.hasMoreElements()) {
+ final String key = (String) propertyNames.nextElement();
+ final Object value = src.get(key);
+ log.debug("{}.{} config: {} = {}", moduleName, providerName, key, value);
+ switch (key) {
+ case "shardNum":
+ cfg.setShardNum(((Number) value).intValue());
+ break;
+ case "segmentInterval":
+ cfg.setSegmentInterval(((Number) value).intValue());
+ break;
+ case "ttl":
+ cfg.setTtl(((Number) value).intValue());
+ break;
+ case "replicas":
+ cfg.setReplicas(((Number) value).intValue());
+ break;
+ case "enableWarmStage":
+ cfg.setEnableWarmStage((boolean) value);
+ break;
+ case "enableColdStage":
+ cfg.setEnableColdStage((boolean) value);
+ break;
+ case "defaultQueryStages":
+ cfg.setDefaultQueryStages((List) value);
+ break;
+ case "additionalLifecycleStages":
+ cfg.setAdditionalLifecycleStages((List) value);
+ break;
+ case "tracePipeline":
+ cfg.setTracePipeline((BanyanDBStorageConfig.TracePipeline) value);
+ break;
+ default:
+ log.warn("{} setting is not supported in {} provider of {} module",
+ key, providerName, moduleName);
+ break;
+ }
+ }
+ }
+
@SuppressWarnings("unchecked")
private static void copyToMetricsMin(
final BanyanDBStorageConfig.MetricsMin cfg, final Properties src,
diff --git a/oap-graalvm-server/src/main/resources/application.yml b/oap-graalvm-server/src/main/resources/application.yml
index e3916f5..37c982f 100644
--- a/oap-graalvm-server/src/main/resources/application.yml
+++ b/oap-graalvm-server/src/main/resources/application.yml
@@ -97,14 +97,14 @@ agent-analyzer:
forceSampleErrorSegment: ${SW_FORCE_SAMPLE_ERROR_SEGMENT:true}
segmentStatusAnalysisStrategy: ${SW_SEGMENT_STATUS_ANALYSIS_STRATEGY:FROM_SPAN_STATUS}
noUpstreamRealAddressAgents: ${SW_NO_UPSTREAM_REAL_ADDRESS:6000,9000}
- meterAnalyzerActiveFiles: ${SW_METER_ANALYZER_ACTIVE_FILES:datasource,threadpool,satellite,go-runtime,python-runtime,continuous-profiling,java-agent,go-agent,ruby-runtime,php-runtime,nodejs-runtime}
+ meterAnalyzerActiveFiles: ${SW_METER_ANALYZER_ACTIVE_FILES:datasource,threadpool,satellite,go-runtime,python-runtime,continuous-profiling,java-agent,go-agent,ruby-runtime,php-runtime,nodejs-runtime,gen-ai-model}
slowCacheReadThreshold: ${SW_SLOW_CACHE_SLOW_READ_THRESHOLD:default:20,redis:10}
slowCacheWriteThreshold: ${SW_SLOW_CACHE_SLOW_WRITE_THRESHOLD:default:20,redis:10}
log-analyzer:
selector: ${SW_LOG_ANALYZER:default}
default:
- lalFiles: ${SW_LOG_LAL_FILES:envoy-als,mesh-dp,mysql-slowsql,pgsql-slowsql,redis-slowsql,k8s-service,nginx,envoy-ai-gateway,miniprogram,default}
+ lalFiles: ${SW_LOG_LAL_FILES:envoy-als,mesh-dp,mysql-slowsql,pgsql-slowsql,redis-slowsql,k8s-service,nginx,envoy-ai-gateway,miniprogram,ai-agent,default}
malFiles: ${SW_LOG_MAL_FILES:"nginx,miniprogram-wechat,miniprogram-alipay"}
event-analyzer:
@@ -115,6 +115,26 @@ gen-ai-analyzer:
selector: ${SW_GENAI_ANALYZER:default}
default:
+ai-evaluation:
+ selector: ${SW_AI_EVALUATION:-}
+ default:
+ sampleRate: ${SW_AI_EVALUATION_SAMPLE_RATE:1000000}
+ bufferSize: ${SW_AI_EVALUATION_BUFFER_SIZE:100}
+ consumerThreads: ${SW_AI_EVALUATION_CONSUMER_THREADS:8}
+ maxContentLength: ${SW_AI_EVALUATION_MAX_CONTENT_LENGTH:16384}
+
+ai-agent-conversation:
+ selector: ${SW_AI_AGENT_CONVERSATION:default}
+ # The GraphQL query module requires this module, so `-` can't remove it; `none` turns it off.
+ none:
+ default:
+ fileReadWindow: ${SW_AI_AGENT_CONVERSATION_FILE_READ_WINDOW:16}
+ roundReadWindow: ${SW_AI_AGENT_CONVERSATION_ROUND_READ_WINDOW:16}
+ maxListLimit: ${SW_AI_AGENT_CONVERSATION_MAX_LIST_LIMIT:10000}
+ viewRequestTimeout: ${SW_AI_AGENT_CONVERSATION_VIEW_REQUEST_TIMEOUT:120}
+ maxFileBytes: ${SW_AI_AGENT_CONVERSATION_MAX_FILE_BYTES:15728640}
+ maxResponseBytes: ${SW_AI_AGENT_CONVERSATION_MAX_RESPONSE_BYTES:104857600}
+
receiver-sharing-server:
selector: ${SW_RECEIVER_SHARING_SERVER:default}
default:
@@ -234,7 +254,7 @@ receiver-otel:
selector: ${SW_OTEL_RECEIVER:default}
default:
enabledHandlers: ${SW_OTEL_RECEIVER_ENABLED_HANDLERS:"otlp-traces,otlp-metrics,otlp-logs"}
- enabledOtelMetricsRules: ${SW_OTEL_RECEIVER_ENABLED_OTEL_METRICS_RULES:"apisix,nginx/*,k8s/*,istio-controlplane,vm,mysql/*,postgresql/*,oap,aws-eks/*,windows,aws-s3/*,aws-dynamodb/*,aws-gateway/*,redis/*,elasticsearch/*,rabbitmq/*,mongodb/*,kafka/*,pulsar/*,bookkeeper/*,rocketmq/*,clickhouse/*,activemq/*,kong/*,flink/*,airflow/*,banyandb/*,envoy-ai-gateway/*,ios/*,miniprogram/*"}
+ enabledOtelMetricsRules: ${SW_OTEL_RECEIVER_ENABLED_OTEL_METRICS_RULES:"apisix,nginx/*,k8s/*,istio-controlplane,vm,mysql/*,postgresql/*,oap,aws-eks/*,windows,aws-s3/*,aws-dynamodb/*,aws-gateway/*,redis/*,elasticsearch/*,rabbitmq/*,mongodb/*,kafka/*,pulsar/*,bookkeeper/*,rocketmq/*,clickhouse/*,activemq/*,kong/*,flink/*,airflow/*,banyandb/*,envoy-ai-gateway/*,ios/*,miniprogram/*,ai-agent/*"}
receiver-zipkin:
selector: ${SW_RECEIVER_ZIPKIN:-}
diff --git a/oap-graalvm-server/src/test/java/org/apache/skywalking/oap/server/graalvm/lal/LALAIAgentTest.java b/oap-graalvm-server/src/test/java/org/apache/skywalking/oap/server/graalvm/lal/LALAIAgentTest.java
new file mode 100644
index 0000000..deb8fbf
--- /dev/null
+++ b/oap-graalvm-server/src/test/java/org/apache/skywalking/oap/server/graalvm/lal/LALAIAgentTest.java
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.skywalking.oap.server.graalvm.lal;
+
+import java.util.List;
+import org.apache.skywalking.oap.log.analyzer.v2.dsl.LalExpression;
+import org.apache.skywalking.oap.log.analyzer.v2.provider.LALConfig;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+
+/**
+ * Pre-compilation test for lal/ai-agent.yaml.
+ */
+class LALAIAgentTest extends LALScriptComparisonBase {
+
+ private static List<LALConfig> RULES;
+
+ @BeforeAll
+ static void loadRules() throws Exception {
+ RULES = loadLALRules("ai-agent.yaml");
+ }
+
+ @Test
+ void allAIAgentRulesPrecompiled() {
+ assertEquals(1, RULES.size(), "ai-agent.yaml should have 1 rule");
+ for (final LALConfig rule : RULES) {
+ final LalExpression expr = loadPrecompiled(rule);
+ assertNotNull(expr,
+ "Pre-compiled class for rule '" + rule.getName() + "' should load");
+ }
+ }
+}
diff --git a/oap-graalvm-server/src/test/java/org/apache/skywalking/oap/server/graalvm/mal/AIAgentRuntimeInstanceTest.java b/oap-graalvm-server/src/test/java/org/apache/skywalking/oap/server/graalvm/mal/AIAgentRuntimeInstanceTest.java
new file mode 100644
index 0000000..785f6a8
--- /dev/null
+++ b/oap-graalvm-server/src/test/java/org/apache/skywalking/oap/server/graalvm/mal/AIAgentRuntimeInstanceTest.java
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.skywalking.oap.server.graalvm.mal;
+
+import java.util.stream.Stream;
+import org.junit.jupiter.api.DynamicTest;
+import org.junit.jupiter.api.TestFactory;
+
+class AIAgentRuntimeInstanceTest extends MALScriptComparisonBase {
+
+ @TestFactory
+ Stream<DynamicTest> allMetricsGroovyVsPrecompiled() {
+ return generateComparisonTests("otel-rules/ai-agent/runtime-instance.yaml");
+ }
+}
diff --git a/oap-graalvm-server/src/test/java/org/apache/skywalking/oap/server/graalvm/mal/AIAgentRuntimeServiceTest.java b/oap-graalvm-server/src/test/java/org/apache/skywalking/oap/server/graalvm/mal/AIAgentRuntimeServiceTest.java
new file mode 100644
index 0000000..1049acb
--- /dev/null
+++ b/oap-graalvm-server/src/test/java/org/apache/skywalking/oap/server/graalvm/mal/AIAgentRuntimeServiceTest.java
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.skywalking.oap.server.graalvm.mal;
+
+import java.util.stream.Stream;
+import org.junit.jupiter.api.DynamicTest;
+import org.junit.jupiter.api.TestFactory;
+
+class AIAgentRuntimeServiceTest extends MALScriptComparisonBase {
+
+ @TestFactory
+ Stream<DynamicTest> allMetricsGroovyVsPrecompiled() {
+ return generateComparisonTests("otel-rules/ai-agent/runtime-service.yaml");
+ }
+}
diff --git a/oap-graalvm-server/src/test/java/org/apache/skywalking/oap/server/graalvm/mal/BanyandbTraceSamplingTest.java b/oap-graalvm-server/src/test/java/org/apache/skywalking/oap/server/graalvm/mal/BanyandbTraceSamplingTest.java
new file mode 100644
index 0000000..bbba297
--- /dev/null
+++ b/oap-graalvm-server/src/test/java/org/apache/skywalking/oap/server/graalvm/mal/BanyandbTraceSamplingTest.java
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.skywalking.oap.server.graalvm.mal;
+
+import java.util.stream.Stream;
+import org.junit.jupiter.api.DynamicTest;
+import org.junit.jupiter.api.TestFactory;
+
+class BanyandbTraceSamplingTest extends MALScriptComparisonBase {
+
+ @TestFactory
+ Stream<DynamicTest> allMetricsGroovyVsPrecompiled() {
+ return generateComparisonTests("otel-rules/banyandb/banyandb-trace-sampling.yaml");
+ }
+}
diff --git a/oap-graalvm-server/src/test/java/org/apache/skywalking/oap/server/graalvm/mal/GenAIModelTest.java b/oap-graalvm-server/src/test/java/org/apache/skywalking/oap/server/graalvm/mal/GenAIModelTest.java
new file mode 100644
index 0000000..9ee7561
--- /dev/null
+++ b/oap-graalvm-server/src/test/java/org/apache/skywalking/oap/server/graalvm/mal/GenAIModelTest.java
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.skywalking.oap.server.graalvm.mal;
+
+import java.util.stream.Stream;
+import org.junit.jupiter.api.DynamicTest;
+import org.junit.jupiter.api.TestFactory;
+
+class GenAIModelTest extends MALScriptComparisonBase {
+
+ @TestFactory
+ Stream<DynamicTest> allMetricsGroovyVsPrecompiled() {
+ return generateComparisonTests("meter-analyzer-config/gen-ai-model.yaml");
+ }
+}
diff --git a/oap-graalvm-server/src/test/java/org/apache/skywalking/oap/server/graalvm/mal/OapTest.java b/oap-graalvm-server/src/test/java/org/apache/skywalking/oap/server/graalvm/mal/OapTest.java
index 6ab6732..14387e7 100644
--- a/oap-graalvm-server/src/test/java/org/apache/skywalking/oap/server/graalvm/mal/OapTest.java
+++ b/oap-graalvm-server/src/test/java/org/apache/skywalking/oap/server/graalvm/mal/OapTest.java
@@ -80,6 +80,10 @@ class OapTest extends MALScriptComparisonBase {
* trace_analysis_error_count{service="oap-svc",host_name="oap-host-1",protocol="grpc"} 10.0
* spans_dropped_count{service="oap-svc",host_name="oap-host-1",protocol="grpc"} 5.0
*
+ * # --- AI Evaluation ---
+ * ai_evaluation_dropped_count{service="oap-svc",host_name="oap-host-1",reason="pipeline_capacity"} 4.0
+ * ai_evaluation_error_count{service="oap-svc",host_name="oap-host-1",reason="timeout"} 3.0
+ *
* # --- Mesh ---
* mesh_analysis_latency_count{service="oap-svc",host_name="oap-host-1"} 3000.0
* mesh_analysis_latency{...le buckets...} histogram
@@ -209,6 +213,18 @@ class OapTest extends MALScriptComparisonBase {
"protocol", "grpc", 5.0 * scale, timestamp)
).build())
+ // --- AI Evaluation ---
+ .put("ai_evaluation_dropped_count",
+ SampleFamilyBuilder.newBuilder(
+ labeled("ai_evaluation_dropped_count", scope,
+ "reason", "pipeline_capacity", 4.0 * scale, timestamp)
+ ).build())
+ .put("ai_evaluation_error_count",
+ SampleFamilyBuilder.newBuilder(
+ labeled("ai_evaluation_error_count", scope,
+ "reason", "timeout", 3.0 * scale, timestamp)
+ ).build())
+
// --- Mesh ---
.put("mesh_analysis_latency_count",
single("mesh_analysis_latency_count", scope,
diff --git a/oap-graalvm-server/src/test/resources/precompiled-yaml-sha256.properties b/oap-graalvm-server/src/test/resources/precompiled-yaml-sha256.properties
index 8476774..2d1a022 100644
--- a/oap-graalvm-server/src/test/resources/precompiled-yaml-sha256.properties
+++ b/oap-graalvm-server/src/test/resources/precompiled-yaml-sha256.properties
@@ -36,6 +36,7 @@ meter-analyzer-config/threadpool.yaml=2dd73ea38a0ab5c33c596847482eb2362e871f43b7
meter-analyzer-config/network-profiling.yaml=c2e942edb80eb7135590e04f54d29ae96d5abc67cd84530c091c8b1758728f3d
meter-analyzer-config/nodejs-runtime.yaml=344ce07e0105ff0aaa2f6c09f768971d8a676b45b3312fb93a037f84de1780d3
meter-analyzer-config/php-runtime.yaml=2d54fe39e6f4cb5d92d6e38f479b87de801ba7b3e2e781985c7484daf515ce72
+meter-analyzer-config/gen-ai-model.yaml=d8d7392910630f19cfe3f4328ab3ee3921546d72977f22d9e78c8f748c6fdc40
# otel-rules/
otel-rules/activemq/activemq-broker.yaml=b2592962a2a900cd61f81674f2e27387cd177759f076d85bf7645e07699022d6
@@ -79,13 +80,13 @@ otel-rules/mysql/mysql-service.yaml=8d912d600c27073a5b4cb14ab4e99869157c1e230274
otel-rules/nginx/nginx-endpoint.yaml=5c8ff99f9324faa8451f32cadd9d14c247e3fffc72cb717295eae7e428a907f9
otel-rules/nginx/nginx-instance.yaml=999269ade04346cc22dbcd1a761128f09afc51c1e44ea0a01ba333caee19190c
otel-rules/nginx/nginx-service.yaml=fdacf1236821635763a601e5146d042c3aa10db69c631e115b8d08a68e9ba896
-otel-rules/oap.yaml=6a61b77b0200ac35fac18d9cd3cfca425c6009461e58a8b9d5040ff6c0410d08
+otel-rules/oap.yaml=8d4724b0f16c3ccae8d0e0a156c6e370de5b38e52148dbffe38c8b95c908b468
otel-rules/postgresql/postgresql-instance.yaml=6e060619d9f9c0fff5607cc59207e9bafd8f1767cd3dcdee48fe4561644388d3
otel-rules/postgresql/postgresql-service.yaml=8c19e9b218241c0c4deb2affd863a2a36950be90b93867002deef427a417a412
otel-rules/pulsar/pulsar-broker.yaml=eec34a2a75efdbd59830743c8b8e2584bfcb2ab769dac3b4506534b758fecb16
otel-rules/pulsar/pulsar-cluster.yaml=b36c640f32dd4a3f7019067cf0420b28176d546495b93d20124da0418809e725
otel-rules/rabbitmq/rabbitmq-cluster.yaml=3c6cf764bc116c2f28ff3a35f9f4f172a3643d4373c7803e0d4000995dfef744
-otel-rules/rabbitmq/rabbitmq-node.yaml=40dadce37e0f53a48133614604c4027ea179466a2830a1b0f89491e0affb5e1a
+otel-rules/rabbitmq/rabbitmq-node.yaml=bb3fe9ef7de94c7f5bb0cf6cb2e27fed3841ba199227dc274bf08a628ba4ed8b
otel-rules/redis/redis-instance.yaml=826dad198aef8867c01bd2dd4dc4ae62041b2819bdd9d908a3de4e34a3070f99
otel-rules/redis/redis-service.yaml=f35d707193ba48e41ab2c6a9e8079eb4392531fc5cade1d0a585df2b656c4bfd
otel-rules/rocketmq/rocketmq-broker.yaml=f51f79b14c32644a3e2eaf6b93e88047e283b8dfa3b94d8ed411fba7e8c6394e
@@ -106,6 +107,10 @@ otel-rules/airflow/airflow-instance.yaml=9fc3d766aae52905bd9c4e499c3f6521fe9372d
otel-rules/airflow/airflow-service.yaml=d96d3d9f926271e7a143fca0fae6ec1c2c5b59261995449843393f9b6e2f274c
otel-rules/banyandb/banyandb-endpoint.yaml=a1b01ab5628c151cb8dbacba118adfdba7c80f6f9dc98ae924fbd530916732d7
otel-rules/banyandb/banyandb-instance-relation.yaml=1ece4bdbe728b1cdb8c48f7b37bc60b320373de709d565843789f828ff423b36
+# 11.1.0-SNAPSHOT sync: AI agent runtime metrics, BanyanDB trace tail sampling
+otel-rules/ai-agent/runtime-instance.yaml=f3f4f08b83f5866e95cf3b778313ff6d1a34c3ffa02d22b72889076c810b45d3
+otel-rules/ai-agent/runtime-service.yaml=feab347d4f1968f8c611e7cf6845044486816b0a81fb7815b0bb57af49c2fd90
+otel-rules/banyandb/banyandb-trace-sampling.yaml=709b509bd0d1096a88066cf10529fb3242fcd35f604c2c0086f2332b96900ebc
# envoy-metrics-rules/
envoy-metrics-rules/envoy.yaml=724fd1bf14a6157b68c5b40499d9ede505c7fceec30146ba5a1822b7cb5e5c1f
diff --git a/oap-graalvm-server/src/test/resources/provider-inventory.properties b/oap-graalvm-server/src/test/resources/provider-inventory.properties
index ff9f991..6972297 100644
--- a/oap-graalvm-server/src/test/resources/provider-inventory.properties
+++ b/oap-graalvm-server/src/test/resources/provider-inventory.properties
@@ -58,6 +58,9 @@ provider.org.apache.skywalking.oap.server.analyzer.provider.AnalyzerModuleProvid
provider.org.apache.skywalking.oap.log.analyzer.v2.provider.LogAnalyzerModuleProvider = ACCEPTED
provider.org.apache.skywalking.oap.server.analyzer.event.EventAnalyzerModuleProvider = ACCEPTED
provider.org.apache.skywalking.oap.analyzer.genai.GenAIAnalyzerModuleProvider = ACCEPTED
+provider.org.apache.skywalking.oap.server.ai.evaluation.AIEvaluationProvider = ACCEPTED
+provider.org.apache.skywalking.oap.server.ai.agent.conversation.AIAgentConversationProvider = ACCEPTED
+provider.org.apache.skywalking.oap.server.ai.agent.conversation.NoneAIAgentConversationProvider = ACCEPTED
# --- Receivers ---
provider.org.apache.skywalking.oap.server.receiver.sharing.server.SharingServerModuleProvider = ACCEPTED
diff --git a/oap-graalvm-server/src/test/resources/rule-file-inventory.properties b/oap-graalvm-server/src/test/resources/rule-file-inventory.properties
index 993c54b..2a2e1fe 100644
--- a/oap-graalvm-server/src/test/resources/rule-file-inventory.properties
+++ b/oap-graalvm-server/src/test/resources/rule-file-inventory.properties
@@ -33,9 +33,10 @@ oal/mesh.oal = ACCEPTED
oal/tcp.oal = ACCEPTED
oal/virtual-gen-ai.oal = ACCEPTED
-# --- meter-analyzer-config (13 files) ---
+# --- meter-analyzer-config (14 files) ---
meter-analyzer-config/continuous-profiling.yaml = ACCEPTED
meter-analyzer-config/datasource.yaml = ACCEPTED
+meter-analyzer-config/gen-ai-model.yaml = ACCEPTED
meter-analyzer-config/go-agent.yaml = ACCEPTED
meter-analyzer-config/go-runtime.yaml = ACCEPTED
meter-analyzer-config/java-agent.yaml = ACCEPTED
@@ -48,10 +49,12 @@ meter-analyzer-config/satellite.yaml = ACCEPTED
meter-analyzer-config/spring-micrometer.yaml = ACCEPTED
meter-analyzer-config/threadpool.yaml = ACCEPTED
-# --- otel-rules (61 files) ---
+# --- otel-rules (64 files) ---
otel-rules/activemq/activemq-broker.yaml = ACCEPTED
otel-rules/activemq/activemq-cluster.yaml = ACCEPTED
otel-rules/activemq/activemq-destination.yaml = ACCEPTED
+otel-rules/ai-agent/runtime-instance.yaml = ACCEPTED
+otel-rules/ai-agent/runtime-service.yaml = ACCEPTED
otel-rules/apisix.yaml = ACCEPTED
otel-rules/airflow/airflow-instance.yaml = ACCEPTED
otel-rules/airflow/airflow-service.yaml = ACCEPTED
@@ -67,6 +70,7 @@ otel-rules/banyandb/banyandb-endpoint.yaml = ACCEPTED
otel-rules/banyandb/banyandb-instance.yaml = ACCEPTED
otel-rules/banyandb/banyandb-instance-relation.yaml = ACCEPTED
otel-rules/banyandb/banyandb-service.yaml = ACCEPTED
+otel-rules/banyandb/banyandb-trace-sampling.yaml = ACCEPTED
otel-rules/bookkeeper/bookkeeper-cluster.yaml = ACCEPTED
otel-rules/bookkeeper/bookkeeper-node.yaml = ACCEPTED
otel-rules/clickhouse/clickhouse-instance.yaml = ACCEPTED
@@ -135,7 +139,8 @@ telegraf-rules/vm.yaml = ACCEPTED
# --- zabbix-rules (1 file) ---
zabbix-rules/agent.yaml = ACCEPTED
-# --- LAL (11 files) ---
+# --- LAL (12 files) ---
+lal/ai-agent.yaml = ACCEPTED
lal/default.yaml = ACCEPTED
lal/envoy-ai-gateway.yaml = ACCEPTED
lal/envoy-als.yaml = ACCEPTED
diff --git a/pom.xml b/pom.xml
index 03ff460..ce6c189 100644
--- a/pom.xml
+++ b/pom.xml
@@ -34,7 +34,7 @@
<maven.compiler.release>25</maven.compiler.release>
<!-- Injected by Makefile from skywalking submodule's <revision> property.
Do not hardcode — always build via `make`. -->
- <skywalking.version>11.0.0</skywalking.version>
+ <skywalking.version>11.1.0-SNAPSHOT</skywalking.version>
<graalvm.version>25.0.0</graalvm.version>
<graalvm.buildtools.version>0.10.4</graalvm.buildtools.version>
<lombok.version>1.18.40</lombok.version>
@@ -164,6 +164,16 @@
<artifactId>gen-ai-analyzer</artifactId>
<version>${skywalking.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>ai-evaluation</artifactId>
+ <version>${skywalking.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.skywalking</groupId>
+ <artifactId>ai-agent-conversation</artifactId>
+ <version>${skywalking.version}</version>
+ </dependency>
<!-- Receivers -->
<dependency>
diff --git a/skywalking b/skywalking
index 6f1fd78..476afad 160000
--- a/skywalking
+++ b/skywalking
@@ -1 +1 @@
-Subproject commit 6f1fd78e872f1d380a14f271c26e8d68eb2430fc
+Subproject commit 476afadecdc0a410bf5c64de48f6ead8e6030b00
diff --git a/test/e2e/cases/ai-agent/docker-compose.yml b/test/e2e/cases/ai-agent/docker-compose.yml
new file mode 100644
index 0000000..62529a1
--- /dev/null
+++ b/test/e2e/cases/ai-agent/docker-compose.yml
@@ -0,0 +1,213 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Upstream test/e2e-v2/cases/ai-agent/banyandb against the native image. The AI Sessionizer (asz)
+# lands Session Data / Session Flow files over OTLP logs; lal/ai-agent.yaml hands them to the
+# pre-compiled ConversationFile output builder, and the ai-agent-conversation module folds them back
+# into asz.view documents that must equal the Sessionizer's own. The one-shot asz steps are upstream's,
+# chained through service_completed_successfully (the asz image is distroless, no shell).
+x-asz: &asz
+ image: ghcr.io/apache/skywalking-ai-sessionizer:${SW_AI_SESSIONIZER_COMMIT}
+ volumes:
+ - aszdata:/asz/data
+ - ../../../../skywalking/test/e2e-v2/cases/ai-agent/fixture.yaml:/asz/fixture.yaml:ro
+ - ../../../../skywalking/test/e2e-v2/cases/ai-agent/three-rounds.yaml:/asz/three-rounds.yaml:ro
+ - ../../../../skywalking/test/e2e-v2/cases/ai-agent/lost-file.yaml:/asz/lost-file.yaml:ro
+ - ../../../../skywalking/test/e2e-v2/cases/ai-agent/workspace-changes.yaml:/asz/workspace-changes.yaml:ro
+ networks:
+ - e2e
+
+services:
+ banyandb:
+ extends:
+ file: ../../../../skywalking/test/e2e-v2/script/docker-compose/base-compose.yml
+ service: banyandb
+ ports:
+ - 17912
+
+ oap:
+ image: skywalking-oap-native:latest
+ expose:
+ - 11800
+ - 12800
+ networks:
+ - e2e
+ ports:
+ - 12800
+ environment:
+ SW_HEALTH_CHECKER: default
+ SW_STORAGE_BANYANDB_TARGETS: banyandb:17912
+ SW_CONFIGURATION: none
+ SW_AI_AGENT_CONVERSATION_MAX_FILE_BYTES: 2097152
+ depends_on:
+ banyandb:
+ condition: service_healthy
+ healthcheck:
+ test: ["CMD-SHELL", "nc -nz 127.0.0.1 11800 || exit 1"]
+ interval: 5s
+ timeout: 60s
+ retries: 120
+
+ asz-volume:
+ image: busybox:1.36
+ command: ["sh", "-c", "chown -R 65532:65532 /asz/data"]
+ volumes:
+ - aszdata:/asz/data
+ networks:
+ - e2e
+
+ # ---- fixture.yaml: three sessions, one stage
+ asz-build:
+ <<: *asz
+ command: ["scenario", "build", "/asz/fixture.yaml", "--format", "sd", "--out", "/asz/data", "--repeat", "3"]
+ depends_on:
+ asz-volume:
+ condition: service_completed_successfully
+ asz-config:
+ image: busybox:1.36
+ command:
+ - sh
+ - -c
+ - "cp /asz/data/asz.yaml /asz/data/asz-local.yaml && printf 'export:\\n otlp:\\n protocol: grpc\\n endpoint: oap:11800\\n service_name: e2e-ai-agent\\n instance_id: e2e-sender\\n' >> /asz/data/asz.yaml"
+ volumes:
+ - aszdata:/asz/data
+ depends_on:
+ asz-build:
+ condition: service_completed_successfully
+ networks:
+ - e2e
+ asz-collect:
+ <<: *asz
+ command: ["collect", "-once", "-config", "/asz/data/asz.yaml"]
+ depends_on:
+ asz-config:
+ condition: service_completed_successfully
+ oap:
+ condition: service_healthy
+ # ---- workspace-changes.yaml: one session whose every tool call changed files
+ wc-build:
+ <<: *asz
+ command: ["scenario", "build", "/asz/workspace-changes.yaml", "--format", "sd", "--out", "/asz/data"]
+ depends_on:
+ asz-collect:
+ condition: service_completed_successfully
+ wc-collect:
+ <<: *asz
+ command: ["collect", "-once", "-config", "/asz/data/asz.yaml"]
+ depends_on:
+ wc-build:
+ condition: service_completed_successfully
+ # ---- three-rounds.yaml: one session in three stages
+ mr-build-1:
+ <<: *asz
+ command: ["scenario", "build", "/asz/three-rounds.yaml", "--format", "sd", "--out", "/asz/data", "--through", "first"]
+ depends_on:
+ wc-collect:
+ condition: service_completed_successfully
+ mr-collect-1:
+ <<: *asz
+ command: ["collect", "-once", "-config", "/asz/data/asz.yaml"]
+ depends_on:
+ mr-build-1:
+ condition: service_completed_successfully
+ mr-build-2:
+ <<: *asz
+ command: ["scenario", "build", "/asz/three-rounds.yaml", "--format", "sd", "--out", "/asz/data", "--through", "second"]
+ depends_on:
+ mr-collect-1:
+ condition: service_completed_successfully
+ mr-collect-2:
+ <<: *asz
+ command: ["collect", "-once", "-config", "/asz/data/asz.yaml"]
+ depends_on:
+ mr-build-2:
+ condition: service_completed_successfully
+ mr-build-3:
+ <<: *asz
+ command: ["scenario", "build", "/asz/three-rounds.yaml", "--format", "sd", "--out", "/asz/data"]
+ depends_on:
+ mr-collect-2:
+ condition: service_completed_successfully
+ mr-collect-3:
+ <<: *asz
+ command: ["collect", "-once", "-config", "/asz/data/asz.yaml"]
+ depends_on:
+ mr-build-3:
+ condition: service_completed_successfully
+ # ---- lost-file.yaml: a landed file lost after a round bound to it
+ lf-build-1:
+ <<: *asz
+ command: ["scenario", "build", "/asz/lost-file.yaml", "--format", "sd", "--out", "/asz/data", "--through", "helped"]
+ depends_on:
+ mr-collect-3:
+ condition: service_completed_successfully
+ lf-collect-1:
+ <<: *asz
+ command: ["collect", "-once", "-config", "/asz/data/asz-local.yaml"]
+ depends_on:
+ lf-build-1:
+ condition: service_completed_successfully
+ lf-build-2:
+ <<: *asz
+ command: ["scenario", "build", "/asz/lost-file.yaml", "--format", "sd", "--out", "/asz/data"]
+ depends_on:
+ lf-collect-1:
+ condition: service_completed_successfully
+ lf-collect-2:
+ <<: *asz
+ command: ["collect", "-once", "-config", "/asz/data/asz-local.yaml"]
+ depends_on:
+ lf-build-2:
+ condition: service_completed_successfully
+ lf-lose:
+ image: busybox:1.36
+ command: ["sh", "-c", "rm /asz/data/c9d9b18b-be85-4906-850d-40a1cd240171/streams/a8e44ac8611c94121/transcript-*.sd"]
+ volumes:
+ - aszdata:/asz/data
+ networks:
+ - e2e
+ depends_on:
+ lf-collect-2:
+ condition: service_completed_successfully
+ lf-push:
+ <<: *asz
+ command: ["push", "-config", "/asz/data/asz.yaml"]
+ depends_on:
+ lf-lose:
+ condition: service_completed_successfully
+
+ asz:
+ <<: *asz
+ command: ["view", "-config", "/asz/data/asz.yaml", "0.0.0.0:8787"]
+ depends_on:
+ lf-push:
+ condition: service_completed_successfully
+
+ # infra-e2e probes published ports with /bin/sh, which the distroless asz image lacks.
+ aszview:
+ image: alpine/socat:1.8.0.0
+ command: ["TCP-LISTEN:8787,fork,reuseaddr", "TCP:asz:8787"]
+ ports:
+ - 8787
+ depends_on:
+ - asz
+ networks:
+ - e2e
+
+volumes:
+ aszdata:
+
+networks:
+ e2e:
diff --git a/test/e2e/cases/ai-agent/e2e.yaml b/test/e2e/cases/ai-agent/e2e.yaml
new file mode 100644
index 0000000..a8d3a0e
--- /dev/null
+++ b/test/e2e/cases/ai-agent/e2e.yaml
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# AI agent conversations E2E — native OAP with BanyanDB.
+# The cases mirror upstream test/e2e-v2/cases/ai-agent/ai-agent-cases.yaml one for one. They are
+# inlined because upstream's queries call verify.sh by an upstream-root-relative path; the script and
+# every expected file are still upstream's, so the assertions track upstream.
+
+setup:
+ env: compose
+ file: docker-compose.yml
+ timeout: 20m
+ init-system-environment: ../../script/env
+ steps:
+ - name: set PATH
+ command: export PATH=/tmp/skywalking-infra-e2e/bin:$PATH
+ - name: install yq
+ command: bash skywalking/test/e2e-v2/script/prepare/setup-e2e-shell/install.sh yq
+ - name: install swctl
+ command: bash skywalking/test/e2e-v2/script/prepare/setup-e2e-shell/install.sh swctl
+
+verify:
+ retry:
+ count: 30
+ interval: 10s
+ cases:
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql service layer AI_AGENT
+ expected: ../../../../skywalking/test/e2e-v2/cases/ai-agent/expected/service.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql instance list --service-name="e2e-ai-agent"
+ expected: ../../../../skywalking/test/e2e-v2/cases/ai-agent/expected/instance.yml
+ - query: bash skywalking/test/e2e-v2/cases/ai-agent/verify.sh list http://${oap_host}:${oap_12800}
+ expected: ../../../../skywalking/test/e2e-v2/cases/ai-agent/expected/conversations.yml
+ - query: bash skywalking/test/e2e-v2/cases/ai-agent/verify.sh list-limit http://${oap_host}:${oap_12800}
+ expected: ../../../../skywalking/test/e2e-v2/cases/ai-agent/expected/conversations-limit.yml
+ - query: bash skywalking/test/e2e-v2/cases/ai-agent/verify.sh list-instance http://${oap_host}:${oap_12800}
+ expected: ../../../../skywalking/test/e2e-v2/cases/ai-agent/expected/conversations-instance.yml
+ - query: bash skywalking/test/e2e-v2/cases/ai-agent/verify.sh list-window http://${oap_host}:${oap_12800}
+ expected: ../../../../skywalking/test/e2e-v2/cases/ai-agent/expected/conversations-window.yml
+ - query: bash skywalking/test/e2e-v2/cases/ai-agent/verify.sh list-filters http://${oap_host}:${oap_12800}
+ expected: ../../../../skywalking/test/e2e-v2/cases/ai-agent/expected/conversations-filters.yml
+ - query: bash skywalking/test/e2e-v2/cases/ai-agent/verify.sh views http://${oap_host}:${oap_12800} http://${aszview_host}:${aszview_8787}
+ expected: ../../../../skywalking/test/e2e-v2/cases/ai-agent/expected/views.yml
+ - query: bash skywalking/test/e2e-v2/cases/ai-agent/verify.sh list-horizon http://${oap_host}:${oap_12800}
+ expected: ../../../../skywalking/test/e2e-v2/cases/ai-agent/expected/list-horizon.yml
+ - query: bash skywalking/test/e2e-v2/cases/ai-agent/verify.sh view-horizon http://${oap_host}:${oap_12800}
+ expected: ../../../../skywalking/test/e2e-v2/cases/ai-agent/expected/view-horizon.yml
+ - query: bash skywalking/test/e2e-v2/cases/ai-agent/verify.sh raw-files http://${oap_host}:${oap_12800}
+ expected: ../../../../skywalking/test/e2e-v2/cases/ai-agent/expected/raw-files.yml
+ - query: bash skywalking/test/e2e-v2/cases/ai-agent/verify.sh reject http://${oap_host}:${oap_12800}
+ expected: ../../../../skywalking/test/e2e-v2/cases/ai-agent/expected/reject.yml
+ - query: bash skywalking/test/e2e-v2/cases/ai-agent/verify.sh multi-round http://${oap_host}:${oap_12800}
+ expected: ../../../../skywalking/test/e2e-v2/cases/ai-agent/expected/multi-round.yml
+ - query: bash skywalking/test/e2e-v2/cases/ai-agent/verify.sh lost-file http://${oap_host}:${oap_12800}
+ expected: ../../../../skywalking/test/e2e-v2/cases/ai-agent/expected/lost-file.yml
+ - query: bash skywalking/test/e2e-v2/cases/ai-agent/verify.sh gap http://${oap_host}:${oap_12800}
+ expected: ../../../../skywalking/test/e2e-v2/cases/ai-agent/expected/gap.yml
+ - query: bash skywalking/test/e2e-v2/cases/ai-agent/verify.sh size http://${oap_host}:${oap_12800}
+ expected: ../../../../skywalking/test/e2e-v2/cases/ai-agent/expected/size.yml
+ - query: bash skywalking/test/e2e-v2/cases/ai-agent/verify.sh changes http://${oap_host}:${oap_12800}
+ expected: ../../../../skywalking/test/e2e-v2/cases/ai-agent/expected/changes.yml
+ - query: bash skywalking/test/e2e-v2/cases/ai-agent/verify.sh metrics http://${oap_host}:${oap_12800}
+ expected: ../../../../skywalking/test/e2e-v2/cases/ai-agent/expected/metrics.yml
diff --git a/test/e2e/cases/virtual-genai/docker-compose.yml b/test/e2e/cases/banyandb-auth-rotation/docker-compose.yml
similarity index 58%
copy from test/e2e/cases/virtual-genai/docker-compose.yml
copy to test/e2e/cases/banyandb-auth-rotation/docker-compose.yml
index cd86f1b..d4aff07 100644
--- a/test/e2e/cases/virtual-genai/docker-compose.yml
+++ b/test/e2e/cases/banyandb-auth-rotation/docker-compose.yml
@@ -13,27 +13,42 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+# Upstream test/e2e-v2/cases/storage/banyandb/auth-rotation against the native image: BanyanDB with
+# basic auth, the OAP booting with admin/admin and then picking up rotated credentials from
+# SW_STORAGE_BANYANDB_SECRETS_MANAGEMENT_FILE without a restart.
services:
banyandb:
extends:
file: ../../../../skywalking/test/e2e-v2/script/docker-compose/base-compose.yml
service: banyandb
- ports:
- - 17912
+ volumes:
+ - ../../../../skywalking/test/e2e-v2/cases/storage/banyandb/auth-rotation/auth-config.yaml:/auth-config.yaml
+ entrypoint: /bin/sh -c
+ command: |
+ "chmod 600 /auth-config.yaml &&
+ /banyand standalone --measure-metadata-cache-wait-duration 1m --stream-metadata-cache-wait-duration 1m --auth-config-file=/auth-config.yaml"
+ networks:
+ - e2e
oap:
image: skywalking-oap-native:latest
expose:
- 11800
- 12800
+ - 17128
networks:
- e2e
ports:
- 12800
+ - 17128
environment:
SW_HEALTH_CHECKER: default
- SW_STORAGE_BANYANDB_TARGETS: banyandb:17912
SW_CONFIGURATION: none
+ SW_STORAGE_BANYANDB_TARGETS: banyandb:17912
+ # Boot credentials; the secrets file does not exist yet, the test creates it (the rotation).
+ SW_STORAGE_BANYANDB_USER: admin
+ SW_STORAGE_BANYANDB_PASSWORD: admin
+ SW_STORAGE_BANYANDB_SECRETS_MANAGEMENT_FILE: /tmp/bydb-secrets.properties
depends_on:
banyandb:
condition: service_healthy
@@ -51,6 +66,14 @@ services:
networks:
- e2e
+ consumer-jar:
+ image: "ghcr.io/apache/skywalking/e2e-service-consumer:${SW_E2E_SERVICE_COMMIT}"
+ entrypoint: ["cp", "/app.jar", "/jars/services_consumer.jar"]
+ volumes:
+ - service-jars:/jars
+ networks:
+ - e2e
+
provider:
image: "ghcr.io/apache/skywalking-java/skywalking-java:${SW_AGENT_JAVA_COMMIT}-java${SW_AGENT_JDK_VERSION}"
command: ["java", "-jar", "/jars/services_provider.jar"]
@@ -67,6 +90,8 @@ services:
SW_LOGGING_OUTPUT: CONSOLE
SW_AGENT_NAME: e2e-service-provider
SW_AGENT_INSTANCE_NAME: provider1
+ SW_AGENT_COLLECTOR_GET_PROFILE_TASK_INTERVAL: 1
+ SW_AGENT_COLLECTOR_GET_AGENT_DYNAMIC_CONFIG_INTERVAL: 1
SW_METER_ACTIVE: 'false'
healthcheck:
test: ["CMD", "bash", "-c", "cat < /dev/null > /dev/tcp/127.0.0.1/9090"]
@@ -79,32 +104,36 @@ services:
provider-jar:
condition: service_completed_successfully
- spring-ai-examples:
- build:
- context: ../../../../skywalking/test/e2e-v2/cases/virtual-genai
- dockerfile: Dockerfile.provider
- args:
- - SW_AGENT_JDK_VERSION=17
- - SW_AGENT_JAVA_COMMIT=${SW_AGENT_JAVA_COMMIT}
- ports:
- - "9260:8080"
+ consumer:
+ image: "ghcr.io/apache/skywalking-java/skywalking-java:${SW_AGENT_JAVA_COMMIT}-java${SW_AGENT_JDK_VERSION}"
+ command: ["java", "-jar", "/jars/services_consumer.jar"]
+ volumes:
+ - service-jars:/jars
networks:
- e2e
+ expose:
+ - 9092
+ ports:
+ - 9092
environment:
- OPENAI_API_KEY: dummy-key-not-used
- SPRING_AI_OPENAI_BASE_URL: http://provider:9090/llm/v1
SW_AGENT_COLLECTOR_BACKEND_SERVICES: oap:11800
SW_LOGGING_OUTPUT: CONSOLE
- SW_AGENT_NAME: e2e-spring-ai
- SW_AGENT_INSTANCE_NAME: spring-ai-examples
+ PROVIDER_URL: http://provider:9090
+ SW_AGENT_NAME: e2e-service-consumer
+ SW_AGENT_INSTANCE_NAME: consumer1
+ SW_AGENT_COLLECTOR_GET_PROFILE_TASK_INTERVAL: 1
+ SW_AGENT_COLLECTOR_GET_AGENT_DYNAMIC_CONFIG_INTERVAL: 1
+ SW_METER_ACTIVE: 'false'
healthcheck:
- test: ["CMD", "sh", "-c", "nc -nz 127.0.0.1 8080"]
+ test: ["CMD", "bash", "-c", "cat < /dev/null > /dev/tcp/127.0.0.1/9092"]
interval: 5s
timeout: 60s
retries: 120
depends_on:
- oap:
+ provider:
condition: service_healthy
+ consumer-jar:
+ condition: service_completed_successfully
volumes:
service-jars:
diff --git a/test/e2e/cases/banyandb-auth-rotation/e2e.yaml b/test/e2e/cases/banyandb-auth-rotation/e2e.yaml
new file mode 100644
index 0000000..4bb5b1f
--- /dev/null
+++ b/test/e2e/cases/banyandb-auth-rotation/e2e.yaml
@@ -0,0 +1,77 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# BanyanDB credential hot-reload E2E — native OAP.
+# The rotation steps are upstream's (test/e2e-v2/cases/storage/banyandb/auth-rotation/e2e.yaml):
+# write the secrets file inside the OAP container and assert the reload through the OAP log, then
+# run the shared storage suite against the reconnected OAP.
+
+setup:
+ env: compose
+ file: docker-compose.yml
+ timeout: 20m
+ init-system-environment: ../../script/env
+ steps:
+ - name: set PATH
+ command: export PATH=/tmp/skywalking-infra-e2e/bin:$PATH
+ - name: install yq
+ command: bash skywalking/test/e2e-v2/script/prepare/setup-e2e-shell/install.sh yq
+ - name: install swctl
+ command: bash skywalking/test/e2e-v2/script/prepare/setup-e2e-shell/install.sh swctl
+
+trigger:
+ action: http
+ interval: 5s
+ times: -1
+ url: http://${consumer_host}:${consumer_9092}/users
+ method: POST
+ body: '{"id":"123","name":"skywalking"}'
+ headers:
+ "Content-Type": "application/json"
+
+verify:
+ retry:
+ count: 20
+ interval: 10s
+ cases:
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql service layer GENERAL
+ expected: ../../../../skywalking/test/e2e-v2/cases/storage/expected/service.yml
+
+ - query: |
+ oap=$(docker ps -q --filter label=com.docker.compose.service=oap)
+ docker exec $oap bash -c 'printf "user=test\npassword=test\n" > /tmp/bydb-secrets.properties'
+ docker logs $oap 2>&1 \
+ | grep -Fq "Applied the BanyanDB credentials reloaded from /tmp/bydb-secrets.properties, user=test" \
+ && echo credentials-applied
+ expected: ../../../../skywalking/test/e2e-v2/cases/storage/banyandb/auth-rotation/expected/credentials-applied.yml
+
+ - query: |
+ oap=$(docker ps -q --filter label=com.docker.compose.service=oap)
+ docker exec $oap bash -c 'printf "user=admin\n" > /tmp/bydb-secrets.properties'
+ docker logs $oap 2>&1 | grep -Fq "Applied incomplete credentials from /tmp/bydb-secrets.properties" \
+ && docker logs $oap 2>&1 | grep -Fq "UNAUTHENTICATED: Invalid credentials" \
+ && echo credentials-incomplete
+ expected: ../../../../skywalking/test/e2e-v2/cases/storage/banyandb/auth-rotation/expected/credentials-incomplete.yml
+
+ - query: |
+ oap=$(docker ps -q --filter label=com.docker.compose.service=oap)
+ docker exec $oap bash -c 'printf "user=admin\npassword=admin\n" > /tmp/bydb-secrets.properties'
+ docker logs $oap 2>&1 \
+ | grep -Fq "Applied the BanyanDB credentials reloaded from /tmp/bydb-secrets.properties, user=admin" \
+ && echo credentials-restored
+ expected: ../../../../skywalking/test/e2e-v2/cases/storage/banyandb/auth-rotation/expected/credentials-restored.yml
+
+ - includes:
+ - ../../../../skywalking/test/e2e-v2/cases/storage/storage-cases.yaml
diff --git a/test/e2e/cases/virtual-genai/docker-compose.yml b/test/e2e/cases/banyandb-ca-rotation/docker-compose.yml
similarity index 56%
copy from test/e2e/cases/virtual-genai/docker-compose.yml
copy to test/e2e/cases/banyandb-ca-rotation/docker-compose.yml
index cd86f1b..1988ec4 100644
--- a/test/e2e/cases/virtual-genai/docker-compose.yml
+++ b/test/e2e/cases/banyandb-ca-rotation/docker-compose.yml
@@ -13,27 +13,41 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+# Upstream test/e2e-v2/cases/storage/banyandb/ca-rotation against the native image: BanyanDB over
+# TLS, the OAP rebuilding its gRPC channel when SW_STORAGE_BANYANDB_SSL_TRUST_CA_PATH is rotated.
services:
banyandb:
extends:
file: ../../../../skywalking/test/e2e-v2/script/docker-compose/base-compose.yml
service: banyandb
- ports:
- - 17912
+ volumes:
+ - ../../../../skywalking/test/e2e-v2/cases/storage/banyandb/tls/tls:/tls
+ command: standalone --tls=true --key-file=/tls/cert.key --cert-file=/tls/cert.crt --http-grpc-cert-file=/tls/ca.crt
+ networks:
+ - e2e
oap:
image: skywalking-oap-native:latest
expose:
- 11800
- 12800
+ - 17128
networks:
- e2e
ports:
- 12800
+ - 17128
+ volumes:
+ - ../../../../skywalking/test/e2e-v2/cases/storage/banyandb/tls/tls:/tls-src:ro
+ - ../../../../skywalking/test/e2e-v2/cases/storage/banyandb/ca-rotation/wrong-ca.crt:/tls-src-wrong/wrong-ca.crt:ro
+ # TLS is only enabled when the trust file exists at boot, and the test rewrites it, so it is
+ # seeded into /tmp before the image's normal command line takes over.
+ entrypoint: ["/bin/sh", "-c", "cp /tls-src/cert.crt /tmp/bydb-ca.crt && exec ./oap-server -Dlog4j2.configurationFile=config/log4j2.xml"]
environment:
SW_HEALTH_CHECKER: default
- SW_STORAGE_BANYANDB_TARGETS: banyandb:17912
SW_CONFIGURATION: none
+ SW_STORAGE_BANYANDB_TARGETS: banyandb:17912
+ SW_STORAGE_BANYANDB_SSL_TRUST_CA_PATH: /tmp/bydb-ca.crt
depends_on:
banyandb:
condition: service_healthy
@@ -51,6 +65,14 @@ services:
networks:
- e2e
+ consumer-jar:
+ image: "ghcr.io/apache/skywalking/e2e-service-consumer:${SW_E2E_SERVICE_COMMIT}"
+ entrypoint: ["cp", "/app.jar", "/jars/services_consumer.jar"]
+ volumes:
+ - service-jars:/jars
+ networks:
+ - e2e
+
provider:
image: "ghcr.io/apache/skywalking-java/skywalking-java:${SW_AGENT_JAVA_COMMIT}-java${SW_AGENT_JDK_VERSION}"
command: ["java", "-jar", "/jars/services_provider.jar"]
@@ -67,6 +89,8 @@ services:
SW_LOGGING_OUTPUT: CONSOLE
SW_AGENT_NAME: e2e-service-provider
SW_AGENT_INSTANCE_NAME: provider1
+ SW_AGENT_COLLECTOR_GET_PROFILE_TASK_INTERVAL: 1
+ SW_AGENT_COLLECTOR_GET_AGENT_DYNAMIC_CONFIG_INTERVAL: 1
SW_METER_ACTIVE: 'false'
healthcheck:
test: ["CMD", "bash", "-c", "cat < /dev/null > /dev/tcp/127.0.0.1/9090"]
@@ -79,32 +103,36 @@ services:
provider-jar:
condition: service_completed_successfully
- spring-ai-examples:
- build:
- context: ../../../../skywalking/test/e2e-v2/cases/virtual-genai
- dockerfile: Dockerfile.provider
- args:
- - SW_AGENT_JDK_VERSION=17
- - SW_AGENT_JAVA_COMMIT=${SW_AGENT_JAVA_COMMIT}
- ports:
- - "9260:8080"
+ consumer:
+ image: "ghcr.io/apache/skywalking-java/skywalking-java:${SW_AGENT_JAVA_COMMIT}-java${SW_AGENT_JDK_VERSION}"
+ command: ["java", "-jar", "/jars/services_consumer.jar"]
+ volumes:
+ - service-jars:/jars
networks:
- e2e
+ expose:
+ - 9092
+ ports:
+ - 9092
environment:
- OPENAI_API_KEY: dummy-key-not-used
- SPRING_AI_OPENAI_BASE_URL: http://provider:9090/llm/v1
SW_AGENT_COLLECTOR_BACKEND_SERVICES: oap:11800
SW_LOGGING_OUTPUT: CONSOLE
- SW_AGENT_NAME: e2e-spring-ai
- SW_AGENT_INSTANCE_NAME: spring-ai-examples
+ PROVIDER_URL: http://provider:9090
+ SW_AGENT_NAME: e2e-service-consumer
+ SW_AGENT_INSTANCE_NAME: consumer1
+ SW_AGENT_COLLECTOR_GET_PROFILE_TASK_INTERVAL: 1
+ SW_AGENT_COLLECTOR_GET_AGENT_DYNAMIC_CONFIG_INTERVAL: 1
+ SW_METER_ACTIVE: 'false'
healthcheck:
- test: ["CMD", "sh", "-c", "nc -nz 127.0.0.1 8080"]
+ test: ["CMD", "bash", "-c", "cat < /dev/null > /dev/tcp/127.0.0.1/9092"]
interval: 5s
timeout: 60s
retries: 120
depends_on:
- oap:
+ provider:
condition: service_healthy
+ consumer-jar:
+ condition: service_completed_successfully
volumes:
service-jars:
diff --git a/test/e2e/cases/banyandb-ca-rotation/e2e.yaml b/test/e2e/cases/banyandb-ca-rotation/e2e.yaml
new file mode 100644
index 0000000..aebc4cf
--- /dev/null
+++ b/test/e2e/cases/banyandb-ca-rotation/e2e.yaml
@@ -0,0 +1,69 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# BanyanDB TLS trust CA hot-reload E2E — native OAP.
+# The rotation steps are upstream's (test/e2e-v2/cases/storage/banyandb/ca-rotation/e2e.yaml):
+# rotate the trust file to an unrelated CA (the rebuilt channel must be rejected), then back, then
+# run the shared storage suite against the reconnected OAP.
+
+setup:
+ env: compose
+ file: docker-compose.yml
+ timeout: 20m
+ init-system-environment: ../../script/env
+ steps:
+ - name: set PATH
+ command: export PATH=/tmp/skywalking-infra-e2e/bin:$PATH
+ - name: install yq
+ command: bash skywalking/test/e2e-v2/script/prepare/setup-e2e-shell/install.sh yq
+ - name: install swctl
+ command: bash skywalking/test/e2e-v2/script/prepare/setup-e2e-shell/install.sh swctl
+
+trigger:
+ action: http
+ interval: 5s
+ times: -1
+ url: http://${consumer_host}:${consumer_9092}/users
+ method: POST
+ body: '{"id":"123","name":"skywalking"}'
+ headers:
+ "Content-Type": "application/json"
+
+verify:
+ retry:
+ count: 20
+ interval: 10s
+ cases:
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql service layer GENERAL
+ expected: ../../../../skywalking/test/e2e-v2/cases/storage/expected/service.yml
+
+ - query: |
+ oap=$(docker ps -q --filter label=com.docker.compose.service=oap)
+ docker exec $oap bash -c 'cat /tls-src-wrong/wrong-ca.crt > /tmp/bydb-ca.crt'
+ docker logs $oap 2>&1 \
+ | grep -Fq "Rebuilt the BanyanDB channel to reload the TLS trust CA from /tmp/bydb-ca.crt" \
+ && docker logs $oap 2>&1 | grep -Fq "PKIX path building failed" \
+ && echo ca-rejected
+ expected: ../../../../skywalking/test/e2e-v2/cases/storage/banyandb/ca-rotation/expected/ca-rejected.yml
+
+ - query: |
+ oap=$(docker ps -q --filter label=com.docker.compose.service=oap)
+ docker exec $oap bash -c 'cat /tls-src/cert.crt > /tmp/bydb-ca.crt'
+ [ "$(docker logs $oap 2>&1 | grep -Fc 'Rebuilt the BanyanDB channel')" -ge 2 ] \
+ && echo ca-restored
+ expected: ../../../../skywalking/test/e2e-v2/cases/storage/banyandb/ca-rotation/expected/ca-restored.yml
+
+ - includes:
+ - ../../../../skywalking/test/e2e-v2/cases/storage/storage-cases.yaml
diff --git a/test/e2e/cases/banyandb-trace-sampling/docker-compose.yml b/test/e2e/cases/banyandb-trace-sampling/docker-compose.yml
new file mode 100644
index 0000000..8b73064
--- /dev/null
+++ b/test/e2e/cases/banyandb-trace-sampling/docker-compose.yml
@@ -0,0 +1,102 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Upstream test/e2e-v2/cases/banyandb/trace-sampling against the native image: a plugin-capable
+# BanyanDB node runs the sw-trace-sampler chain, an OTel collector scrapes its :2121 endpoint, and
+# otel-rules/banyandb/banyandb-trace-sampling.yaml turns that into meter_banyandb_trace_sampling_*.
+# See the upstream compose for why each BanyanDB/OAP knob is set the way it is.
+services:
+ plugin-carrier:
+ image: "ghcr.io/apache/skywalking-banyandb:${SW_BANYANDB_COMMIT}-plugins-carrier"
+ command: sh -c "cp /plugins/*.so /shared/ && ls -l /shared"
+ volumes:
+ - plugins:/shared
+ networks:
+ - e2e
+
+ # Distroless host: no shell, so no healthcheck and no published ports.
+ banyandb:
+ image: "ghcr.io/apache/skywalking-banyandb:${SW_BANYANDB_COMMIT}-plugins"
+ command: standalone --measure-metadata-cache-wait-duration 1m --stream-metadata-cache-wait-duration 1m --trace-pipeline-native-plugin-enabled=true --trace-pipeline-trusted-plugin-dir=/plugins --trace-max-merge-parts=2 --trace-flush-timeout=500ms
+ volumes:
+ - plugins:/plugins
+ depends_on:
+ plugin-carrier:
+ condition: service_completed_successfully
+ networks:
+ - e2e
+
+ otel-collector:
+ image: otel/opentelemetry-collector:${OTEL_COLLECTOR_VERSION}
+ networks:
+ - e2e
+ command: [ "--config=/etc/otel-collector-config.yaml" ]
+ volumes:
+ - ../../../../skywalking/test/e2e-v2/cases/banyandb/trace-sampling/otel-collector-config.yaml:/etc/otel-collector-config.yaml
+ expose:
+ - 55678
+ depends_on:
+ banyandb:
+ condition: service_started
+
+ # The distroless BanyanDB has no healthcheck, and the native OAP reaches storage well before a
+ # JVM one would, so it waits for the port itself instead of exiting on the first refused connect.
+ oap:
+ image: skywalking-oap-native:latest
+ entrypoint: ["/bin/sh", "-c", "until nc -z banyandb 17912; do sleep 1; done; exec ./oap-server -Dlog4j2.configurationFile=config/log4j2.xml"]
+ expose:
+ - 11800
+ - 12800
+ networks:
+ - e2e
+ ports:
+ - 12800
+ environment:
+ SW_HEALTH_CHECKER: default
+ SW_CONFIGURATION: none
+ SW_STORAGE_BANYANDB_TARGETS: "banyandb:17912"
+ SW_STORAGE_BANYANDB_TRACE_PIPELINE_ENABLED: "true"
+ SW_STORAGE_BANYANDB_TRACE_SHARD_NUM: "1"
+ SW_STORAGE_BANYANDB_TRACE_SAMPLER_DURATION_THRESHOLD_MS: "60000"
+ SW_STORAGE_BANYANDB_TRACE_SAMPLER_KEEP_ERRORS: "false"
+ SW_STORAGE_BANYANDB_TRACE_SAMPLER_HEALTHY_SAMPLE_RATE: "0"
+ SW_STORAGE_BANYANDB_TRACE_PIPELINE_MERGE_GRACE_SECONDS: "20"
+ depends_on:
+ banyandb:
+ condition: service_started
+ healthcheck:
+ test: ["CMD-SHELL", "nc -nz 127.0.0.1 11800 || exit 1"]
+ interval: 5s
+ timeout: 60s
+ retries: 120
+
+ # Fuel for the sampler only; nothing asserts the traces. Upstream bind-mounts the jar it builds,
+ # the published image carries the same Java 8 jar. MINUS_DAYS=0 keeps them in the hot stage.
+ trace-mocker:
+ image: "ghcr.io/apache/skywalking/trace-mocker:${SW_E2E_SERVICE_COMMIT}"
+ environment:
+ OAP_HOST: oap
+ MINUS_DAYS: 0
+ networks:
+ - e2e
+ depends_on:
+ oap:
+ condition: service_healthy
+
+volumes:
+ plugins:
+
+networks:
+ e2e:
diff --git a/test/e2e/cases/banyandb-trace-sampling/e2e.yaml b/test/e2e/cases/banyandb-trace-sampling/e2e.yaml
new file mode 100644
index 0000000..eb1dfc3
--- /dev/null
+++ b/test/e2e/cases/banyandb-trace-sampling/e2e.yaml
@@ -0,0 +1,38 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# BanyanDB trace tail sampling monitoring E2E — native OAP.
+# Reuses upstream verification cases and expected files.
+
+setup:
+ env: compose
+ file: docker-compose.yml
+ timeout: 25m
+ init-system-environment: ../../script/env
+ steps:
+ - name: set PATH
+ command: export PATH=/tmp/skywalking-infra-e2e/bin:$PATH
+ - name: install yq
+ command: bash skywalking/test/e2e-v2/script/prepare/setup-e2e-shell/install.sh yq
+ - name: install swctl
+ command: bash skywalking/test/e2e-v2/script/prepare/setup-e2e-shell/install.sh swctl
+
+verify:
+ retry:
+ count: 60
+ interval: 10s
+ cases:
+ - includes:
+ - ../../../../skywalking/test/e2e-v2/cases/banyandb/trace-sampling/trace-sampling-cases.yaml
diff --git a/test/e2e/cases/virtual-genai/docker-compose.yml b/test/e2e/cases/virtual-genai/docker-compose.yml
index cd86f1b..5188963 100644
--- a/test/e2e/cases/virtual-genai/docker-compose.yml
+++ b/test/e2e/cases/virtual-genai/docker-compose.yml
@@ -30,10 +30,17 @@ services:
- e2e
ports:
- 12800
+ volumes:
+ - ../../../../skywalking/test/e2e-v2/cases/virtual-genai/ai-evaluation.yml:/skywalking/config/ai-evaluation.yml
environment:
SW_HEALTH_CHECKER: default
SW_STORAGE_BANYANDB_TARGETS: banyandb:17912
SW_CONFIGURATION: none
+ SW_AI_EVALUATION: default
+ SW_AI_EVALUATION_SAMPLE_RATE: 1000000
+ AI_EVALUATION_ENDPOINT: http://mock-llm-server:9090/llm/evaluation/v1/chat/completions
+ AI_EVALUATION_MODEL: e2e-judge
+ AI_EVALUATION_API_KEY: e2e-api-key
depends_on:
banyandb:
condition: service_healthy
@@ -43,60 +50,48 @@ services:
timeout: 60s
retries: 120
- provider-jar:
- image: "ghcr.io/apache/skywalking/e2e-service-provider:${SW_E2E_SERVICE_COMMIT}"
- entrypoint: ["cp", "/app.jar", "/jars/services_provider.jar"]
- volumes:
- - service-jars:/jars
- networks:
- - e2e
-
- provider:
- image: "ghcr.io/apache/skywalking-java/skywalking-java:${SW_AGENT_JAVA_COMMIT}-java${SW_AGENT_JDK_VERSION}"
- command: ["java", "-jar", "/jars/services_provider.jar"]
- volumes:
- - service-jars:/jars
+ # The OpenAI-compatible stand-in for both the chat completions Spring AI calls and the judge
+ # completions OAP's AI evaluation calls. Upstream bind-mounts the jar it builds; the published
+ # image carries the same jar.
+ mock-llm-server:
+ image: "ghcr.io/apache/skywalking/e2e-mock-llm-server:${SW_E2E_SERVICE_COMMIT}"
networks:
- e2e
expose:
- 9090
- ports:
- - 9090
- environment:
- SW_AGENT_COLLECTOR_BACKEND_SERVICES: oap:11800
- SW_LOGGING_OUTPUT: CONSOLE
- SW_AGENT_NAME: e2e-service-provider
- SW_AGENT_INSTANCE_NAME: provider1
- SW_METER_ACTIVE: 'false'
healthcheck:
test: ["CMD", "bash", "-c", "cat < /dev/null > /dev/tcp/127.0.0.1/9090"]
interval: 5s
timeout: 60s
retries: 120
- depends_on:
- oap:
- condition: service_healthy
- provider-jar:
- condition: service_completed_successfully
- spring-ai-examples:
- build:
- context: ../../../../skywalking/test/e2e-v2/cases/virtual-genai
- dockerfile: Dockerfile.provider
- args:
- - SW_AGENT_JDK_VERSION=17
- - SW_AGENT_JAVA_COMMIT=${SW_AGENT_JAVA_COMMIT}
+ spring-ai-jar:
+ image: "ghcr.io/apache/skywalking/e2e-spring-ai-service:${SW_E2E_SERVICE_COMMIT}"
+ entrypoint: ["cp", "/app.jar", "/jars/spring_ai_service.jar"]
+ volumes:
+ - service-jars:/jars
+ networks:
+ - e2e
+
+ # Spring AI needs Java 17, so this pins the agent image's java17 tag.
+ spring-ai-service:
+ image: "ghcr.io/apache/skywalking-java/skywalking-java:${SW_AGENT_JAVA_COMMIT}-java17"
+ command: ["java", "-jar", "/jars/spring_ai_service.jar"]
+ volumes:
+ - service-jars:/jars
ports:
- "9260:8080"
networks:
- e2e
environment:
- OPENAI_API_KEY: dummy-key-not-used
- SPRING_AI_OPENAI_BASE_URL: http://provider:9090/llm/v1
+ OPENAI_API_KEY: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+ SPRING_AI_OPENAI_BASE_URL: http://mock-llm-server:9090/llm/v1
SW_AGENT_COLLECTOR_BACKEND_SERVICES: oap:11800
SW_LOGGING_OUTPUT: CONSOLE
SW_AGENT_NAME: e2e-spring-ai
- SW_AGENT_INSTANCE_NAME: spring-ai-examples
+ SW_AGENT_INSTANCE_NAME: spring-ai-service
+ SW_PLUGIN_SPRINGAI_COLLECT_INPUT_MESSAGES: "true"
+ SW_PLUGIN_SPRINGAI_COLLECT_OUTPUT_MESSAGES: "true"
healthcheck:
test: ["CMD", "sh", "-c", "nc -nz 127.0.0.1 8080"]
interval: 5s
@@ -105,6 +100,10 @@ services:
depends_on:
oap:
condition: service_healthy
+ mock-llm-server:
+ condition: service_healthy
+ spring-ai-jar:
+ condition: service_completed_successfully
volumes:
service-jars:
diff --git a/test/e2e/script/env b/test/e2e/script/env
index bd9dbad..7be0642 100644
--- a/test/e2e/script/env
+++ b/test/e2e/script/env
@@ -17,17 +17,18 @@
# Update when syncing the skywalking submodule.
SW_AGENT_JAVA_COMMIT=ac0df43d7140e726eba9e5e5b1b75cf364c71dff
SW_BANYANDB_COMMIT=3b83e18fb0481d02e44eaa5df137fcf7b000754b
-SW_CTL_COMMIT=85e5afdb3d55c6e5af66a472c3fe8ac024d11690
+SW_CTL_COMMIT=1b6837da6361f1d735ac9ef1ea8cfc245918ff35
SW_KUBERNETES_COMMIT_SHA=da0e267f877b9b8e5f7728ae4ea7dc7723a2a073
+SW_AI_SESSIONIZER_COMMIT=dd083ccb141d2bd2678003f1a31094b2a37b40c1
# UI image pin for the istio-als e2e test (ghcr.io/apache/skywalking/ui:<sha>).
-# NOT bumped to the synced submodule commit 6f1fd78e87 (v11.0.0): 11.0.0 dropped the bundled UI,
+# NOT bumped to the synced submodule commit: 11.0.0 dropped the bundled UI,
# so ghcr.io/apache/skywalking/ui is no longer published at recent commits. Kept at the
# last commit whose UI image exists. The pins below track the sync.
SW_UPSTREAM_COMMIT=f79451d3de23edb15277f5b6e315dc6779e73e1f
# Last upstream commit that touched test/e2e-v2/java-test-service/ (image must be published)
-SW_E2E_SERVICE_COMMIT=95a296e2c810e6280b16975dd411e9c595a16836
+SW_E2E_SERVICE_COMMIT=51e727359fd00f006cf7a98511ae205176171969
# OTEL collector version for infrastructure monitoring tests
OTEL_COLLECTOR_VERSION=0.102.1