You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by wu...@apache.org on 2026/04/02 08:11:53 UTC
(skywalking) branch feature/zipkin-virtual-genai-e2e created (now 4489858e99)
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a change to branch feature/zipkin-virtual-genai-e2e
in repository https://gitbox.apache.org/repos/asf/skywalking.git
at 4489858e99 Add Zipkin Virtual GenAI e2e test with zipkin_json exporter
This branch includes the following new commits:
new 4489858e99 Add Zipkin Virtual GenAI e2e test with zipkin_json exporter
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
(skywalking) 01/01: Add Zipkin Virtual GenAI e2e test with zipkin_json exporter
Posted by wu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch feature/zipkin-virtual-genai-e2e
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit 4489858e99f149f377fb1a9a0a961e88b2f27abe
Author: Wu Sheng <wu...@foxmail.com>
AuthorDate: Thu Apr 2 16:11:34 2026 +0800
Add Zipkin Virtual GenAI e2e test with zipkin_json exporter
Use zipkin_json exporter instead of zipkin (proto-http) to avoid
irreconcilable protobuf dependency conflict:
- opentelemetry-exporter-zipkin-proto-http requires protobuf~=3.12 (<4.0)
- opentelemetry-proto requires protobuf>=5.0
Remove opentelemetry-exporter-zipkin and opentelemetry-exporter-zipkin-proto-http
from Dockerfile, keep opentelemetry-exporter-zipkin-json for clean single
pip install with no dependency conflicts.
---
.github/workflows/skywalking.yaml | 2 +
docs/en/changes/changes.md | 5 +-
.../cases/zipkin-virtual-genai/docker-compose.yml | 75 ++++++++++++++++++++++
.../zipkin-virtual-genai/e2e.yaml} | 47 ++++++++------
.../zipkin-virtual-genai/expected/instance.yml} | 31 ++-------
.../expected/metrics-has-value-label.yml} | 46 ++++++-------
.../expected/metrics-has-value.yml} | 43 ++++++-------
.../zipkin-virtual-genai/expected/service.yml} | 33 +++-------
.../zipkin-virtual-genai/zipkin-virtual-genai.yaml | 65 +++++++++++++++++++
.../script/dockerfile/Dockerfile-openai.python | 2 -
10 files changed, 228 insertions(+), 121 deletions(-)
diff --git a/.github/workflows/skywalking.yaml b/.github/workflows/skywalking.yaml
index 6b31fa8408..833b1c8863 100644
--- a/.github/workflows/skywalking.yaml
+++ b/.github/workflows/skywalking.yaml
@@ -637,6 +637,8 @@ jobs:
config: test/e2e-v2/cases/virtual-genai/e2e.yaml
- name: OTLP Virtual GenAI
config: test/e2e-v2/cases/otlp-virtual-genai/e2e.yaml
+ - name: Zipkin Virtual GenAI
+ config: test/e2e-v2/cases/zipkin-virtual-genai/e2e.yaml
- name: Nginx
config: test/e2e-v2/cases/nginx/e2e.yaml
diff --git a/docs/en/changes/changes.md b/docs/en/changes/changes.md
index 88032a9829..21bca80f75 100644
--- a/docs/en/changes/changes.md
+++ b/docs/en/changes/changes.md
@@ -3,9 +3,8 @@
#### Project
#### OAP Server
-* Remove Groovy v1-v2 checker modules. Replace with v2-only DSL script execution tests in their
- respective analyzer modules (hierarchy, log-analyzer) and a separate `meter-analyzer-scripts-test`
- module for MAL tests requiring ProcessRegistry FQCN isolation. Delete `test/script-cases/` directory.
+* Add Zipkin Virtual GenAI e2e test. Use `zipkin_json` exporter to avoid protobuf dependency conflict
+ between `opentelemetry-exporter-zipkin-proto-http` (protobuf~=3.12) and `opentelemetry-proto` (protobuf>=5.0).
#### UI
diff --git a/test/e2e-v2/cases/zipkin-virtual-genai/docker-compose.yml b/test/e2e-v2/cases/zipkin-virtual-genai/docker-compose.yml
new file mode 100644
index 0000000000..83af7a65a8
--- /dev/null
+++ b/test/e2e-v2/cases/zipkin-virtual-genai/docker-compose.yml
@@ -0,0 +1,75 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: "3"
+
+services:
+ oap:
+ extends:
+ file: ../../script/docker-compose/base-compose.yml
+ service: oap
+ environment:
+ SW_QUERY_ZIPKIN: default
+ SW_RECEIVER_ZIPKIN: default
+ ports:
+ - "12800:12800"
+ - "11800:11800"
+ - "9411:9411"
+ networks:
+ - e2e
+
+ banyandb:
+ extends:
+ file: ../../script/docker-compose/base-compose.yml
+ service: banyandb
+ ports:
+ - 17912
+
+ provider:
+ extends:
+ file: ../../script/docker-compose/base-compose.yml
+ service: provider
+ ports:
+ - 9090
+ depends_on:
+ oap:
+ condition: service_healthy
+
+ provider-py:
+ build:
+ context: ../../script/
+ dockerfile: dockerfile/Dockerfile-openai.python
+ networks:
+ - e2e
+ ports:
+ - "9091:9091"
+ environment:
+ OPENAI_BASE_URL: http://provider:9090/llm/otlp/v1
+ OPENAI_API_KEY: xxxxxxxxxxxxxxxxxxxxxxxx
+ OTEL_TRACES_EXPORTER: zipkin_json
+ OTEL_EXPORTER_ZIPKIN_ENDPOINT: http://oap:9411/api/v2/spans
+ OTEL_LOG_LEVEL: debug
+ depends_on:
+ oap:
+ condition: service_healthy
+ healthcheck:
+ test: ["CMD", "curl", "-f", "http://127.0.0.1:9091/"]
+ interval: 5s
+ timeout: 10s
+ retries: 5
+ command: ["opentelemetry-instrument", "python3", "/openai-call.py"]
+
+networks:
+ e2e:
diff --git a/test/e2e-v2/script/dockerfile/Dockerfile-openai.python b/test/e2e-v2/cases/zipkin-virtual-genai/e2e.yaml
similarity index 52%
copy from test/e2e-v2/script/dockerfile/Dockerfile-openai.python
copy to test/e2e-v2/cases/zipkin-virtual-genai/e2e.yaml
index 4416074005..0147b6d000 100644
--- a/test/e2e-v2/script/dockerfile/Dockerfile-openai.python
+++ b/test/e2e-v2/cases/zipkin-virtual-genai/e2e.yaml
@@ -13,27 +13,32 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-FROM python:3.10-slim
+# This file is used to show how to write configuration files and can be used to test.
-WORKDIR /app
+setup:
+ env: compose
+ file: docker-compose.yml
+ timeout: 20m
+ init-system-environment: ../../script/env
+ steps:
+ - name: set PATH
+ command: export PATH=/tmp/skywalking-infra-e2e/bin:$PATH
+ - name: install yq
+ command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh yq
+ - name: install swctl
+ command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh swctl
-RUN apt-get update && apt-get install -y --no-install-recommends \
- curl \
- && rm -rf /var/lib/apt/lists/*
-
-RUN pip install --no-cache-dir --upgrade pip && \
- pip install --no-cache-dir \
- openai \
- opentelemetry-api \
- opentelemetry-sdk \
- opentelemetry-instrumentation-openai \
- opentelemetry-exporter-otlp-proto-grpc \
- opentelemetry-exporter-zipkin \
- opentelemetry-exporter-zipkin-json \
- opentelemetry-exporter-zipkin-proto-http \
- opentelemetry-distro
-
-RUN opentelemetry-bootstrap --action=install
-
-COPY python/openai-call.py /openai-call.py
+trigger:
+ action: http
+ interval: 3s
+ times: 5
+ url: http://localhost:9091/
+ method: GET
+verify:
+ retry:
+ count: 60
+ interval: 3s
+ cases:
+ - includes:
+ - ./zipkin-virtual-genai.yaml
diff --git a/test/e2e-v2/script/dockerfile/Dockerfile-openai.python b/test/e2e-v2/cases/zipkin-virtual-genai/expected/instance.yml
similarity index 55%
copy from test/e2e-v2/script/dockerfile/Dockerfile-openai.python
copy to test/e2e-v2/cases/zipkin-virtual-genai/expected/instance.yml
index 4416074005..f74704b2b8 100644
--- a/test/e2e-v2/script/dockerfile/Dockerfile-openai.python
+++ b/test/e2e-v2/cases/zipkin-virtual-genai/expected/instance.yml
@@ -13,27 +13,10 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-FROM python:3.10-slim
-
-WORKDIR /app
-
-RUN apt-get update && apt-get install -y --no-install-recommends \
- curl \
- && rm -rf /var/lib/apt/lists/*
-
-RUN pip install --no-cache-dir --upgrade pip && \
- pip install --no-cache-dir \
- openai \
- opentelemetry-api \
- opentelemetry-sdk \
- opentelemetry-instrumentation-openai \
- opentelemetry-exporter-otlp-proto-grpc \
- opentelemetry-exporter-zipkin \
- opentelemetry-exporter-zipkin-json \
- opentelemetry-exporter-zipkin-proto-http \
- opentelemetry-distro
-
-RUN opentelemetry-bootstrap --action=install
-
-COPY python/openai-call.py /openai-call.py
-
+{{- contains . }}
+- id: {{ notEmpty .id }}
+ name: gpt-4.1-mini-2025-04-14
+ instanceuuid: {{ notEmpty .instanceuuid }}
+ attributes: []
+ language: UNKNOWN
+{{- end }}
diff --git a/test/e2e-v2/script/dockerfile/Dockerfile-openai.python b/test/e2e-v2/cases/zipkin-virtual-genai/expected/metrics-has-value-label.yml
similarity index 55%
copy from test/e2e-v2/script/dockerfile/Dockerfile-openai.python
copy to test/e2e-v2/cases/zipkin-virtual-genai/expected/metrics-has-value-label.yml
index 4416074005..c983c0e19b 100644
--- a/test/e2e-v2/script/dockerfile/Dockerfile-openai.python
+++ b/test/e2e-v2/cases/zipkin-virtual-genai/expected/metrics-has-value-label.yml
@@ -13,27 +13,27 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-FROM python:3.10-slim
-
-WORKDIR /app
-
-RUN apt-get update && apt-get install -y --no-install-recommends \
- curl \
- && rm -rf /var/lib/apt/lists/*
-
-RUN pip install --no-cache-dir --upgrade pip && \
- pip install --no-cache-dir \
- openai \
- opentelemetry-api \
- opentelemetry-sdk \
- opentelemetry-instrumentation-openai \
- opentelemetry-exporter-otlp-proto-grpc \
- opentelemetry-exporter-zipkin \
- opentelemetry-exporter-zipkin-json \
- opentelemetry-exporter-zipkin-proto-http \
- opentelemetry-distro
-
-RUN opentelemetry-bootstrap --action=install
-
-COPY python/openai-call.py /openai-call.py
+debuggingtrace: null
+type: TIME_SERIES_VALUES
+results:
+ {{- contains .results }}
+ - metric:
+ labels:
+ {{- contains .metric.labels }}
+ - key: "p"
+ value: {{ notEmpty .value }}
+ {{- end}}
+ values:
+ {{- contains .values }}
+ - id: {{ notEmpty .id }}
+ value: {{ .value }}
+ traceid: null
+ owner: null
+ - id: {{ notEmpty .id }}
+ value: null
+ traceid: null
+ owner: null
+ {{- end}}
+ {{- end}}
+error: null
diff --git a/test/e2e-v2/script/dockerfile/Dockerfile-openai.python b/test/e2e-v2/cases/zipkin-virtual-genai/expected/metrics-has-value.yml
similarity index 55%
copy from test/e2e-v2/script/dockerfile/Dockerfile-openai.python
copy to test/e2e-v2/cases/zipkin-virtual-genai/expected/metrics-has-value.yml
index 4416074005..979b9b2577 100644
--- a/test/e2e-v2/script/dockerfile/Dockerfile-openai.python
+++ b/test/e2e-v2/cases/zipkin-virtual-genai/expected/metrics-has-value.yml
@@ -13,27 +13,22 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-FROM python:3.10-slim
-
-WORKDIR /app
-
-RUN apt-get update && apt-get install -y --no-install-recommends \
- curl \
- && rm -rf /var/lib/apt/lists/*
-
-RUN pip install --no-cache-dir --upgrade pip && \
- pip install --no-cache-dir \
- openai \
- opentelemetry-api \
- opentelemetry-sdk \
- opentelemetry-instrumentation-openai \
- opentelemetry-exporter-otlp-proto-grpc \
- opentelemetry-exporter-zipkin \
- opentelemetry-exporter-zipkin-json \
- opentelemetry-exporter-zipkin-proto-http \
- opentelemetry-distro
-
-RUN opentelemetry-bootstrap --action=install
-
-COPY python/openai-call.py /openai-call.py
-
+debuggingtrace: null
+type: TIME_SERIES_VALUES
+results:
+ {{- contains .results }}
+ - metric:
+ labels: []
+ values:
+ {{- contains .values }}
+ - id: {{ notEmpty .id }}
+ value: {{ notEmpty .value }}
+ traceid: null
+ owner: null
+ - id: {{ notEmpty .id }}
+ value: null
+ traceid: null
+ owner: null
+ {{- end}}
+ {{- end}}
+error: null
diff --git a/test/e2e-v2/script/dockerfile/Dockerfile-openai.python b/test/e2e-v2/cases/zipkin-virtual-genai/expected/service.yml
similarity index 55%
copy from test/e2e-v2/script/dockerfile/Dockerfile-openai.python
copy to test/e2e-v2/cases/zipkin-virtual-genai/expected/service.yml
index 4416074005..5471c7d596 100644
--- a/test/e2e-v2/script/dockerfile/Dockerfile-openai.python
+++ b/test/e2e-v2/cases/zipkin-virtual-genai/expected/service.yml
@@ -13,27 +13,12 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-FROM python:3.10-slim
-
-WORKDIR /app
-
-RUN apt-get update && apt-get install -y --no-install-recommends \
- curl \
- && rm -rf /var/lib/apt/lists/*
-
-RUN pip install --no-cache-dir --upgrade pip && \
- pip install --no-cache-dir \
- openai \
- opentelemetry-api \
- opentelemetry-sdk \
- opentelemetry-instrumentation-openai \
- opentelemetry-exporter-otlp-proto-grpc \
- opentelemetry-exporter-zipkin \
- opentelemetry-exporter-zipkin-json \
- opentelemetry-exporter-zipkin-proto-http \
- opentelemetry-distro
-
-RUN opentelemetry-bootstrap --action=install
-
-COPY python/openai-call.py /openai-call.py
-
+{{- contains . }}
+- id: {{ b64enc "openai" }}.0
+ name: openai
+ group: ""
+ shortname: openai
+ layers:
+ - VIRTUAL_GENAI
+ normal: false
+{{- end }}
diff --git a/test/e2e-v2/cases/zipkin-virtual-genai/zipkin-virtual-genai.yaml b/test/e2e-v2/cases/zipkin-virtual-genai/zipkin-virtual-genai.yaml
new file mode 100644
index 0000000000..a3c1a998dd
--- /dev/null
+++ b/test/e2e-v2/cases/zipkin-virtual-genai/zipkin-virtual-genai.yaml
@@ -0,0 +1,65 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This file is used to show how to write configuration files and can be used to test.
+
+cases:
+ # service cases
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql service ls
+ expected: expected/service.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=gen_ai_provider_resp_time --service-id=b3BlbmFp.0
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=gen_ai_provider_sla --service-id=b3BlbmFp.0
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=gen_ai_provider_cpm --service-id=b3BlbmFp.0
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=gen_ai_provider_latency_percentile --service-id=b3BlbmFp.0
+ expected: expected/metrics-has-value-label.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=gen_ai_provider_input_tokens_sum --service-id=b3BlbmFp.0
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=gen_ai_provider_input_tokens_avg --service-id=b3BlbmFp.0
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=gen_ai_provider_output_tokens_sum --service-id=b3BlbmFp.0
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=gen_ai_provider_output_tokens_avg --service-id=b3BlbmFp.0
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=gen_ai_provider_total_estimated_cost --service-id=b3BlbmFp.0
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=gen_ai_provider_avg_estimated_cost --service-id=b3BlbmFp.0
+ expected: expected/metrics-has-value.yml
+
+ # instance cases
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql instance ls --service-id=b3BlbmFp.0
+ expected: expected/instance.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=gen_ai_model_call_cpm --service-id=b3BlbmFp.0 --instance-name=gpt-4.1-mini-2025-04-14
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=gen_ai_model_sla --service-id=b3BlbmFp.0 --instance-name=gpt-4.1-mini-2025-04-14
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=gen_ai_model_latency_avg --service-id=b3BlbmFp.0 --instance-name=gpt-4.1-mini-2025-04-14
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=gen_ai_model_latency_percentile --service-id=b3BlbmFp.0 --instance-name=gpt-4.1-mini-2025-04-14
+ expected: expected/metrics-has-value-label.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=gen_ai_model_input_tokens_sum --service-id=b3BlbmFp.0 --instance-name=gpt-4.1-mini-2025-04-14
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=gen_ai_model_input_tokens_avg --service-id=b3BlbmFp.0 --instance-name=gpt-4.1-mini-2025-04-14
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=gen_ai_model_output_tokens_sum --service-id=b3BlbmFp.0 --instance-name=gpt-4.1-mini-2025-04-14
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=gen_ai_model_output_tokens_avg --service-id=b3BlbmFp.0 --instance-name=gpt-4.1-mini-2025-04-14
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=gen_ai_model_total_estimated_cost --service-id=b3BlbmFp.0 --instance-name=gpt-4.1-mini-2025-04-14
+ expected: expected/metrics-has-value.yml
+ - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=gen_ai_model_avg_estimated_cost --service-id=b3BlbmFp.0 --instance-name=gpt-4.1-mini-2025-04-14
+ expected: expected/metrics-has-value.yml
diff --git a/test/e2e-v2/script/dockerfile/Dockerfile-openai.python b/test/e2e-v2/script/dockerfile/Dockerfile-openai.python
index 4416074005..c23f03d603 100644
--- a/test/e2e-v2/script/dockerfile/Dockerfile-openai.python
+++ b/test/e2e-v2/script/dockerfile/Dockerfile-openai.python
@@ -28,9 +28,7 @@ RUN pip install --no-cache-dir --upgrade pip && \
opentelemetry-sdk \
opentelemetry-instrumentation-openai \
opentelemetry-exporter-otlp-proto-grpc \
- opentelemetry-exporter-zipkin \
opentelemetry-exporter-zipkin-json \
- opentelemetry-exporter-zipkin-proto-http \
opentelemetry-distro
RUN opentelemetry-bootstrap --action=install