You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by "wu-sheng (via GitHub)" <gi...@apache.org> on 2026/04/20 11:44:52 UTC
[PR] Implement SWIP-12: WeChat & Alipay Mini Program monitoring [skywalking]
wu-sheng opened a new pull request, #13835:
URL: https://github.com/apache/skywalking/pull/13835
### Implement SWIP-12 — WeChat & Alipay Mini Program monitoring
This PR implements the SWIP-12 design (already merged in #13834) end to end. The SDK side ([SkyAPM mini-program-monitor](https://github.com/SkyAPM/mini-program-monitor)) has already shipped v0.4.0 with the componentIds / `miniprogram.platform` tag / clean `serviceInstance` defaults that this implementation depends on.
Delivered in 5 staged commits for reviewability:
**Stage 1 — Layer enum, componentIds, listener layer mapping.** Adds `WECHAT_MINI_PROGRAM(48)` and `ALIPAY_MINI_PROGRAM(49)` to `Layer.java`; registers `WeChat-MiniProgram: 10002` and `AliPay-MiniProgram: 10003` under the JavaScript block of `component-libraries.yml`; refactors `CommonAnalysisListener` to take `IComponentLibraryCatalogService` via constructor and extends `identifyServiceLayer(SpanLayer)` to also accept `componentId` so SDK-emitted exit spans land on the correct layer. All 5 call sites in `RPCAnalysisListener` + `EndpointDepFromCrossThreadAnalysisListener` pass `span.getComponentId()` through. Factories resolve the catalog service from `CoreModule`. No new SPI — the existing `SegmentParserListenerManager` pipeline picks up the new mapping automatically. Existing listener unit tests updated with a mock catalog stubbed to return `10002`/`10003` so span `componentId=0` doesn't accidentally collide.
**Stage 2 — MAL / LAL / log-MAL rules + `application.yml` wiring.** Four MAL files under `otel-rules/miniprogram/` — per-platform (WeChat / Alipay) × per-scope (service / instance), each with a top-level `filter` gating on `miniprogram_platform` so the two layers never double-count. Service-scoped rules follow the iOS precedent of avoiding `service_instance_id` fragmentation, and add endpoint-scoped variants via the chained `.endpoint(...)` override (APISIX pattern) so per-page dashboards populate from the same file. Alipay drops the WeChat-only metrics (`route_duration`, `script_duration`, `package_load_duration`, `first_paint`). One LAL file `lal/miniprogram.yaml` uses `layer: auto` and script-side dispatch on `miniprogram.platform` — one rule file produces two layers. Its `metrics {}` block emits `miniprogram_error_count` samples that the log-MAL rule `log-mal-rules/miniprogram.yaml` aggregates into per-layer `error_count`. `application.yml` defaults append `miniprogram/
*` to `enabledOtelMetricsRules` and `miniprogram` to `lalFiles` / `malFiles`.
> Note: the SWIP draft's ternary-in-`layer` expression and the nested `metrics { metric_name { … } }` block form don't match the actual LAL grammar — rewritten as `if/else-if` and the flat `name`/`labels`/`value`/`timestamp` form the grammar accepts.
**Stage 3 — UI dashboards + menu.** Two new folders (`ui-initialized-templates/wechat_mini_program/` and `alipay_mini_program/`), each shipping 4 JSON templates: `-root` (`isRoot: true`, service-list landing), `-service` (`Tab` with Overview / Instance / Endpoint / Trace / Log), `-instance` (per-version perf), `-endpoint` (per-page perf). WeChat dashboards use the full perf set; Alipay drops WeChat-only metrics and labels launch/render panels as approximations. Service dashboard adds a dedicated Trace tab — native segments are queryable in the normal trace UI, unlike iOS. `menu.yaml` extends the Mobile group (added by SWIP-11) with two new entries. `UITemplateCheckerTest` passes.
**Stage 4 — Docs.** `docs/en/setup/backend/backend-wechat-mini-program-monitoring.md` and `backend-alipay-mini-program-monitoring.md` with prerequisites, OAP configuration (when defaults are overridden), SDK setup snippet, metric catalog, error-log categories, entity model, SDK version compatibility (v0.2.x / v0.3.x / v0.4.0+), dashboard overview, platform-specific limitations. `docs/menu.yml` extended under Mobile Monitoring. Changelog entry under OAP Server and Documentation. SWIP readme and security README were already updated in the design PR — left untouched.
**Stage 5 — E2E + CI matrix + config-dump.** Two cases under `test/e2e-v2/cases/miniprogram/{wechat,alipay}/` drive hand-crafted OTLP metric + OTLP error-log payloads via curl (iOS-style, no external sim-image dependency). Verify that service is registered under the correct layer, MAL service/instance metrics populate, log-MAL `error_count` is non-zero, and the LAL error log persists. Native-segment layer mapping is covered by the Stage 1 unit-test updates. Both cases registered in `.github/workflows/skywalking.yaml`. `test/e2e-v2/cases/storage/expected/config-dump.yml` mirrors the `application.yml` default changes so the storage e2e passes.
### <Feature description>
- [x] If this is non-trivial feature, paste the links/URLs to the design doc. — `docs/en/swip/SWIP-12.md`
- [x] Update the documentation to include this new feature. — two setup guides + `docs/menu.yml` entries + changelog
- [x] Tests(including UT, IT, E2E) are added to verify the new feature. — listener UT updates, two new e2e cases registered in CI
- [ ] If it's UI related, attach the screenshots below. — dashboards render against live metric data; will attach screenshots after CI e2e runs seed data into a preview env
- [ ] If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #<issue number>.
- [x] Update the [`CHANGES` log](https://github.com/apache/skywalking/blob/master/docs/en/changes/changes.md). — entries under `#### OAP Server` and `#### Documentation`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement SWIP-12: WeChat & Alipay Mini Program monitoring [skywalking]
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng merged PR #13835:
URL: https://github.com/apache/skywalking/pull/13835
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Implement SWIP-12: WeChat & Alipay Mini Program monitoring [skywalking]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #13835:
URL: https://github.com/apache/skywalking/pull/13835#discussion_r3110397125
##########
test/e2e-v2/cases/miniprogram/wechat/e2e.yaml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# WeChat Mini Program e2e test — drives OTLP metrics + OTLP error log
+# payloads via curl (iOS-style, no external sim image dependency) and
+# verifies service / metric / log observable under the WECHAT_MINI_PROGRAM
+# layer. Native-segment handling is covered by the layer-mapping unit
+# tests in RPCAnalysisListenerTest / EndpointDepFromCrossThreadAnalysisListenerTest.
+
+setup:
+ env: compose
+ file: docker-compose.yml
+ timeout: 20m
+ init-system-environment: ../../../script/env
+ steps:
+ - name: set PATH
+ command: export PATH=/tmp/skywalking-infra-e2e/bin:$PATH
+ - name: install yq
+ command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh yq
+ - name: install swctl
+ command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh swctl
+ - name: send WeChat Mini Program JS error log (OTLP)
+ command: |
+ TS_NS=$(date +%s)000000000
+ set -e
+ curl -sS -f --retry 30 --retry-delay 5 --retry-connrefused --retry-all-errors --max-time 10 \
+ -X POST -H 'Content-Type: application/json' \
+ http://${oap_host}:${oap_12800}/v1/logs \
+ -d "{\"resourceLogs\":[{\"resource\":{\"attributes\":[{\"key\":\"service.name\",\"value\":{\"stringValue\":\"wechat-mp-demo\"}},{\"key\":\"service.version\",\"value\":{\"stringValue\":\"v1.2.0\"}},{\"key\":\"service.instance.id\",\"value\":{\"stringValue\":\"v1.2.0\"}},{\"key\":\"miniprogram.platform\",\"value\":{\"stringValue\":\"wechat\"}}]},\"scopeLogs\":[{\"scope\":{\"name\":\"mini-program-monitor\"},\"logRecords\":[{\"timeUnixNano\":\"$TS_NS\",\"severityNumber\":17,\"severityText\":\"ERROR\",\"body\":{\"stringValue\":\"TypeError: Cannot read property 'foo' of undefined\"},\"attributes\":[{\"key\":\"exception.type\",\"value\":{\"stringValue\":\"js\"}},{\"key\":\"exception.stacktrace\",\"value\":{\"stringValue\":\"at pages/index/index.js:42\"}},{\"key\":\"miniprogram.page.path\",\"value\":{\"stringValue\":\"pages/index/index\"}}]}]}]}]}"
+
+trigger:
+ action: http
+ interval: 3s
+ times: 10
+ url: http://${oap_host}:${oap_12800}/v1/metrics
+ method: POST
+ headers:
+ Content-Type: application/json
+ body: '{"resourceMetrics":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"wechat-mp-demo"}},{"key":"service.version","value":{"stringValue":"v1.2.0"}},{"key":"service.instance.id","value":{"stringValue":"v1.2.0"}},{"key":"miniprogram.platform","value":{"stringValue":"wechat"}}]},"scopeMetrics":[{"scope":{"name":"mini-program-monitor"},"metrics":[{"name":"miniprogram.app_launch.duration","unit":"ms","gauge":{"dataPoints":[{"timeUnixNano":"0","asDouble":180.0,"attributes":[{"key":"miniprogram.page.path","value":{"stringValue":"pages/index/index"}}]}]}},{"name":"miniprogram.first_render.duration","unit":"ms","gauge":{"dataPoints":[{"timeUnixNano":"0","asDouble":95.0,"attributes":[{"key":"miniprogram.page.path","value":{"stringValue":"pages/index/index"}}]}]}}]}]}]}'
Review Comment:
The OTLP metrics payload sets `timeUnixNano` to `"0"`. SkyWalking’s OTLP metric receiver propagates this timestamp through MAL and uses it to compute the time bucket, so `0` will bucket the metric at epoch and it won’t show up in normal “recent” queries, making this e2e case flaky/incorrect. Populate `timeUnixNano` with the current time (e.g., `$(date +%s)000000000`) like the log case does, or omit the field if the receiver treats missing timestamps as “now”.
##########
oap-server/analyzer/agent-analyzer/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/listener/CommonAnalysisListener.java:
##########
@@ -20,20 +20,38 @@
import org.apache.skywalking.apm.network.language.agent.v3.SpanLayer;
import org.apache.skywalking.oap.server.core.analysis.Layer;
+import org.apache.skywalking.oap.server.core.config.IComponentLibraryCatalogService;
/**
* The common logic for specific listeners.
*/
abstract class CommonAnalysisListener {
+
+ private final int wechatMiniProgramComponentId;
+ private final int alipayMiniProgramComponentId;
+
+ protected CommonAnalysisListener(IComponentLibraryCatalogService componentLibraryCatalogService) {
+ this.wechatMiniProgramComponentId = componentLibraryCatalogService.getComponentId("WeChat-MiniProgram");
+ this.alipayMiniProgramComponentId = componentLibraryCatalogService.getComponentId("AliPay-MiniProgram");
+ }
+
/**
- * Identify the layer of span's service/instance owner. Such as ${@link Layer#FAAS} and ${@link Layer#GENERAL}.
+ * Identify the layer of span's service/instance owner. Such as ${@link Layer#FAAS} and ${@link Layer#GENERAL}.
Review Comment:
Javadoc uses `${@link ...}` which isn’t valid Javadoc syntax and won’t render as a link. Use `{@link Layer#FAAS}` / `{@link Layer#GENERAL}` instead.
##########
oap-server/analyzer/agent-analyzer/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/parser/listener/CommonAnalysisListener.java:
##########
@@ -20,20 +20,38 @@
import org.apache.skywalking.apm.network.language.agent.v3.SpanLayer;
import org.apache.skywalking.oap.server.core.analysis.Layer;
+import org.apache.skywalking.oap.server.core.config.IComponentLibraryCatalogService;
/**
* The common logic for specific listeners.
*/
abstract class CommonAnalysisListener {
+
+ private final int wechatMiniProgramComponentId;
+ private final int alipayMiniProgramComponentId;
+
+ protected CommonAnalysisListener(IComponentLibraryCatalogService componentLibraryCatalogService) {
+ this.wechatMiniProgramComponentId = componentLibraryCatalogService.getComponentId("WeChat-MiniProgram");
+ this.alipayMiniProgramComponentId = componentLibraryCatalogService.getComponentId("AliPay-MiniProgram");
+ }
+
Review Comment:
`IComponentLibraryCatalogService#getComponentId` unboxes the lookup result; if the component name isn’t present in `component-libraries.yml`, it throws a `NullPointerException`. Because this constructor now looks up hard-coded component names, a missing/renamed component entry would crash listener construction and break trace analysis. Consider guarding the lookup (e.g., catch `NullPointerException` and treat it as “unknown” with a sentinel id) so the fallback layer logic still works even if the catalog is customized.
##########
test/e2e-v2/cases/miniprogram/alipay/e2e.yaml:
##########
@@ -0,0 +1,74 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Alipay Mini Program e2e test — mirrors the WeChat case with
+# miniprogram.platform=alipay so the platform-gated filter on the
+# MAL / LAL rules routes to the ALIPAY_MINI_PROGRAM layer. WeChat-only
+# metrics (route / script / package_load / first_paint.time) are not
+# emitted and not verified.
+
+setup:
+ env: compose
+ file: docker-compose.yml
+ timeout: 20m
+ init-system-environment: ../../../script/env
+ steps:
+ - name: set PATH
+ command: export PATH=/tmp/skywalking-infra-e2e/bin:$PATH
+ - name: install yq
+ command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh yq
+ - name: install swctl
+ command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh swctl
+ - name: send Alipay Mini Program JS error log (OTLP)
+ command: |
+ TS_NS=$(date +%s)000000000
+ set -e
+ curl -sS -f --retry 30 --retry-delay 5 --retry-connrefused --retry-all-errors --max-time 10 \
+ -X POST -H 'Content-Type: application/json' \
+ http://${oap_host}:${oap_12800}/v1/logs \
+ -d "{\"resourceLogs\":[{\"resource\":{\"attributes\":[{\"key\":\"service.name\",\"value\":{\"stringValue\":\"alipay-mp-demo\"}},{\"key\":\"service.version\",\"value\":{\"stringValue\":\"v1.2.0\"}},{\"key\":\"service.instance.id\",\"value\":{\"stringValue\":\"v1.2.0\"}},{\"key\":\"miniprogram.platform\",\"value\":{\"stringValue\":\"alipay\"}}]},\"scopeLogs\":[{\"scope\":{\"name\":\"mini-program-monitor\"},\"logRecords\":[{\"timeUnixNano\":\"$TS_NS\",\"severityNumber\":17,\"severityText\":\"ERROR\",\"body\":{\"stringValue\":\"TypeError: Cannot read property 'foo' of undefined\"},\"attributes\":[{\"key\":\"exception.type\",\"value\":{\"stringValue\":\"js\"}},{\"key\":\"exception.stacktrace\",\"value\":{\"stringValue\":\"at pages/index/index.js:42\"}},{\"key\":\"miniprogram.page.path\",\"value\":{\"stringValue\":\"pages/index/index\"}}]}]}]}]}"
+
+trigger:
+ action: http
+ interval: 3s
+ times: 10
+ url: http://${oap_host}:${oap_12800}/v1/metrics
+ method: POST
+ headers:
+ Content-Type: application/json
+ body: '{"resourceMetrics":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"alipay-mp-demo"}},{"key":"service.version","value":{"stringValue":"v1.2.0"}},{"key":"service.instance.id","value":{"stringValue":"v1.2.0"}},{"key":"miniprogram.platform","value":{"stringValue":"alipay"}}]},"scopeMetrics":[{"scope":{"name":"mini-program-monitor"},"metrics":[{"name":"miniprogram.app_launch.duration","unit":"ms","gauge":{"dataPoints":[{"timeUnixNano":"0","asDouble":220.0,"attributes":[{"key":"miniprogram.page.path","value":{"stringValue":"pages/index/index"}}]}]}},{"name":"miniprogram.first_render.duration","unit":"ms","gauge":{"dataPoints":[{"timeUnixNano":"0","asDouble":110.0,"attributes":[{"key":"miniprogram.page.path","value":{"stringValue":"pages/index/index"}}]}]}}]}]}]}'
Review Comment:
The OTLP metrics payload sets `timeUnixNano` to `"0"`. The OTLP metric pipeline uses the datapoint timestamp to compute MAL time buckets; a zero timestamp buckets the metrics at epoch, so `metrics exec` won’t find them in a normal time range. Use a real current timestamp for the datapoints (or omit the field if supported) to make the e2e assertion reliable.
##########
oap-server/server-starter/src/main/resources/lal/miniprogram.yaml:
##########
@@ -0,0 +1,68 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# WeChat / Alipay Mini Program error-log processing.
+# One rule covers both runtimes — layer is decided script-side from the
+# miniprogram.platform resource attribute, using the SWIP-11 layer:auto
+# mechanism. Also emits a `miniprogram_error_count` counter sample for
+# every error log, picked up by log-mal-rules/miniprogram.yaml and
+# aggregated into per-platform `error_count` metrics.
+
+rules:
+ - name: miniprogram-errors
+ layer: auto
+ dsl: |
+ filter {
+ def platform = sourceAttribute("miniprogram.platform")
+ if (platform != "wechat" && platform != "alipay") {
+ abort {}
+ }
+ if (tag("exception.type") == null) {
Review Comment:
`tag("...")` in LAL resolves missing tags to an empty string (not `null`). As written, `if (tag("exception.type") == null)` will never abort, so non-error logs (or error logs missing `exception.type`) can still pass and emit `miniprogram_error_count` with an empty `exception_type` label. Check for empty string instead (or use an `isEmpty()`-style guard) before proceeding.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org