You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by wu...@apache.org on 2026/04/27 08:39:05 UTC

(skywalking) branch feature/runtime-rule-hot-update updated: docs(changes): rewrite Project section in user-perspective voice

This is an automated email from the ASF dual-hosted git repository.

wu-sheng pushed a commit to branch feature/runtime-rule-hot-update
in repository https://gitbox.apache.org/repos/asf/skywalking.git


The following commit(s) were added to refs/heads/feature/runtime-rule-hot-update by this push:
     new d75da8dff0 docs(changes): rewrite Project section in user-perspective voice
d75da8dff0 is described below

commit d75da8dff0651ceea1319f0bb9f251e1d8821d7d
Author: Wu Sheng <wu...@foxmail.com>
AuthorDate: Mon Apr 27 16:38:57 2026 +0800

    docs(changes): rewrite Project section in user-perspective voice
    
    Reframe the runtime-rule, schema-drift, and BanyanDB schema-fence entries to
    say what an operator can do now and what they get, rather than narrating the
    internal mechanism. The technical reference for endpoints, lifecycle states,
    self-heal, and split-brain handling already lives in
    docs/en/setup/backend/backend-runtime-rule-api.md and the design doc — the
    changelog now points at them instead of duplicating them.
---
 docs/en/changes/changes.md | 78 +++++++++++++++++++++++-----------------------
 1 file changed, 39 insertions(+), 39 deletions(-)

diff --git a/docs/en/changes/changes.md b/docs/en/changes/changes.md
index 8ccc3d4b74..7a3816d9ca 100644
--- a/docs/en/changes/changes.md
+++ b/docs/en/changes/changes.md
@@ -2,47 +2,47 @@
 
 #### Project
 
-* Runtime rule hot-update: change MAL and LAL rule files without restarting OAP. A new admin HTTP endpoint
-  (port `17128`, **disabled by default**) accepts `POST /runtime/rule/addOrUpdate?catalog=&name=` with the
-  rule file as the raw request body and persists it into the configured storage backend. Every node in the cluster
-  converges to the new content within one reconciler tick (default 30 s). Lifecycle is strictly
-  `ACTIVE → /inactivate → INACTIVE → /delete → absent`: `/inactivate` is the soft-pause path (broadcasts
-  `Suspend`, tears down OAP-internal state — handlers, prototypes, applied bookkeeping — but preserves the
-  BanyanDB measure and its data) and `/delete` is the destructive endpoint (requires the rule to already be
-  `INACTIVE`, returns `HTTP 409 requires_inactivate_first` otherwise; drops the backend measure, then removes
-  the tombstone row). Reactivation via `/addOrUpdate` against an `INACTIVE` row reuses the preserved measure
-  and is therefore lossless. `GET /runtime/rule?catalog=&name=` fetches a single rule's YAML (raw or JSON
-  envelope on `Accept: application/json`) with `ETag`/`If-None-Match` support; `GET /runtime/rule/bundled?catalog=`
-  lists every static (bundled) rule for the catalog with an `overridden` flag so UIs can render the runtime
-  + static merge. `GET /runtime/rule/list` reports the current state of every known rule (running,
-  suspended, skipped, overridden) in NDJSON; `GET /runtime/rule/dump` returns a tar.gz archive for backup / DR.
-  A bundled operator CLI at `tools/runtime-rule.sh` wraps the API. Cluster coordination uses a single-direction
-  `Suspend` RPC plus a paired `Resume` and a 60 s self-heal timer so aborted or crashed updates roll back
-  automatically; concurrent writes to different nodes during a brief topology flip are detected as
-  split-brain and return `HTTP 409 split_brain_detected`. DB-backed overrides take effect at the next OAP boot
-  via a classpath-discovered extension point consulted by the MAL and LAL engines, so restarts never regress to
-  the static file content. **Security notice:** the admin port has no authentication — operators must
-  gateway-protect it with IP allow-lists and never expose it to the public internet. See
-  `docs/en/setup/backend/backend-runtime-rule-api.md` for the full operator guide.
-* **Boot-time schema-drift handling changed.** When OAP boots and finds that the storage backend holds a metric
-  under the same name but a different shape than the current static / override rule declares, the metric is now
-  **skipped with an ERROR log line** (backend shape vs. declared shape, with remediation hint) and OAP continues to
-  boot. Previously this situation was silently tolerated on BanyanDB and JDBC (`ALREADY_EXISTS` was swallowed,
-  leading to runtime drift and lost samples) and caused a hard boot failure on Elasticsearch field-type conflicts.
-  All three backends now report the drift explicitly and leave the affected metric unregistered until the operator
-  reshapes it via `POST /runtime/rule/addOrUpdate` (the on-demand workflow, which is the only path that changes
-  backend schema). Skipped metrics are also surfaced in `GET /runtime/rule/list` under a new `bootSkippedReason`
-  field so operators can audit them without grepping logs.
+* **Runtime rule hot-update for MAL and LAL.** Operators can now ship metric (MAL) and log
+  (LAL) rule changes without restarting OAP. A push to a new admin endpoint persists the rule
+  to the configured storage backend, and every node in the cluster converges to the new
+  content within ~30 seconds. Common workflows:
+  * `addOrUpdate` — create or replace a rule. Body is the raw YAML you would normally ship in
+    `application.yml`'s rule directories. Returns 200 once the rule is applied locally and
+    persisted; peers pick it up on their next reconciler tick.
+  * `inactivate` — soft-pause a rule. The OAP stops emitting metrics for that rule but the
+    backend measure (and its history) is preserved, so a later `addOrUpdate` to the same
+    `(catalog, name)` is lossless. This is the safe way to take a rule offline.
+  * `delete` — destructive removal. Requires the rule to already be `INACTIVE` (otherwise
+    returns 409); drops the backend measure and removes the row.
+  * `get` / `bundled` / `list` / `dump` — read-side endpoints for fetching a single rule's
+    YAML (with `ETag` support), listing the static-vs-runtime overlay per catalog,
+    inspecting cluster-wide rule state in NDJSON, and exporting all rules as a tar.gz for
+    backup / DR.
+  Hot-updates survive OAP restart: at boot the MAL and LAL engines merge the disk baseline
+  with persisted runtime rules, so the cluster never silently regresses to the bundled
+  defaults. A bundled operator CLI at `tools/runtime-rule.sh` wraps the API for shell use.
+  **The endpoint is disabled by default and listens on port `17128` when enabled. It has
+  no built-in authentication — operators must gateway-protect it with IP allow-lists and
+  never expose it to the public internet.** Full setup, security guidance, and the
+  endpoint reference are in `docs/en/setup/backend/backend-runtime-rule-api.md`.
+* **Schema-drift is now visible at boot, not silent.** If the storage backend already holds
+  a metric whose shape doesn't match what the current rule declares (e.g., a rule was edited
+  on disk while OAP was offline), OAP now skips the affected metric, logs an ERROR with the
+  declared-vs-backend diff, and continues booting. Previously BanyanDB and JDBC silently
+  accepted the divergence (samples were quietly dropped) and Elasticsearch failed the boot
+  outright. Skipped metrics surface in `GET /runtime/rule/list` under a new
+  `bootSkippedReason` field so they're easy to audit. To re-shape a drifted metric, push the
+  desired YAML through `POST /runtime/rule/addOrUpdate` — that is the only path that
+  changes backend schema, and it runs through the verified hot-update flow described above.
 * Bump infra-e2e to testcontainers-go v0.42.0 (apache/skywalking-infra-e2e#146), which uses Docker Compose v2 plugin natively and removes docker-compose v1 dependency.
 * Remove deprecated `version` field from all docker-compose files for Compose v2 compatibility.
-* **BanyanDB schema-visibility fence.** Replaces the metadata-poll verify path with the new `SchemaBarrierService`
-  RPCs from `banyandb-client-proto` (`AwaitRevisionApplied` after Create / Update; `AwaitSchemaDeleted` after a
-  Delete that returned `mod_revision == 0`). Every install / drop now blocks until every BanyanDB data node has
-  caught up to the etcd `mod_revision` of the write, then returns. Combined with the runtime-rule
-  `localSuspend` / `localResume` bracket around an apply, this enforces a clean **t0..tn data boundary**:
-  samples written at or after the operator's `200 OK` use the new shape; samples written before use the old
-  shape. The previous "poll `findMeasure` until you can read your own write" loop is removed (registry-side
-  visibility never implied data-node visibility), as is `PostApplyVerifier`'s 5 s retry budget.
+* **Stronger consistency guarantee for BanyanDB schema changes.** Adopting the new
+  `SchemaBarrierService` API in BanyanDB, every schema install or drop driven by OAP now
+  waits until every BanyanDB data node has actually caught up to the change before
+  returning. For runtime hot-updates this means the operator's `200 OK` is the moment the
+  cluster's data boundary moves: samples written from that point on use the new shape, and
+  samples written before still go to the old shape — no overlap, no rejected writes during
+  the cutover.
 
 #### OAP Server
 * Add Zipkin Virtual GenAI e2e test. Use `zipkin_json` exporter to avoid protobuf dependency conflict