[PR] fix(loki-logger): resolve log labels per request to avoid label leakage [apisix]

["shreemaan-abhishek (via GitHub)" <gi...@apache.org>]

shreemaan-abhishek opened a new pull request, #13562:
URL: https://github.com/apache/apisix/pull/13562

   ### Description
   
   When `loki-logger` is configured with a dynamic label value (e.g. a global rule with `"log_labels": {"job": "apisix", "service_name": "$service_name"}`), the label leaked across services: one service's `service_name` got pinned onto every other service's logs, and the value froze after the first request.
   
   Two defects combined:
   
   1. **The shared plugin `conf.log_labels` was mutated in place.** While resolving variables, the resolved literal was written back into `conf.log_labels`. After the first request `"$service_name"` became a literal, and `core.utils.resolve_var` returns `n_resolved == 0` for a literal, so it was never re-resolved again, freezing the value.
   
   2. **Labels were resolved once per batch processor and shared by all entries.** The resolution and the flush closure sat *after* the `batch_processor_manager:add_entry` early-return, so they ran only for the request that first created the processor. The closure built a single Loki stream and put every batched line into it under one label set.
   
   This is worst with a **global rule**, which has exactly one `conf` (the batch processor is keyed by `plugin.conf_version(conf)`), so one shared processor per worker froze the label for all subsequent requests from any service.
   
   ### Fix
   
   - **Clone** `conf.log_labels` before resolving, so the shared conf is never mutated.
   - **Resolve per request, before the early-return**, and stamp the result onto the entry (`entry.loki_labels`), mirroring the existing `entry.loki_log_time`. The entry is the only channel through the batch processor.
   - At flush time, **group entries into multiple Loki streams** keyed by their resolved label set, so each request is logged under its own labels. In Loki a stream is one unique label set, and the push payload supports multiple stream objects — the old code forced everything into one.
   - The grouping key sorts label names and joins `key=value` pairs with a NUL separator (avoids relying on JSON key ordering).
   - Internal fields (`loki_log_time`, `loki_labels`) are stripped from the entry before encoding the log line.
   
   No schema change; pure runtime behavior.
   
   ### Tests
   
   Added `t/plugin/loki-logger.t` TEST 17–20: one route with `log_labels.service = "$http_x_service_name"`, hit twice with different header values against the same conf, then assert Loki has a distinct stream per value and each line sits under its own label. Fails on the old frozen-label code, passes on the fix. These are integration tests needing a live Loki at `127.0.0.1:3100` (like the existing TEST 4/7/14).
   
   ### Checklist
   
   - [x] I have explained the need for this PR and the problem it solves
   - [x] I have explained the changes or the new features added to this PR
   - [x] I have added tests corresponding to this change
   - [x] I have updated the documentation to reflect this change (no doc change needed — the documented behavior was already correct; this fix makes the implementation match it)
   - [x] I have verified that the code passes lint (`luacheck`)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] fix(loki-logger): resolve log labels per request to avoid label leakage [apisix]

["shreemaan-abhishek (via GitHub)" <gi...@apache.org>]

shreemaan-abhishek merged PR #13562:
URL: https://github.com/apache/apisix/pull/13562


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org