You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by GitBox <gi...@apache.org> on 2026/04/20 08:57:58 UTC
[GH] (rocketmq/fix/otel-1.44-reusable-pool-race): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git/fix/otel-1.44-reusable-pool-race has failed.
Run started by GitHub user Houlong66 (triggered by Houlong66).
Head commit for run:
cd983864741a0445d9e0ebbdeddadc02ad47d31a / 浩龙 <li...@alibaba-inc.com>
fix(metrics): mitigate OTel 1.44 OtlpGrpcMetricExporter pool race
OpenTelemetry Java 1.44.0 ~ 1.46.x ships OtlpGrpcMetricExporter with
MemoryMode.REUSABLE_DATA by default. The underlying
MetricReusableDataMarshaler.marshalerPool is a non-thread-safe
ArrayDeque accessed concurrently by the reader thread (poll) and the
OkHttp callback thread (add, via whenComplete). With
BatchSplittingMetricExporter issuing N concurrent sub-batch exports
per cycle, the pool races and leaks marshalers (~132 KiB each) until
OOM. Fixed upstream in 1.47.0 via open-telemetry/opentelemetry-java#7041
(ArrayDeque -> ConcurrentLinkedDeque).
- Bump OpenTelemetry to 1.47.0 in pom.xml so the upstream race fix is
in effect.
- Default OtlpGrpcMetricExporter to MemoryMode.IMMUTABLE_DATA to
preserve the pre-1.44 default behavior; exposed via
brokerConfig.metricsExportOtelMemoryMode ("IMMUTABLE_DATA" /
"REUSABLE_DATA", case-insensitive). Operators may opt in to
REUSABLE_DATA when running on OTel >= 1.47.
- Cap concurrent in-flight sub-batches in BatchSplittingMetricExporter
with a Semaphore controlled by
brokerConfig.metricsExportBatchMaxConcurrent (default 4; set to 1
to serialize and match pre-batch behavior; 0 or Integer.MAX_VALUE
means unlimited).
- Add brokerConfig.metricsExportBatchSplitEnabled (default true) as
an escape hatch to bypass BatchSplittingMetricExporter entirely,
restoring the raw OtlpGrpcMetricExporter wiring.
- Defensively snapshot MetricData points before export to avoid
ArrayIndexOutOfBoundsException in NumberDataPointMarshaler when
async instrument callbacks mutate point collections during export.
Report URL: https://github.com/apache/rocketmq/actions/runs/24657530322
With regards,
GitHub Actions via GitBox