You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "stankiewicz (via GitHub)" <gi...@apache.org> on 2026/10/09 09:45:47 UTC
[PR] [Solace] Add OpenTelemetry distributed tracing support to SolaceIO (#40175) [beam]
stankiewicz opened a new pull request, #40482:
URL: https://github.com/apache/beam/pull/40482
Fixes #40175
Supersedes #40187
### Summary
Adds OpenTelemetry distributed tracing support to both `SolaceIO.Read` and `SolaceIO.Write`, aligned with `KafkaIO` and `PubsubIO` and utilizing `Solace.Record` user properties introduced in #40108 for W3C Trace Context propagation (`traceparent`, `tracestate`).
### Changes
- **Build & Dependencies (`sdks/java/io/solace/build.gradle`)**:
- Added `opentelemetry_bom`, `opentelemetry_api`, and `opentelemetry_context` to `implementation`.
- Added `opentelemetry_sdk` and `opentelemetry_sdk_testing` to `testImplementation`.
- **Read-Side Trace Extraction (`SolaceIO.Read` & `OpenTelemetryHeaderConsumer`)**:
- Added `withEnableOpenTelemetryTracing()` on `SolaceIO.Read` (default `false`).
- Applied `ParDo.of(new OpenTelemetryHeaderConsumer<>())` in `SolaceIO.Read.expand()`.
- Initialized `Tracer` (`"SolaceIO"`) from `options.as(SdkHarnessOptions.class).getOpenTelemetry()`.
- Extracted W3C trace context (`traceparent`, `tracestate`, with case-insensitive header lookup) from `Solace.Record.getUserProperties()` via `W3CTraceContextPropagator.getInstance().extract(...)` and `TextMapGetter<Record>`.
- Created a `"SolaceIO.Read"` span with `SpanKind.CONSUMER` and `setParent(context)` wrapping `receiver.output(element)` so Beam propagates the OpenTelemetry context in `WindowedValue`.
- **Write-Side Trace Injection (`SolaceIO.Write` & `OpenTelemetryHeaderPropagator`)**:
- Added `withEnableOpenTelemetryTracing()` on `SolaceIO.Write` (default `false`).
- Added `toBuilder()` to `Solace.Record`.
- Applied `ParDo.of(new OpenTelemetryHeaderPropagator())` in `SolaceIO.Write.expand()`.
- Created a `"SolaceIO.Write"` span with `SpanKind.PRODUCER` and `setParent(Context.current())`, injecting the active W3C trace context into a copy of `Solace.Record`'s `userProperties` via `W3CTraceContextPropagator.getInstance().inject(...)` and `TextMapSetter<Record>`.
- **Unit Tests**:
- `SolaceIOReadTest`: Verified configuration toggle and `SpanKind.CONSUMER` span creation, parent trace/span ID linkage, case-insensitive `TraceParent` extraction, and `tracestate` preservation using `InMemorySpanExporter`.
- `SolaceIOWriteTest`: Verified configuration toggle, `SpanKind.PRODUCER` span creation for both `STREAMING` and `BATCHED` writers, W3C `traceparent` injection into `Solace.Record` user properties and JCSMP `SDTMap` message properties, and end-to-end consumer-to-producer trace context continuity.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Solace] Add OpenTelemetry distributed tracing support to SolaceIO (#40175) [beam]
Posted by "stankiewicz (via GitHub)" <gi...@apache.org>.
stankiewicz merged PR #40482:
URL: https://github.com/apache/beam/pull/40482
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Solace] Add OpenTelemetry distributed tracing support to SolaceIO (#40175) [beam]
Posted by "stankiewicz (via GitHub)" <gi...@apache.org>.
stankiewicz commented on code in PR #40482:
URL: https://github.com/apache/beam/pull/40482#discussion_r4262855726
##########
sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/SolaceIO.java:
##########
@@ -793,20 +824,32 @@ public PCollection<T> expand(PBegin input) {
Coder<T> coder = inferCoder(input.getPipeline(), configuration.getTypeDescriptor());
- return input.apply(
- org.apache.beam.sdk.io.Read.from(
- new UnboundedSolaceSource<>(
- initializedQueue,
- sempClientFactory,
- sessionServiceFactory,
- configuration.getMaxNumConnections(),
- configuration.getDeduplicateRecords(),
- coder,
- configuration.getTimestampFn(),
- configuration.getWatermarkIdleDurationThreshold(),
- configuration.getParseFn(),
- configuration.getAckDeadline(),
- configuration.getNackOnTimeout())));
+ PCollection<T> output =
+ input.apply(
+ org.apache.beam.sdk.io.Read.from(
+ new UnboundedSolaceSource<>(
+ initializedQueue,
+ sempClientFactory,
+ sessionServiceFactory,
+ configuration.getMaxNumConnections(),
+ configuration.getDeduplicateRecords(),
+ coder,
+ configuration.getTimestampFn(),
+ configuration.getWatermarkIdleDurationThreshold(),
+ configuration.getParseFn(),
+ configuration.getAckDeadline(),
+ configuration.getNackOnTimeout())));
+
+ if (configuration.getEnableOpenTelemetryTracing()) {
+ output =
Review Comment:
nice find! will do
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Solace] Add OpenTelemetry distributed tracing support to SolaceIO (#40175) [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #40482:
URL: https://github.com/apache/beam/pull/40482#issuecomment-6079296737
Assigning reviewers:
R: @chamikaramj for label java.
Note: If you would like to opt out of this review, comment `assign to next reviewer`.
Available commands:
- `stop reviewer notifications` - opt out of the automated review tooling
- `remind me after tests pass` - tag the comment author after tests pass
- `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
The PR bot will only process comments in the main thread (not review comments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Solace] Add OpenTelemetry distributed tracing support to SolaceIO (#40175) [beam]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #40482:
URL: https://github.com/apache/beam/pull/40482#issuecomment-6078668696
## [Codecov](https://app.codecov.io/gh/apache/beam/pull/40482?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 65.26%. Comparing base ([`76f6e6d`](https://app.codecov.io/gh/apache/beam/commit/76f6e6d7c0c1b2ce582c73d068ad9260cf15c5f3?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`878de01`](https://app.codecov.io/gh/apache/beam/commit/878de01ee62014fd042b9eefa3ac4eca0148d4dd?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 103 commits behind head on master.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #40482 +/- ##
============================================
+ Coverage 56.16% 65.26% +9.09%
- Complexity 2288 13917 +11629
============================================
Files 1124 1720 +596
Lines 178336 107826 -70510
Branches 1489 11243 +9754
============================================
- Hits 100170 70369 -29801
+ Misses 75645 33270 -42375
- Partials 2521 4187 +1666
```
| [Flag](https://app.codecov.io/gh/apache/beam/pull/40482/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [go](https://app.codecov.io/gh/apache/beam/pull/40482/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java](https://app.codecov.io/gh/apache/beam/pull/40482/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `65.26% <ø> (-10.53%)` | :arrow_down: |
| [python](https://app.codecov.io/gh/apache/beam/pull/40482/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/beam/pull/40482?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Solace] Add OpenTelemetry distributed tracing support to SolaceIO (#40175) [beam]
Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on code in PR #40482:
URL: https://github.com/apache/beam/pull/40482#discussion_r4262431732
##########
sdks/java/io/solace/src/main/java/org/apache/beam/sdk/io/solace/SolaceIO.java:
##########
@@ -793,20 +824,32 @@ public PCollection<T> expand(PBegin input) {
Coder<T> coder = inferCoder(input.getPipeline(), configuration.getTypeDescriptor());
- return input.apply(
- org.apache.beam.sdk.io.Read.from(
- new UnboundedSolaceSource<>(
- initializedQueue,
- sempClientFactory,
- sessionServiceFactory,
- configuration.getMaxNumConnections(),
- configuration.getDeduplicateRecords(),
- coder,
- configuration.getTimestampFn(),
- configuration.getWatermarkIdleDurationThreshold(),
- configuration.getParseFn(),
- configuration.getAckDeadline(),
- configuration.getNackOnTimeout())));
+ PCollection<T> output =
+ input.apply(
+ org.apache.beam.sdk.io.Read.from(
+ new UnboundedSolaceSource<>(
+ initializedQueue,
+ sempClientFactory,
+ sessionServiceFactory,
+ configuration.getMaxNumConnections(),
+ configuration.getDeduplicateRecords(),
+ coder,
+ configuration.getTimestampFn(),
+ configuration.getWatermarkIdleDurationThreshold(),
+ configuration.getParseFn(),
+ configuration.getAckDeadline(),
+ configuration.getNackOnTimeout())));
+
+ if (configuration.getEnableOpenTelemetryTracing()) {
+ output =
Review Comment:
We expect `T` to be of type `Solace.Record.class` right ?
If so should we add a check here for that to prevent unintended issues in the reader ?
Also add a test pls.
AI suggestion:
```
if (configuration.getEnableOpenTelemetryTracing()) {
checkArgument(
TypeDescriptor.of(Solace.Record.class).isSupertypeOf(configuration.getTypeDescriptor()),
"SolaceIO.Read: withEnableOpenTelemetryTracing() requires Solace.Record output "
+ "(SolaceIO.read()); user properties are not available after a custom parseFn.");
// ... apply OpenTelemetryHeaderConsumer as today
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org