You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Valentin Brückel <br...@predic8.de> on 2025/09/03 11:29:55 UTC
Propagating OpenTelemetry tracing data through Google PubSub Component
Hello everybody,
A customer of ours is planning to use Google PubSub for their Camel projects. As they are using OpenTelemetry,
they need to propagate tracing data through PubSub messages.
As including camel-google-pubsub-starter and camel-opentelemetry2-starter did not work out of the box and initially
attempted to propagate the "traceparent" Exchange header using interceptors. While they succeeded in attaching the
header to published PubSub messages and reading it back from consumed messages, they found that this had no effect on
OpenTelemetry and a new trace was created for each Exchange.
After a bit of digging, we found that OpenTelemetry hooks into RoutePolicySupport.onExchangeBegin() and
RoutePolicySupport.onExchangeDone() and any write access to the Exchange headers is too late in the Exchange's lifecycle
to have any effect.
Since we couldn't find a way to hook into the camel-google-pubsub component's behavior, our workaround was to clone the
component, adding a couple of lines to add handling for the "traceparent" header.
A POC is found at: https://github.com/bridgerdier/pubsub-trace-propagation
Actual changes:
* https://github.com/bridgerdier/pubsub-trace-propagation/blob/d69503510c0929cc98e7a5be9d0133ea202a2c6a/src/main/java/de/predic8/customers/pubsubtracepropagation/pubsubopentelemetry/CamelOpentelemetryMessagereceiver.java#L53
* https://github.com/bridgerdier/pubsub-trace-propagation/blob/d69503510c0929cc98e7a5be9d0133ea202a2c6a/src/main/java/de/predic8/customers/pubsubtracepropagation/pubsubopentelemetry/GooglePubsubOpentelemetryConsumer.java#L181
* https://github.com/bridgerdier/pubsub-trace-propagation/blob/d69503510c0929cc98e7a5be9d0133ea202a2c6a/src/main/java/de/predic8/customers/pubsubtracepropagation/pubsubopentelemetry/GooglePubsubOpentelemetryProducer.java#L75
Since this solution is intrusive and not really maintainable, we would be very grateful for a way to support using
OpenTelemetry with the google-pubsub component out-of-the-box. The preferred solution would probably run along the
lines of mapping all Exchange headers to Message attributes and vice versa.
https://cloud.google.com/pubsub/docs/publisher#using-attributes
I am however not very well versed in PubSub semantics or Camel/Tracing internals, so there might be a better way to
achieve the desired result.
Thanks in advance,
Valentin Brückel
[cid:0da6f921-4d0b-4fe9-aec3-993cac979ecb]
predic8 GmbH
Koblenzer Str. 65
53173 Bonn
predic8.de<https://www.predic8.de/>
Tel: +4<https://predic8-wiki.atlassian.net/wiki/spaces/inet/pages/372670466/Outlook#>9 (0) 228 5552576-0
Fax: +49 (0) 228 5552576-50
Amtsgericht Bonn
HRB 16152
Geschäftsführer: Thomas Bayer, Tobias Polley
Re: Propagating OpenTelemetry tracing data through Google PubSub Component
Posted by Pasquale Congiusti <pa...@gmail.com>.
Hi again,
I had a deeper look at the implementation and I compared it with the other
components. As you suggested in your first message, the most convenient
approach would be to map all headers, excluding those that we're certain
are not needed. This is the strategy applied to most of the components.
Once that is in place, any context propagation mechanism (traceparent in
otel, but also any other header used in different technologies) will work
out of the box.
I've tracked a Jira issue [1]. Feel free to pick it up and work to a final
implementation. If you're needing any support, don't hesitate to write a
comment on the same issue.
Regards,
Pasquale.
[1] https://issues.apache.org/jira/browse/CAMEL-22403
On Wed, Sep 3, 2025 at 1:42 PM Pasquale Congiusti <
pasquale.congiusti@gmail.com> wrote:
> Hi Valentin,
> thanks for providing all the info and working on a POC. I'll be giving a
> look during the next days in order to see how we can enhance the telemetry
> component. In general the components should be agnostic to any side feature
> like telemetry, but let's see what we can achieve starting from the POC
> you're done.
>
> Pasquale.
>
> On Wed, Sep 3, 2025 at 1:30 PM Valentin Brückel <br...@predic8.de>
> wrote:
>
>> Hello everybody,
>>
>> A customer of ours is planning to use Google PubSub for their Camel
>> projects. As they are using OpenTelemetry,
>> they need to propagate tracing data through PubSub messages.
>>
>> As including camel-google-pubsub-starter and camel-opentelemetry2-starter
>> did not work out of the box and initially
>> attempted to propagate the "traceparent" Exchange header using
>> interceptors. While they succeeded in attaching the
>> header to published PubSub messages and reading it back from consumed
>> messages, they found that this had no effect on
>> OpenTelemetry and a new trace was created for each Exchange.
>>
>> After a bit of digging, we found that OpenTelemetry hooks into
>> RoutePolicySupport.onExchangeBegin() and
>> RoutePolicySupport.onExchangeDone() and any write access to the Exchange
>> headers is too late in the Exchange's lifecycle
>> to have any effect.
>>
>> Since we couldn't find a way to hook into the camel-google-pubsub
>> component's behavior, our workaround was to clone the
>> component, adding a couple of lines to add handling for the "traceparent"
>> header.
>> A POC is found at:
>> https://github.com/bridgerdier/pubsub-trace-propagation
>>
>> Actual changes:
>> *
>> https://github.com/bridgerdier/pubsub-trace-propagation/blob/d69503510c0929cc98e7a5be9d0133ea202a2c6a/src/main/java/de/predic8/customers/pubsubtracepropagation/pubsubopentelemetry/CamelOpentelemetryMessagereceiver.java#L53
>> *
>> https://github.com/bridgerdier/pubsub-trace-propagation/blob/d69503510c0929cc98e7a5be9d0133ea202a2c6a/src/main/java/de/predic8/customers/pubsubtracepropagation/pubsubopentelemetry/GooglePubsubOpentelemetryConsumer.java#L181
>> *
>> https://github.com/bridgerdier/pubsub-trace-propagation/blob/d69503510c0929cc98e7a5be9d0133ea202a2c6a/src/main/java/de/predic8/customers/pubsubtracepropagation/pubsubopentelemetry/GooglePubsubOpentelemetryProducer.java#L75
>>
>> Since this solution is intrusive and not really maintainable, we would be
>> very grateful for a way to support using
>> OpenTelemetry with the google-pubsub component out-of-the-box. The
>> preferred solution would probably run along the
>> lines of mapping all Exchange headers to Message attributes and vice
>> versa.
>> https://cloud.google.com/pubsub/docs/publisher#using-attributes
>>
>> I am however not very well versed in PubSub semantics or Camel/Tracing
>> internals, so there might be a better way to
>> achieve the desired result.
>>
>> Thanks in advance,
>>
>> Valentin Brückel
>>
>>
>>
>> *predic8 GmbH*
>>
>> Koblenzer Str. 65
>>
>> 53173 Bonn
>>
>>
>>
>> predic8.de <https://www.predic8.de/>
>>
>> Tel: +4
>> <https://predic8-wiki.atlassian.net/wiki/spaces/inet/pages/372670466/Outlook#>9
>> (0) 228 5552576-0
>>
>> Fax: +49 (0) 228 5552576-50
>>
>>
>>
>> Amtsgericht Bonn
>>
>> HRB 16152
>>
>> Geschäftsführer: Thomas Bayer, Tobias Polley
>>
>
Re: Propagating OpenTelemetry tracing data through Google PubSub Component
Posted by Pasquale Congiusti <pa...@gmail.com>.
Hi Valentin,
thanks for providing all the info and working on a POC. I'll be giving a
look during the next days in order to see how we can enhance the telemetry
component. In general the components should be agnostic to any side feature
like telemetry, but let's see what we can achieve starting from the POC
you're done.
Pasquale.
On Wed, Sep 3, 2025 at 1:30 PM Valentin Brückel <br...@predic8.de> wrote:
> Hello everybody,
>
> A customer of ours is planning to use Google PubSub for their Camel
> projects. As they are using OpenTelemetry,
> they need to propagate tracing data through PubSub messages.
>
> As including camel-google-pubsub-starter and camel-opentelemetry2-starter
> did not work out of the box and initially
> attempted to propagate the "traceparent" Exchange header using
> interceptors. While they succeeded in attaching the
> header to published PubSub messages and reading it back from consumed
> messages, they found that this had no effect on
> OpenTelemetry and a new trace was created for each Exchange.
>
> After a bit of digging, we found that OpenTelemetry hooks into
> RoutePolicySupport.onExchangeBegin() and
> RoutePolicySupport.onExchangeDone() and any write access to the Exchange
> headers is too late in the Exchange's lifecycle
> to have any effect.
>
> Since we couldn't find a way to hook into the camel-google-pubsub
> component's behavior, our workaround was to clone the
> component, adding a couple of lines to add handling for the "traceparent"
> header.
> A POC is found at: https://github.com/bridgerdier/pubsub-trace-propagation
>
> Actual changes:
> *
> https://github.com/bridgerdier/pubsub-trace-propagation/blob/d69503510c0929cc98e7a5be9d0133ea202a2c6a/src/main/java/de/predic8/customers/pubsubtracepropagation/pubsubopentelemetry/CamelOpentelemetryMessagereceiver.java#L53
> *
> https://github.com/bridgerdier/pubsub-trace-propagation/blob/d69503510c0929cc98e7a5be9d0133ea202a2c6a/src/main/java/de/predic8/customers/pubsubtracepropagation/pubsubopentelemetry/GooglePubsubOpentelemetryConsumer.java#L181
> *
> https://github.com/bridgerdier/pubsub-trace-propagation/blob/d69503510c0929cc98e7a5be9d0133ea202a2c6a/src/main/java/de/predic8/customers/pubsubtracepropagation/pubsubopentelemetry/GooglePubsubOpentelemetryProducer.java#L75
>
> Since this solution is intrusive and not really maintainable, we would be
> very grateful for a way to support using
> OpenTelemetry with the google-pubsub component out-of-the-box. The
> preferred solution would probably run along the
> lines of mapping all Exchange headers to Message attributes and vice
> versa.
> https://cloud.google.com/pubsub/docs/publisher#using-attributes
>
> I am however not very well versed in PubSub semantics or Camel/Tracing
> internals, so there might be a better way to
> achieve the desired result.
>
> Thanks in advance,
>
> Valentin Brückel
>
>
>
> *predic8 GmbH*
>
> Koblenzer Str. 65
>
> 53173 Bonn
>
>
>
> predic8.de <https://www.predic8.de/>
>
> Tel: +4
> <https://predic8-wiki.atlassian.net/wiki/spaces/inet/pages/372670466/Outlook#>9
> (0) 228 5552576-0
>
> Fax: +49 (0) 228 5552576-50
>
>
>
> Amtsgericht Bonn
>
> HRB 16152
>
> Geschäftsführer: Thomas Bayer, Tobias Polley
>